Describe the bug
Collections keep the keys of deleted rows in CollectionStateManager.syncedKeys, causing the set—and memory usage—to grow over time.
Every synced operation currently adds its key to syncedKeys:
|
for (const operation of transaction.operations) { |
|
const key = operation.key as TKey |
|
this.syncedKeys.add(key) |
|
|
When processing a delete, TanStack DB removes the row from syncedData and the other per-row tracking structures, but does not remove it from syncedKeys:
|
case `delete`: |
|
this.syncedData.delete(key) |
|
this.syncedMetadata.delete(key) |
|
// Clean up origin and pending tracking for deleted rows |
|
this.rowOrigins.delete(key) |
|
this.pendingLocalChanges.delete(key) |
|
this.pendingLocalOrigins.delete(key) |
|
this.pendingOptimisticUpserts.delete(key) |
|
this.pendingOptimisticDeletes.delete(key) |
|
this.pendingOptimisticDirectUpserts.delete(key) |
|
this.pendingOptimisticDirectDeletes.delete(key) |
|
break |
This means a long-running collection that receives and deletes many uniquely keyed rows can retain every key it has processed, even when the collection itself is empty.
I reproduced this with @tanstack/db 0.8.6 and the current main branch at commit bda2303a339a8e16d2f861775c448149ef7e986f.
To reproduce
Create a collection, sync 50,000 uniquely keyed rows, and then sync-delete all of them:
import { createCollection } from '@tanstack/db'
let controls
const collection = createCollection({
id: 'synced-keys-reproduction',
getKey: (row) => row.id,
startSync: true,
sync: {
sync: (params) => {
controls = params
},
},
})
controls.begin()
for (let id = 0; id < 50_000; id++) {
controls.write({
type: 'insert',
value: { id },
})
}
controls.commit()
controls.markReady()
await collection.stateWhenReady()
console.log('after insert', {
rows: collection.state.size,
syncedData: collection._state.syncedData.size,
syncedKeys: collection._state.syncedKeys.size,
})
controls.begin()
for (let id = 0; id < 50_000; id++) {
controls.write({
type: 'delete',
key: id,
})
}
controls.commit()
console.log('after delete', {
rows: collection.state.size,
syncedData: collection._state.syncedData.size,
syncedKeys: collection._state.syncedKeys.size,
})
Actual result:
after insert { rows: 50000, syncedData: 50000, syncedKeys: 50000 }
after delete { rows: 0, syncedData: 0, syncedKeys: 50000 }
The problem can also be reproduced with only two rows:
- Sync-insert keys
1 and 2.
- Sync-delete key
1.
syncedKeys still contains {1, 2} instead of only {2}.
Expected behavior
A synced delete should remove that row's key from syncedKeys, just as it removes the row from syncedData and the other per-row tracking structures.
After deleting all 50,000 rows, the result should be:
after delete { rows: 0, syncedData: 0, syncedKeys: 0 }
Screenshots
Current behavior
The collection contains no rows after the deletes, but syncedKeys still contains all 50,000 keys.
Behavior with the proposed fix
After adding this.syncedKeys.delete(key) to the delete branch, syncedKeys returns to zero.
Chrome heap snapshot
Chrome identifies the retained set directly as:
syncedKeys in CollectionStateManager
In this isolated 50,000-key reproduction:
- The selected
Set retained approximately 1.8 MB.
- Its
CollectionStateManager retained approximately 3.9 MB in total.
- The retainer chain continued through the collection's
_state.
Desktop
- OS: macOS
- Browser: Chrome
- Package version:
@tanstack/db 0.8.6
- Tested against
main: bda2303a339a8e16d2f861775c448149ef7e986f
Smartphone
Not applicable.
Additional context
A minimal fix is to remove the key in the synced delete branch:
case `delete`:
this.syncedKeys.delete(key)
this.syncedData.delete(key)
// ...
I also prepared a focused regression test that:
- Sync-inserts keys
1 and 2.
- Confirms
syncedKeys contains both keys.
- Sync-deletes key
1.
- Confirms
syncedKeys contains only key 2.
The test fails against the current implementation and passes with the proposed fix.
I can submit the fix, regression test, and patch changeset as a PR.
Describe the bug
Collections keep the keys of deleted rows in
CollectionStateManager.syncedKeys, causing the set—and memory usage—to grow over time.Every synced operation currently adds its key to
syncedKeys:db/packages/db/src/collection/state.ts
Lines 1015 to 1018 in bda2303
When processing a delete, TanStack DB removes the row from
syncedDataand the other per-row tracking structures, but does not remove it fromsyncedKeys:db/packages/db/src/collection/state.ts
Lines 1063 to 1074 in bda2303
This means a long-running collection that receives and deletes many uniquely keyed rows can retain every key it has processed, even when the collection itself is empty.
I reproduced this with
@tanstack/db0.8.6 and the currentmainbranch at commitbda2303a339a8e16d2f861775c448149ef7e986f.To reproduce
Create a collection, sync 50,000 uniquely keyed rows, and then sync-delete all of them:
Actual result:
The problem can also be reproduced with only two rows:
1and2.1.syncedKeysstill contains{1, 2}instead of only{2}.Expected behavior
A synced delete should remove that row's key from
syncedKeys, just as it removes the row fromsyncedDataand the other per-row tracking structures.After deleting all 50,000 rows, the result should be:
Screenshots
Current behavior
The collection contains no rows after the deletes, but
syncedKeysstill contains all 50,000 keys.Behavior with the proposed fix
After adding
this.syncedKeys.delete(key)to the delete branch,syncedKeysreturns to zero.Chrome heap snapshot
Chrome identifies the retained set directly as:
In this isolated 50,000-key reproduction:
Setretained approximately 1.8 MB.CollectionStateManagerretained approximately 3.9 MB in total._state.Desktop
@tanstack/db0.8.6main:bda2303a339a8e16d2f861775c448149ef7e986fSmartphone
Not applicable.
Additional context
A minimal fix is to remove the key in the synced delete branch:
I also prepared a focused regression test that:
1and2.syncedKeyscontains both keys.1.syncedKeyscontains only key2.The test fails against the current implementation and passes with the proposed fix.
I can submit the fix, regression test, and patch changeset as a PR.