Skip to content

Implemented per-collection TTL property [CBL-9299] - #2585

Open
snej wants to merge 3 commits into
release/3.4from
feature/collection-ttl-CBL-9299
Open

snej wants to merge 3 commits into
release/3.4from
feature/collection-ttl-CBL-9299

Conversation

@snej

@snej snej commented Nov 13, 2026

Copy link
Copy Markdown
Collaborator

Implemented CBL-9299.

  • Added ttl column to kvmeta table, with corresponding schema bump
  • Accessors for TTL on C4Collection, KeyStore, etc.
  • SQLiteKeyStore fetches TTL and lastSequence in a single query, to avoid an extra query per save.
  • SQLiteKeyStore::set sets the expiration based on the TTL, if any.
  • CollectionImpl::documentSaved checks if there's a TTL, and if so notifies the Housekeeper (just like setExpiration does.)

- Added `ttl` column to `kvmeta` table, with corresponding schema bump
- Accessors for TTL on C4Collection, KeyStore, etc.
- SQLiteKeyStore fetches TTL and lastSequence in a single query,
  to avoid an extra query per save.
- `SQLiteKeyStore::set` sets the expiration based on the TTL, if any.
- `CollectionImpl::documentSaved` checks if there's a TTL, and if so
  notifies the Housekeeper (just like setExpiration does.)
@snej
snej requested a review from a team November 13, 2026 00:26
@github-actions

github-actions Bot commented Nov 13, 2026 •

Copy link
Copy Markdown

This is a release branch and commits are restricted.

Please confirm this PR is one of the following:

  • A response to a customer ask
  • A change per our security policy
  • A non-functional change (e.g. changes needed for building an older version)
  • A bug fix for a x.y.0 release
  • A change that has been granted an exception (please comment)

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Unresolved defects break reads on older read-only schemas, prevent TTL renewal on some saves, and carry deleted collections’ TTL settings into recreated collections.

6 open issues
What changed in this PR

Adds per-collection time-to-live (TTL) support to LiteCore, connecting collection configuration to SQLite-backed document expiration and background cleanup.

Changes:

  • Persists TTL in collection metadata with a schema upgrade and combined metadata reads.
  • Exposes TTL through C, C++, and storage APIs.
  • Applies expiration during saves, notifies housekeeping, and adds persistence and expiration tests.
File Description
LiteCore/​Storage/​SQLiteKeyStore.hh Declares TTL accessors and cached state.
LiteCore/​Storage/​SQLiteKeyStore.cc Reads, caches, and applies TTL during writes.
LiteCore/​Storage/​SQLiteDataFile.hh Adds combined metadata and the TTL schema version.
LiteCore/​Storage/​SQLiteDataFile.cc Creates, migrates, reads, and writes TTL metadata.
LiteCore/​Storage/​KeyStore.hh Defines the storage-level TTL interface.
LiteCore/​Storage/​BothKeyStore.hh Delegates TTL access to the live store.
LiteCore/​Database/​CollectionImpl.hh Exposes TTL and connects saves to housekeeping.
C/​tests/​c4DatabaseTest.cc Tests TTL persistence and automatic expiration.
C/​scripts/​c4.txt Adds TTL functions to the export list.
C/​include/​c4Collection.h Declares and documents public C TTL APIs.
C/​include/​c4Base.h Defines millisecond intervals and timestamp addition.
C/​Cpp_include/​c4Collection.hh Adds C++ collection TTL methods.
C/​c4CAPI.cc Implements C wrappers for TTL access.

Review effort: Balanced

Comment thread LiteCore/Storage/SQLiteDataFile.cc Resolved
Comment thread LiteCore/Storage/SQLiteDataFile.cc Resolved
Comment thread LiteCore/Storage/SQLiteKeyStore.cc Resolved
Comment thread LiteCore/Database/CollectionImpl.hh Resolved
Comment thread LiteCore/Database/CollectionImpl.hh Resolved
Comment thread LiteCore/Storage/SQLiteKeyStore.cc Resolved
@cbl-bot

cbl-bot commented Nov 13, 2026 •

Copy link
Copy Markdown

Code Coverage Results:

Type Percentage
branches 64.62
functions 77.45
instantiations 71.1
lines 76.05
regions 72.26

- Don't persist TTL across collection deletion + recreation. Deleting
  a collection should delete its row in kvmeta.
- Saving a doc when no new sequence is created (i.e. by replicator)
  should also apply the TTL.
- Stopped BothKeyStore from overwriting the expiration set by TTL.
- Prevent SQLite error in getKVMeta() when the database hasn't been
  upgraded so kvmeta doesn't have a ttl column.
- Prevent caller setting ridiculously large TTL that would overflow
  the expiration_t.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Resetting collection sequences while retaining replication checkpoints can cause push replication to skip new documents when a collection is recreated.

3 open issues
1 previously missed issue

In code that hasn't changed since last review

Medium severity Forward TTL expiration notifications to a housekeeping connection

LiteCore/​Database/​CollectionImpl.hh:338

openAgain() disables housekeeping on replication's write connection, so this notification cannot start expiry there and is not forwarded to the primary connection. If the primary collection has no active expiry timer, pulled documents receive expiration timestamps but remain unpurged until reopening or another operation starts expiry. Forward TTL-expiration notifications after commit to a housekeeping-enabled connection, starting its expiry task if needed.

5 resolved since last review

Review effort: Balanced

Comment thread LiteCore/Storage/SQLiteKeyStore.cc Resolved
Comment thread LiteCore/Storage/SQLiteKeyStore.cc Resolved
- Don't persist TTL across collection deletion + recreation. Deleting
  a collection should delete its row in kvmeta.
- Saving a doc when no new sequence is created (i.e. by replicator)
  should also apply the TTL.
- Stopped BothKeyStore from overwriting the expiration set by TTL.
- Prevent SQLite error in getKVMeta() when the database hasn't been
  upgraded so kvmeta doesn't have a ttl column.
- Prevent caller setting ridiculously large TTL that would overflow
  the expiration_t.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

An unconditional null housekeeper dereference causes TTL-enabled saves and positive expiration changes to crash on reopened connections, including those used by replication.

1 open issue
3 resolved since last review

Review effort: Balanced

void documentExpirationChanged(C4Timestamp expiration) {
if ( expiration > C4Timestamp::None ) {
if ( !_expiryStarted ) startHousekeeping(HousekeeperTask::Expiry);
_housekeeper->documentExpirationChanged(expiration);

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants