Repository navigation
[NIDX-02 workstream] Replace the Property shard index #14007
Description
Activity
- addeddatabaseBanyanDB - SkyWalking native databaseBanyanDB - SkyWalking native database
on Aug 25, 2026 - changed the title
[-][NIDX-02] Replace the Property shard index with the native implementation[/-][+][NIDX-02 workstream] Replace the Property shard index[/+]on Aug 25, 2026 Thanks for raising the writer-ownership and protected-snapshot concerns.
I checked the current Property shard model. There is no persisted per-shard metadata identifying the implementation that created or owns it: the shard directory provides only the group and shard ID, while ICE/snapshot metadata remains implementation-neutral so both implementations can reopen the same files during the rollback window. The native index will also neither create nor require an index-local PID or lock file; those runtime files are outside the compatibility format.
I updated the issue body to make the cutover contract explicit. NIDX-02 now requires the Property database to establish exclusive ownership before scanning or opening shard stores, resolve one Property-wide implementation selection, and make
property/db.newSharduse only that selection for every existing and new shard. Rollback must stop admission, drain callbacks, close all writers, change the selection, and reopen the same files. Simultaneous native and legacy writers are prohibited.The merge/expiry/GC part was already covered by the design: merge operates from a pinned source snapshot, expiry clones deletion state, replacement publication precedes collection, and a file remains protected while referenced by a retained manifest, active reader, or backup. CRC32 remains present only for layout compatibility and is not calculated, validated, or used for generation selection or GC.
#14002 is closed and the native reader is on
mainat31b32ca2, so this workstream is decomposed.Ordered TDD leaves
- #14073 — NIDX-02A: encode Property documents as native ICE v3 segment bytes
- #14074 — NIDX-02B: drive the native encoder through the segment plugin contract, blocked by [NIDX-02A] Encode Property documents as native ICE v3 segment bytes #14073
- #14075 — NIDX-02C: cut the Property database over to the native writer, blocked by [NIDX-02B] Drive the native encoder through the segment plugin contract #14074
Only the oldest unblocked open leaf may carry
Backlog. Do not label this parent or multiple leaves. An open pull request does not unblock its successor; the preceding leaf must merge first.Two notes on the split
The writer is separated from the cutover on purpose. A writer-only ticket would normally be format-only and therefore forbidden, but
nativeice.Openreads a shard directory directly and the package imports no retired dependency, so #14073 has two real consumers already onmain— the merged reader and the pinned compatibility reader. That makes it vertical without touchingnewShard. The dangerous diff, where native bytes first go under a live Property writer, is isolated in #14075.The dependency-policy question is isolated in #14075. Registration through the aliased index package is the only place in the ladder that cannot avoid the retired token in an added line; #14073 and #14074 reach everything they need through the neutral
segmentalias. So the exemption decision gates only the last leaf and does not hold up the encoder.#14074 is provisional. Its consumer is the index library's lifecycle rather than a live Property caller. If that is judged not vertical, fold it into #14075 and close it as superseded; that decision should be made before it becomes the oldest unblocked leaf.
[NIDX-02 workstream] Replace the Property shard index
Parent: #13990
Blocked by: #14002
End-state boundary
banyand/property/db.newShardselects the native implementation for existing and new Property shards. The completed workstream owns compatible Property writes, exact/boolean/range queries, explicit sort, callbacks, snapshot publication/recovery, merge-time expiry, reference-safe GC, repair integration, backup, and same-file rollback.The retained legacy Property constructor is an explicit rollback choice during the compatibility window, not a second active writer.
Cutover and writer ownership
The implementation choice is Property-database-wide during NIDX-02, not inferred independently for each shard.
property/db.newSharduses that selection for every existing and newly created shard.Required behavior
Just-in-time decomposition gate
When #14002 closes:
Backlogonly to the first unblocked leaf.Do not pre-file format-only writer, snapshot, merge, or GC tickets. Each leaf must activate a live Property behavior in its merge.
Completion criteria
All future leaves merge; the complete
property/db.newShardrole is native; the two-binary compatibility and crash matrices pass; and the workstream closes before NIDX-03 decomposition begins.Lexical non-regression
Repository changes for every implementation leaf before the final removal may delete existing references but must add zero new case-insensitive
blugetokens and zero matching tracked paths. The gate includes imports and aliases, function/type/variable names, filenames and runtime names, strings, comments and messages, tests, fixture/provenance data, scripts, configuration, and generated assets. Compatibility evidence uses neutrallegacy oracleorcompatibility writerlabels plus an immutable revision or content hash rather than adding a retired module name.This lexical gate applies to repository changes, not to issue or archived-design prose that names the dependency in order to specify its removal.
Design
BDB-NIDX-SPEC-001 revision 0.2 — NIDX-02 behavior