Repository navigation

[TRTLLM-12891][perf] Integrate Mooncake connector into KVCMv2 - #19171

Open

brb-nv wants to merge 42 commits into

NVIDIA:mainfrom

brb-nv:user/brb/m3-mooncake-store-main

Commits on Oct 6, 2026

[TRTLLM-12891][feat] Mooncake store part 2: connector worker, scheduler, and LlmArgs surface
brb-nv
committed
86e7654
[None][chore] Address comments from CodeRabbit
brb-nv
committed
2601012
[None][fix] Address comments from zhaoyuanh-nvidia
brb-nv
committed
0d7f01a
[None][chore] Address comments from CodeRabbit
brb-nv
committed
a7e9152
[None][feat] BREAKING: Refine Mooncake connector API
brb-nv
committed
b8320c6
[None][fix] Address comments from CodeRabbit
brb-nv
committed
23dc810
[None][fix] Drop the CMake-generated mooncake Python package
brb-nv
committed
394a16d
[None][fix] Address comments from nv-xtf
brb-nv
committed
9042c48
[None][fix] Address comments from eopXD
brb-nv
committed
c307456
[None][infra] Update CI container images to include install_mooncake.sh change
brb-nv
committed
592c88a
clean up
brb-nv
committed
9866f70
[None][chore] Drop low-value Mooncake store test coverage
brb-nv
committed
0694206
[None][test] Add Mooncake store tests against a live pool
brb-nv
committed
292b6fa
[None][test] Fix small-cache test
brb-nv
committed
ff38630
[None][chore] Trim the Mooncake slurm benchmark scripts to the pool lifecycle
brb-nv
committed
09a81c4
[None][chore] Export only the Mooncake connector names reached from outside
brb-nv
committed
8f2d5be
[None][chore] Fold the duplicate mooncake-store check in the executor creator
brb-nv
committed
62f4cf9
[None][chore] Revert comment rewording that carries no logic change
brb-nv
committed
3c449aa
[None][fix] Address comments from Mgluhovskoi
brb-nv
committed
1ec10ee
[None][fix] Address comments from chienchunhung
brb-nv
committed
d41a2d4
[None][chore] Drop test-list comments and tighten the prose
brb-nv
committed
0c2db73
[None][fix] Keep Mooncake pool capacity alive across a worker's lifetime
brb-nv
committed
5fc209a
formatting
brb-nv
committed
2ec0c4c
[None][test] Give the connector test doubles the attributes the executor now reads
brb-nv
committed
36c1af3
[None][test] Give the preemption test stubs the state the KV cache manager now keeps
brb-nv
committed
9c49022
[None][fix] Let a connector worker release what it holds at shutdown
brb-nv
committed
87ef412
[None][chore] Prune defensive getattr and consolidate connector tests
brb-nv
committed
3832edc
Settle mooncake-store effective settings in the LLM constructor
brb-nv
committed
a257b55
Bypass mooncake-store for requests carrying a LoRA adapter
brb-nv
committed
0a8ab71
Keep a cancelled request's KV pages pinned while a connector save reads them
brb-nv
committed
400c1da
Rename the stale connector save references in the contract test
brb-nv
committed
a337bc2
Cover a cancelled request's held pages end to end
brb-nv
committed
77abb72
Let a capacity-only role keep its native cache tiers
brb-nv
committed
817f05c
Keep a capacity-only connector off the per-request path
brb-nv
committed
1623c0c
Offer a request its connector save at most once
brb-nv
committed
dfb9c2a
Drop a mooncake-store save that fails instead of failing the rank
brb-nv
committed
f519c88
Derive a rank's pool address from the route to the master
brb-nv
committed
e26a42c
Load the mooncake-store settings only for that connector
brb-nv
committed
192d59e
Document the mooncake-store connector and the optional worker hooks
brb-nv
committed
5fa5382

Commits on Oct 7, 2026

Trim the connector prose and name the override pair
brb-nv
committed
cccfe35
Take a capacity-only connector off every per-request path
brb-nv
committed
f8e232c
fix tests
brb-nv
committed
466d350