Skip to content

feat: remote files parity (fetch-on-miss, peer Files index, precache, Library) - #3872

Open
damianvtran wants to merge 22 commits into
mainfrom
feat/remote-files-parity
Open

damianvtran wants to merge 22 commits into
mainfrom
feat/remote-files-parity

Conversation

@damianvtran

@damianvtran damianvtran commented Nov 12, 2026 •

Copy link
Copy Markdown
Owner

Core half of remote files parity (design: docs/design/remote-files-parity.md, §7 "Core PR", C1–C6). UI half: damianvtran/local-operator-ui#1880.

Operator request (verbatim)

"I noticed I can't see images and artifacts on remote nodes, can you have a session work on this remotely and test it, make sure that it works. We should be able to preview assets (sounds, music, images, video, etc.) from the local device and also see those show up in the canvas and ideally the library as well, these should be indexed too in order to be performant and not slow down the library view. It might make sense to cache or precache assets on active sessions so that clicking into them is fast and we can see the previews, but make sure it doesn't blow up RAM."
"Right now it just says remote files can't be opened on this device but we need to fix that and make it so that remote files show up just like local files maybe just with a small non-intrusive indication somewhere that they're remote but they should show up under the same classified sticky headers, sections, etc. and we should be able to download them to local, and do other file actions."

Commits

  1. 1af12f60e perf(network): memoise and stream the referenced-digest scan (C1): referenced_attachments_in scans in 4 MiB windows (64-byte overlap) instead of read_bytes(), memoised on the journal's identity, so one 32 MiB pull scans the owner transcript once instead of once per chunk.
  2. cb746abbf feat(desktop): fetch peer attachments on miss (C2): attachment / waveform / supplement document / child-route bytes of a peer session resolve: local store → replica (G1) → bounded remote cache → ONE single-flight pull from the owner (S1 net_file_get, land="cache"). Streamed from an fd with Range/206 (server/utils/byte_range.py); 504 retryable (attachment_owner_unreachable, attachment_fetch_pending + Retry-After), 409 permanent (attachment_peer_outdated, attachment_on_peer). New remote_files/{cache,fetch}.py, LRU eviction under flock, config key desktop.remote_files_cache_bytes.
  3. fc4d0aa96 refactor(session-files): move family discovery out of the desktop module (C3): pure move into session_files/family.py (re-exported under the same names) + family_for/family_blocking, so the relay never imports server.utils.desktop_sessions/fastapi.
  4. 0614e8aa5 feat(network): net_session_files peer index (C4): owner-answered Files and user-attachment index (view capability, mesh-file-index-v1), display paths only, per-row remote_read/remote_unreadable, page trimmed to one link record; viewer …/files, …/user-attachments answer peers with remote, origin_device; new peer-only GET …/files/{id}/content and POST …/files/{id}/download; features.remote_files = 1; DESKTOP_API "Remote files".
  5. 2655dbaf7 feat(desktop): precache active remote sessions (C5).
  6. ab6f4a92b feat(library): index remote sessions' files (C6): schema v3, identity.remote_key, library/remote.py feeder, LibraryRow.remote.
  7. 9a6ef5661 docs(design): record where the core remote-files PR differs from the plan (doc only).
  8. 14d8601fd fix(remote-files): round-1 remediation for #3872 (#issuecomment-6831178771).
  9. e3ac0f2ae fix(library): serve cached remote assets through the remote content path (QA r2 Q2-1/Q2-2/Q2-3).
  10. 0cd05c51e test(i18n): raise the eager wire ceiling for the remote-files keys (80 KiB; measured basis in the test).
  11. 5d7e1a5e4 chore(i18n): bind the remote-files audit rows (14 rows, evidence, bundled.gen.ts back to base).
  12. 3a842e70b fix(remote-files): serve a peer's cached bytes with its owner offline, sandbox them on the phone (round-3 R3-1..R3-4, Q3-1/Q3-2, #issuecomment-6833147857). Digest routes (Library open, transcript attachment/waveform/document/child) check this device's store → replica → cache FIRST (local_peer_blob), gated by the Library index's recorded origin device as the existence proof; the owner's row only on a miss; miss + silent owner → 504 attachment_owner_unreachable. Mobile /content and /thumbnail plus the desktop thumbnail apply servable_type + sandbox to remote refs.

Wire shapes match the UI PR's assumptions: content 504 retryable / 409 (session_files_remote_unreadable, attachment_on_peer) / 404 permanent; download envelope {path,name,bytes,sha256,created,origin_device}; origin_device = {id,name}; every remote row carries both remote_read and remote_unreadable; LibraryRow.remote = {device_id, device_name} with ref.path: null.

Acceptance → test

All on the REAL two-relay loopback (mesh fixture from test_mesh_file_pull.py / _pair), B's desktop routes on an ASGI app over B's root.

  • C1: test_remote_files_scan.py::test_a_32_mib_pull_scans_the_owner_transcript_once, …straddling_a_window_boundary…, …memo_follows_an_append_and_is_bounded
  • (a) G2 png/wav/mp4 into cache, not store → test_remote_files_fetch.py::test_a_…
  • (b) replica-held blob, zero relay calls → test_b_…
  • (c) Range → 206 → test_c_…
  • (d) unreachable owner 504 then 200 → test_d_…
  • (e) slow pull: 504 fetch_pending + Retry-After, retry joins the same transfer → test_e_…
  • (f) revoked view → 409, nothing cached (I1-1) → test_f_…
  • (g) unreferenced digest → 409, nothing cached → test_g_…
  • (h) owner without file transfer → 409 outdated → test_h_…
  • (i) remote subagent image via child route → test_i_…
  • (j) remote wav waveform → test_j_…
  • (k) 64 MiB served with tracemalloc peak < 4 MiB (I1-3) → test_k_…
  • (l) eviction within budget (I4-1) → test_l_…
  • C3: session_files/test_family_module.py (same objects re-exported; relay import path loads neither fastapi nor the desktop module)
  • C4 I2-1/I2-2/I2-3, I3-2, Downloads fallback, capability skew, >8 MiB page trimmed with link alive, user attachments, sensitive rule → test_remote_files_index.py (11 cells)
  • C5 active set, re-list gating, media-only ≤32 MiB, budget stop, on-screen request not queued behind precache, RSS delta < 64 MiB for 10×16 MiB → test_remote_files_precache.py (6 cells)
  • C6 I5-1 (thread-scoped stat_path spy), I5-2 (desktop_mesh._call raises; list/detail/thumbnail of every remote asset 200/404), thumbnail after open survives eviction, local+remote digest = one asset with two occurrences, removal after two answered catalogue reads (silent device keeps rows) → test_remote_files_library.py (6 cells)

Budgets and their basis

  • Cache: desktop.remote_files_cache_bytes (LIVE), default 1 GiB = ~55× a typical active set (16 sessions × 6 digests × p90 187.5 KiB ≈ 18 MiB), 3.8× the largest measured pad (269 MiB) — design §4. 0 = staging only, precache off.
  • Precache: active = moved within 2 h (11 of 239 sessions on the measuring host) + last 8 opened; ≤16 sessions/pass; re-list ≤ once per 60 s and only when mtime moved; per file ≤32 MiB (the UI's MAX_DOWNLOAD_BYTES); 1 precache pull per owner inside the owner's semaphore of 2.
  • Timeouts: viewer waits REMOTE_FETCH_WAIT_S = 20 s for a pull before answering 504 pending (the pull continues); net_session_files owner deadline 30 s (1.5 s scan budget + discovery + stats, building covers slower), viewer client 45 s; Library remote feed every ≤300 s while in a network.
  • RAM: measured in tests — 64 MiB serve tracemalloc peak < 4 MiB (base buffered 235.6 MiB); 160 MiB precache RSS delta < 64 MiB; download copies in 1 MiB blocks.

RED on base / negative controls

  • C1: base scanned the owner transcript 65 times for one 32 MiB pull (assert 65 == 1).
  • C2 on base source: (a)/(b) 409 == 200 attachment_on_peer; (i) 404 == 200 child_not_found; (k) serving buffered 235.6 MiB.
  • C4 with base desktop_session_files.py: KeyError: 'remote' (base answered unsupported, no rows). _fit disabled → the oversize page closes the link → 504.
  • C5 with OWNER_CONCURRENCY=1: "the on-screen request waited 1.54s behind precache".
  • C6 on C5 head: library.remote absent (ImportError); storing the display path as path → I5-1 fails ('report.md' is None).

i18n

New keys (en + context; fr/es/zh-CN/ru/vi/ur/hi translated in this PR, scripts/i18n/check.py --catalogues-only: 0 problems): wire.errors.desktop.{attachment_owner_unreachable, attachment_fetch_pending, attachment_peer_outdated, session_files_owner_unreachable, session_files_remote_unreadable, session_files_sensitive_name, session_files_download_exhausted, session_files_download_mismatch}, wire.settings.desktop_remote_files_cache_bytes.{label,help}. shipped.json is at base bytes. All 14 ledger rows are bound in 5d7e1a5e4, from the independent round-2 audit (#issuecomment-6831544890): generate.py --check 161 artifacts byte-identical, ledger.py validate valid and append-only, check.py --audit-base origin/main ok.

Targeted test evidence

One authorized convergence 7fb7cfc2d122cbb21096081b67a401636914bbcf: merges exact main/#3892 4d113fc6c23b3fbb2afa426c0c6747ee0c399467 into reviewed 360315b47. Sole conflict resolved through manager-approved immutable provenance revision 72043c4ecbfddd734abec28694b05a163786bf0c, authenticating the existing b0edf02e5/#1877 fixture; no fixture/hash refresh. Actual repo strict checker fetched and authenticated fixture, manifest and 36 model sources: PASS; checker controls 15 passed. Scoped overlap/neighbors pyright 0 errors/warnings; overlap-only Black/isort/flake8 PASS. 95 non-overlap feature files byte-identical, all 97 non-metadata feature +/- patches identical, 19 range-diff pairs =. Only metadata patch differs (4 lines); overlaps are parity-sources.json, relay.py, desktop_sessions.py. Full ### Rebase proof and before/after hosted discriminator are in ### Convergence after reproducible parity prerequisite. CONVERGENCE REVIEW NEEDED by independent reviewer; manager owns one gate rearm. No gate arm/cancel, full matrix, artifact/native QA rerun or PR merge. Actual model openai/gpt-6.1-sol.

Required-CI test-type repair 360315b4776f61f5ff3e7fd330c70e8f7ffb8614 (actual openai/gpt-6.1-sol): fixes the index polling helper's possibly-unbound page and three local attachment union unpacks through explicit runtime invariants; no ignores/casts or product changes. Scoped CI-pinned pyright 1.1.414 on the two test modules plus actual server/utils/desktop_sessions.py types: 4 errors before → 0 errors/warnings after. Isolated bounded real-loopback index matrix plus exact local attachment/refusal selectors: 24 passed, 5 deprecation warnings in 5.90s; unrelated MP3 fixture cases not selected. Direct polling success/exhaustion controls PASS; all original assertion ASTs retained. Exact-file Black 26.1.0, isort 5.13.2, flake8 7.4.1, whitespace PASS. Shipping source hash remains 512e12dcc19b7ab1a900073c108fc320be022d9a9065239ef0229bd2b207caaa. See ### CI remediation — test type narrowing for T1–T4 and exact commands/proof. No gate rearm/cancellation, upstream convergence or full matrix; prerequisite #3892/convergence remain manager-owned.

Late lint-only repair f49dcc68bc18daae64429429e72de065471ab1e8 (actual model openai/gpt-6.1-sol): CI-pinned Black 26.1.0 reformats exactly two expressions in test_remote_files_hardening.py and test_slim_wheel.py. Reproduced 2-file FAIL before, exact-file Black/isort 5.13.2/flake8 7.4.1 and whitespace checks PASS after; isolated bounded minimal compression/symlink selections 3 passed in 0.77s. Complete location-independent ASTs exactly equal in both tests. All tracked product/security implementation/scripts/catalogues plus pyproject bytes retain SHA-256 512e12dcc19b7ab1a900073c108fc320be022d9a9065239ef0229bd2b207caaa; shipping subtree diff empty. No installs, artifact-QA rerun, full suite or workflow manipulation. Exact commands/proof in ### CI remediation — test formatting; static delta review manager-owned.

Wheel-size required-CI repair f10d7c17956ead2ae06b1d6bb356b04fee771556 (model openai/gpt-6.1-sol): lossless DEFLATE level 9 in the existing repacker only; no product/assets/version/budget changes. Failed hosted wheel 18,350,518 bytes (438 over). Actual local uv build --wheel --no-build-isolation --offline with cached setuptools 84.2.1 / wheel 0.48.0 (no installs), paired same-input original/new repacks 18,344,938 → 18,251,825 bytes, saving 93,113; final headroom 98,255. All 3,165 ordered member names, timestamps/permissions, uncompressed SHA-256s and RECORD bytes identical; CRC + RECORD validation PASS. Isolated bounded targeted test_slim_wheel.py + mobile/test_web_package_data.py: 25 passed; old compression fails new regression. Wheel-loaded base imports, runtime-resource reads and CLI --version PASS; flake8/compileall/diff whitespace PASS. Repack wall 31.51s → 33.95s. Exact commands, baseline discrepancy disclosure and independent-QA wheel path/hash in ### CI remediation — wheel size. No full-suite/workflow manipulation/merge/release.

Directory-removal guard CI repair 928c9fc280ab109de37d9b6551a3bf13f7f3fd63 (actual model openai/gpt-6.1-sol; source-delta review/QA manager-owned): test_no_session_deletion.py RED before: 1 failed, 613 passed; focused confinement controls RED: 3 failed, 3 passed. Fixed proven cache-parent/root symlink traversal and non-digest selection using held no-follow directory fd + fd-relative unlink; land unchanged and O_EXCL collision controls PASS. Final isolated serial run_bounded --timeout 600 --memory-mb 3000 -- .venv/bin/python -m pytest -q -n 0 -p no:cacheprovider tests/unit/session/test_no_session_deletion.py tests/unit/network/test_remote_files_hardening.py tests/unit/network/test_remote_files_index.py tests/unit/network/test_remote_files_fetch.py::test_l_eviction_keeps_the_cache_within_its_budget: 654 passed, 5 warnings in 62.04s; cleanup proven-empty. Flake8 (three changed files) + diff whitespace PASS. Exact evidence/proof in the ### CI remediation — directory-removal guard comment. No catalogue or other product changes.

Condensation CI remediation bec264a29 (fixture identity only): desktop generated bytes from damianvtran/local-operator-ui@5033f642352e93c30b699ff62e5bae0e1939da6c; SHA-256 523e8a5959fa198795ee2e963139560906aa5f425ff9be77c167a369303a35f1, model sourceHash eb9d2c6d83bb277ee50d1018009a86d5218b5d5d9263bacfa25de1e339be5a82. Strict JSON comparison proves all 700 cases, latch, modelSources and every field except desktop.sourceHash unchanged. Commands below ran from local_operator/mobile/web, each under .venv/bin/python scripts/run_bounded.py --timeout 600 (repository-relative wrapper paths resolved absolutely):

  • node scripts/check-condense-parity-latest.mjs --latest --require-sibling --sibling <committed-desktop-fixture-snapshot>: existing branch bytes FAIL (rc=1) → copied fixture PASS (rc=0).
  • node scripts/check-condense-parity-latest.mjs: PASS, 2 fixtures match their pins.
  • node node_modules/vitest/vitest.mjs run src/lib/condense.test.ts --maxWorkers=1 --minWorkers=1 (--memory-mb 3000, isolated HOME/config, inherited CMUX_/LOP_ stripped): 1 file / 1,361 tests passed. Reused existing dependencies with byte-identical pnpm-lock.yaml; no install.
  • git diff --check: PASS. No executable code, catalogues or condensation decisions changed.

Convergence f7337ad27944df4eef4a3026831fb1df9b386ab1 merges core main 97eed22c9a81191ebee3a7490d3001d77a14e5ad (old head bec264a29, old merge base 89678ffc7aa536076cc3c16d9d204829a4e83a97). Two conflicts only: generated fixture and pins, resolved to the already-proven desktop bytes. Final head repeats strict drift guard/self-consistency PASS, condensation 1,361 passed, and test_settings_io.py -k "remote_files_cache_bytes or network.credentials.github.repositories" 4 passed. Network neighbors (test_remote_files_index.py, test_mesh_file_pull.py, test_create_idempotency.py): 48 passed / 1 failed. Failure at test_create_idempotency.py:114 sees an additional net_definitions frame; the single test fails serially too and with exact upstream relay source (SHA256 838ae42704fcc6bfdc1b13fda18ca3c125deddefa91d76a9b45380086488ec95) plus byte-identical upstream test, establishing it is not introduced by remote-file relay additions. This source-control run uses the current package for other modules; upstream relay refuses the branch-only remote-files slice as expected. No out-of-scope change made. Full logs retained in the delegated session's scratchpad://condensation-parity/.

rebase_proof.py is CONVERGENCE REVIEW NEEDED, not mechanical: 14/14 range-diff pairs =, but 6 fixture/pin +/- lines differ and upstream overlaps five files. All non-condensation PR +/- lines and file headers are byte-identical; manager owns one convergence delta review and CI/merge gate.

Head 3a842e70b (round-3 remediation, #issuecomment-6833147857). Isolated HOME/LOCAL_OPERATOR_CONFIG_DIR, CMUX stripped, each run python3 scripts/run_bounded.py --timeout 600 --memory-mb 3000 -- .venv/bin/python -m pytest -q -p no:cacheprovider …:

tests/unit/network/test_remote_files_offline_owner.py                       → 12 passed
tests/unit/network/test_remote_files_{hardening,library,fetch,index,precache,scan}.py → 55 passed
tests/unit/library/test_{route,service}.py tests/unit/mobile/test_library_routes.py
  tests/unit/i18n/test_catalogue_sizes.py tests/unit/server/test_desktop_remote_open.py
  tests/unit/server/test_supplement_routes.py                                → 94 passed
tests/unit/server/test_desktop_sessions.py tests/unit/network/test_remote_viewer.py
  -k "attachment or waveform or child or peer or remote"                     → 67 passed, 2 errors (pre-existing: same 2 on 5d7e1a5e4 source)
RED on 5d7e1a5e4: test_remote_files_offline_owner.py → 9 failed, 2 passed (the 2 are controls)

git diff 5d7e1a5e4..HEAD -- local_operator/i18n empty; generate.py --check 161 artifacts byte-identical.

Head 5d7e1a5e4: remote-files files → 55 passed; library route/service + i18n catalogue tests → 60 passed; RED on 14d8601: test_remote_files_hardening.py -k q2 → 3 failed.
Earlier head 14d8601fd: the same remote-files files + test_mesh_file_pull.py → 81 passed; tests/unit/test_settings_io.py, server/test_desktop_remote_open.py, session_files/test_{family_module,route}.py, i18n/test_language_manifest.py → 805 passed (round 2 did not touch these surfaces). RED on 9a6ef56: test_remote_files_hardening.py → 14 failed.

Release: patch — Fetch and cache remote-session assets for desktop previews and verified downloads, with indexed file listings and bounded disk use.

The owner authorises every phase of a mesh file pull by the session's
referenced-digest set; each phase re-read the whole transcript (65 reads of
a 64 MiB journal for one 32 MiB pull). Memoise the set per transcript
(path, size, mtime_ns, inode) in an LRU of 8 and scan in 4 MiB windows with
a 64-byte overlap instead of read_bytes().

docs/design/remote-files-parity.md C1.
A peer-held session's attachment, waveform, supplement document and a
remote subagent's (child route) image now resolve through one lookup
order: this device's store, the session's replica (closes G1), the new
bounded remote-files cache, then ONE pull from the owner over the relay
(session_file_pull land=cache; closes G2). The owner still applies S1
confinement on every phase.

- remote_files/cache.py: <config>/cache/remote-files in the store layout,
  LRU on .bin mtime under flock, budget desktop.remote_files_cache_bytes
  (LIVE, default 1 GiB; 0 = staging only); I4-1 keeps the file just landed.
- remote_files/fetch.py: in-process single-flight per (owner, member, key),
  Semaphore(2) per owner, 20 s shielded wait (loopback max 6.0 s x 3) after
  which the pull keeps running and the route answers pending.
- Statuses: 504 attachment_owner_unreachable / attachment_fetch_pending
  (+Retry-After: 2) are retryable; 409 attachment_peer_outdated and
  attachment_on_peer are permanent. Never 503 (reads as the plane closing).
- Peer bytes are STREAMED from an fd opened before answering, with Range
  (206), via server/utils/byte_range.py lifted from the Library route.
- i18n: the PR's nine new keys (incl. C4's Files copy) in en + fr, es,
  zh-CN, ru, vi, ur, hi; ledger audit rows pending an independent auditor.

docs/design/remote-files-parity.md C2.
The relay must list a session's family for net_session_files (C4) without
importing server.utils.desktop_sessions (+27 MiB RSS, +0.8 s, fastapi).
Pure move of _persisted_children, _contained_child_dir, _prove_child,
contained_descendants, FAMILY_MAX_* and SubagentChildUnavailable into
session_files/family.py, re-exported under the same names; the files
route's _family is lifted as family.family_for. Two route tests now patch
the names where they live.

docs/design/remote-files-parity.md C3.
A peer session's Files panel is the OWNER's own index, asked over the mesh:
- net_session_files (view, owner) runs session_files.service.list_files over
  its journal with the shared family definition; cursors/revisions pass
  through opaquely, paths leave only in display form, credential-shaped files
  are never listed. Each page row adds remote_read / remote_unreadable under
  S1 confinement (advisory: net_file_get re-checks every byte). A page over
  one link record is trimmed and next_cursor re-minted (an oversize record
  closed the link: 504 in the negative control).
- peer_session_files (local) forwards one phase after checking the owner's
  mesh-file-index-v1; an older owner reads as peer_update_required.
- Viewer routes: …/files and …/user-attachments answer peers with remote,
  origin_device (from the catalogue row), unsupported_reason, device_name;
  new peer-only GET …/files/{id}/content (streamed, Range) and POST
  …/files/{id}/download (~/Downloads or <config>/downloads, never overwrites,
  O_EXCL temp + 1 MiB streamed copy + fsync + content-address check + link).
- pull_file land=cache skips the transfer when the bytes are already cached.
- features.remote_files = 1; DESKTOP_API.md 'Remote files'.

docs/design/remote-files-parity.md C4.
The catalogue poll (DesktopSessions.list with peers) hands its remote rows to
remote_files.precache: active = reachable, unarchived, moved within 2 h, plus
the last 8 remote sessions opened here (files_target's peer branch), at most
16 per pass, each re-listed at most once per 60 s and only when its mtime
moved. A pass lists user attachments + image/audio/video file rows through
the owner's index and pulls each readable file at most 32 MiB (the UI's
MAX_DOWNLOAD_BYTES) into the bounded cache through the same fetch-on-miss;
one precache pull per owner inside the owner's Semaphore(2), so an on-screen
request always has a slot (negative control: with one slot it waited 1.54 s).
A pass stops before exceeding the budget; budget 0 disables precache.

docs/design/remote-files-parity.md C5.
The Library indexer's reconcile now runs library/remote.py: for each reachable
remote session whose catalogue mtime moved, the owner's Files index is listed
(peer_session_files, pages of 200) and its rows become occurrence rows in the
SAME tables, so grid/facets/search/dedup need no second path. Metadata only.

- schema v3: idx_assets.remote/remote_device/remote_device_name,
  idx_occurrences.origin_device/origin_device_name/remote_file_id,
  idx_sessions.origin_device/misses (a v2 index rebuilds once).
- identity.remote_key r:<device>:<session>:<display path> with path NULL, so a
  remote asset never resolves to a file on this disk (I5-1); digest rows keep
  a:<digest>, so bytes held locally and remotely are ONE asset.
- resolve_content answers content_remote for peer-only bytes; ONLY the
  …/content route fetches (remote_session_files.library_content). List,
  detail and thumbnail never dial (I5-2). thumbnail_source reads the remote
  cache; a digest asset's liveness stamp is a:<digest>, so evicting the cached
  source keeps its thumbnail.
- Sessions absent from two ANSWERED catalogue reads are dropped; an
  unreachable device's rows are kept; a session that became local is left to
  the local scanner. LibraryRow.remote = {device_id, device_name}.

docs/design/remote-files-parity.md C6.
@damianvtran

Copy link
Copy Markdown
Owner Author

Agent review — round 1

Reviewer: reviewer (lopdev team, independent; served model anthropic/claude-opus-5-5, no fallback)
Scope: 89678ff..9a6ef56
Worktree: .worktrees/remote-files-parity @ 9a6ef56 (read-only), security lens throughout.

Verdict: changes requested. 2 MAJOR, 3 MINOR, 1 NIT. The authority reuse, streaming and status mapping are solid; both majors have a small fix.

What I verified (no finding)

  • Authority. net_session_files is view in OP_CAPABILITY, in NET_OPS/SLICE_PEER_OPS/_owning_document. It goes through the generic Authorizer._session_scope (outer session_id must be in local_session_ids()). The handler refuses a nested frame, the same as net_file_get (feat(network): pull one file from a peer's session, verified end to end #3794 R1-3). Totality and authorizer tests pass. The listing is advisory: every byte still goes through net_file_get, which re-checks the referenced digest on every phase and keeps pad-only confinement. Owner paths leave only in display form, and stored_path is nulled.
  • Download landing. The temp file is opened O_EXCL, the copy is streamed in 1 MiB blocks, fsynced and sha256-checked against the content address, then published with os.link (no replace). A same-bytes file is reused and a collision becomes (n). sanitise_name and _bare_name_denied run on the owner name, so there is no traversal through the landing name. file_id is typed on the route and _ID_RE-checked on the owner.
  • RAM. byte_range.stream_handle serves from an fd opened before answering. adopt_file hashes in 1 MiB blocks. The reference scan uses 4 MiB windows with a 64 B overlap: the longest compact match is 48 B, so a boundary-straddling reference is still found.
  • Cache. Eviction only scans cache/remote-files, so the authoritative store is never evicted. The pass holds flock plus a thread guard.
  • Status mapping and version skew. Statuses are 504 (Retry-After when pending), 409 (outdated, unreadable, on_peer) and 404. That matches the four wire assumptions in UI test(mobile): pin the §1.2 shape and the one-build pairing #1880. Owners without mesh-file-index-v1 get peer_update_required, and owners without mesh-file-v1 get 409 outdated.
  • Library. A thumbnail read for remote content maps to content_missing, so it never dials. The v3 bump uses the existing drop-derived/rebuild path that v2 used; metadata is untouched.

Findings

R1-1 · MAJOR · security: an owner-supplied digest and member id become viewer filesystem paths

  • Where: server/utils/remote_session_files.py:175-178. _resolve passes the owner's row-phase reply (source["attachment"], source["session_id"]) unvalidated into peer_blob → cache.lookup → cache.locations, which builds root/attachments/f"{digest}.bin" (remote_files/cache.py:96-100). The result is then opened by _open and streamed by the content route. The library feed is not affected, because identity.digest_key validates the digest there.
  • Repro (scratch config with an attachments/ dir):
    • Call cache.lookup(root, ("0123456789ab",), "../../home/secret/notes").
    • Actual: ('…/r1/home/secret/notes.bin', 'store', 'application/octet-stream', b'LOCAL-ONLY-BYTES').
    • A malformed member id ("../../x") raises an uncaught SyncRefused from replica_dir, which the errors() ladder has no arm for, so the route answers 500.
  • Impact: a peer can make the viewer open and stream any *.bin file the viewer can read. The bytes go to the viewer's own renderer, not back to the peer, so this breaks the "no traversal" confinement but does not exfiltrate data. The download leg refuses it, because its sha check fails.
  • Fix: in _resolve, require fullmatch("[0-9a-f]{32}") on the attachment and SESSION_ID.fullmatch on member_id, and answer 409 session_files_remote_unreadable otherwise. Optionally add the same shape check in cache.locations as a backstop. Add a regression test with a stub ask that returns a traversal digest.

R1-2 · MAJOR · precache passes are not gated when opened through files_target

  • Where: desktop_sessions.files_target (~8927) calls precache.opened_session on every Files-panel request: each listing page, building poll, content open and download. opened_session → _enqueue skips the RESCAN_S/mtime gate that observe applies (remote_files/precache.py:110-118).
  • Cost: every pass makes two owner peer_session_files list ops (user_attachments plus files, limit 200) and one unbudgeted row op per media path row. Those asks are not under the precache semaphore, so they compete with on-screen listing asks on the link and on the owner's op thread. That works against "precache stays behind on-screen requests" and against the design's at-most-once-per-60 s trigger.
  • Repro: Precacher with prepare stubbed to its own first line (it records passes), same unchanged mtime:
    • 12 × observe → 1 pass.
    • 12 × opened_session 0.1 s apart → 12 passes.
  • Fix: in opened_session, keep the LRU update but enqueue only when there is no passes[session_id] from the last RESCAN_S with the same mtime, or when the session is new to opened. Add a regression test asserting N files_target calls cause one pass.

R1-3 · MINOR · the eviction race is handled only on the transcript route

  • _peer_file re-looks-up once on FileNotFoundError. remote_session_files._open (content route and library_content) and land (download) do not. A blob evicted between lookup and open gives:
    • 500 on content and library content (the errors() ladder has no OSError arm);
    • 409 session_files_download_failed on download, which the UI reads as permanent.
  • Also, _run evicts with keep={own digest} only, so a concurrent landing can evict another in-flight landing before its lookup. That only happens under a tight budget; at budget 0 it causes 504 ping-pong.
  • Fix: reuse _peer_file's retry for these call sites, and pass keep = the digests of all unfinished _TASKS.

R1-4 · MINOR · a long CJK file name cannot be downloaded

  • land names its temp file .{name}.{12 hex}.part, which adds 19 bytes. sanitise_name allows 80 Unicode characters, so "文"*100+".png" → 84 chars / 244 bytes, a valid final name, but the temp name is 263 bytes.
  • Repro: land(...) → OSError 36 (ENAMETOOLONG) → 409 session_files_download_failed. zh-CN is a shipped locale.
  • Fix: use a fixed short temp name (.lop-download-<hex>.part).

R1-5 · MINOR · remote Library content is served inline with a peer-supplied type

  • remote_session_files.library_content serves asset.content_type, which for a remote row is the owner's listing ref.content_type, inline. It has nosniff but no CSP. The session-files content route next to it maps non-passive types to octet-stream and adds Content-Security-Policy: sandbox.
  • The local Library route has the same shape (pre-existing), but here both the type and the bytes come from another device.
  • Fix: apply the same ATTACHMENT_SERVABLE_MIME_TYPES | _PASSIVE_TEXT allowlist and the sandbox CSP in library_content.

R1-6 · NIT — single-flight is keyed (owner, member_id, key), so the same digest asked for through the parent's transcript route and through a child-member Files row starts two transfers. It is correct, just duplicated work. Key on (owner, key) for digest sources.

Tests (run here, bounded, isolated HOME/config, CMUX stripped)

  • PR files test_remote_files_{fetch,index,library,precache,scan}.py, test_family_module.py, test_desktop_remote_open.py, plus network/test_authorizer.py and network/test_definitions.py (totality): 155 passed (42.8 s).
  • Acceptance letters a–l (fetch), I2/I3 (index), I5 (library) and the precache RSS/budget/priority cases run on the two-relay loopback (the mesh fixture from test_mesh_file_pull). The 64 MiB serve is driven at the ASGI boundary under tracemalloc.
  • Coder's "3 pre-existing test_route.py failures": not reproducible on head.
    • tests/unit/session_files/test_route.py alone: 35 passed (isolated HOME).
    • The 3 -k materialise tests with the real HOME and isolated config: 3 passed.
    • The PR's new files plus the whole tests/unit/session_files/ in one process (pollution check): 321 passed, 1 skipped.
    • So there is no failure to attribute to base, and I did not run a base comparison.

Not reviewed (other streams): i18n catalogues, shipped.json, UI #1880.

@damianvtran

Copy link
Copy Markdown
Owner Author

i18n audit — round 1

Reviewer: i18n auditor plus English copy review (lopdev reviewer role). Instance a3fade2429f4, task core-r1-i18n-copy. Served model anthropic/claude-opus-5-5, no fallback. I did not write or translate this code.
Scope: 89678ff..9a6ef56. That is the 10 new keys (wire.errors.desktop.{attachment_owner_unreachable, attachment_fetch_pending, attachment_peer_outdated, session_files_owner_unreachable, session_files_remote_unreadable, session_files_sensitive_name, session_files_download_exhausted, session_files_download_mismatch}, wire.settings.desktop_remote_files_cache_bytes.{label,help}) across 7 locales × 2 namespaces = 14 pairs. Contract: AGENTS.md §Internationalization, i18n/glossary.md, the 7 style guides and docs/I18N.md step 6. Precedent: #3705.

Verdict: clean. There is no BLOCKER or MAJOR. Five MINOR and two NIT findings are below. Six pairs can be bound now. Eight bind once the auditor-supplied corrections in I1-4 are applied byte-exactly and get a second independent read, as in the #3440 precedent. The en findings (I1-1, I1-2, I1-3, I1-5) can be deferred. If any of them is taken, the 7 wire.errors.desktop receipts go stale and need a round-2 audit (see COMMANDS.md §C).

Mechanical checks (all 14 pairs, at head and at the corrected bytes): 0 problems

  • Key set equals en. The pre-existing keys are byte-identical to the merge base. Every pair's last ledger row at base is passed, and its source_sha256 is the base en hash. So it is a delta audit of the added keys plus a carry of the unchanged ones.
  • Placeholder names match en per message and per select arm. session_files_remote_unreadable has the same 6 arms in the same order with other present, {device} sits inside the arms, and {reason} never renders. Braces balance.
  • No bidi control characters (ur follows style guide §RTL: logical order, LTR islands for lop network update {device} and 256 MiB, and Urdu ۔ ،). All text is NFC. The 256 and 99 figures and the command are byte-identical in every locale.

Findings

I1-1 — MINOR — en: the 504 offline sentences promise an automatic load, and they don't match the UI's shared offline sentence.
attachment_owner_unreachable (en/wire.errors.desktop.json:18) says "It loads once {device} is back online", and session_files_owner_unreachable (:83) says "They appear once…". The key's own context note (context.json:32) says "The desktop retries on its own and then offers a retry". So after the retries stop, nothing loads until the user acts. The UI renders its own sentence for this state (remoteUnreachableText, ui remote-files.ts:124, ui.canvas.files.remoteUnreachable: "{device} isn't answering. Try again when it's online."). So these core sentences reach envelope consumers only, which is why this is MINOR. Suggested wording, in this namespace's uncontracted register (0 contractions and 18 uses of "cannot" in wire.errors.desktop):

  • attachment_owner_unreachable: {device} is not answering. Try again when it is online.
  • session_files_owner_unreachable: {device} is not answering, so this conversation's files cannot be listed. Try again when it is online.
  • In both context notes, change "keep both mentions" to one mention.

I1-2 — MINOR — en: attachment_fetch_pending (:17) says "It appears in a moment". The 504 sends Retry-After: 2, and the bytes only appear when someone asks again. A transfer that can run to 256 MiB is not "a moment". The UI already words this "Still fetching from {device}. Try again in a moment." Suggested: This file is still on its way from {device}. Try again in a moment.

I1-3 — MINOR — en: attachment_peer_outdated (:19) gives a command that can't always be run. The command is not backticked, unlike its sibling session_is_remote_open (`lop sessions move …`) and en style "Backticked literals (commands…)". {device} is the owner's label, or UNNAMED_DEVICE = "unnamed device" (resume.py:2904; desktop_sessions.py:6862). So it can render lop network update unnamed device, or an unquoted name with spaces, and neither runs. lop network update accepts a device id (network/cli.py:912). Suggested: {device} runs a version that cannot send files yet. Update it with + "lop network update {peer}" + , then try again., with a new {peer} param = row.owner_device (the id). The minimum fix is backticks alone.

I1-4 — MINOR — locales: glossary and style-guide deviations, measured against the existing catalogue. I give byte-exact corrected values (full table below). apply_corrections.py applies them deterministically.

  • a fr: identifiant secret should be fichier d’identifiants (glossary §2 credentials = identifiants; fr catalogue has 23× "identifiants", 1× "identifiant secret", the new one). 256 Mio should be 256 MiB (5× MiB in the fr catalogue, 1× Mio, the new one; the en context says "Keep the 256 MiB figure").
  • b zh-CN: all 8 new strings put spaces between Chinese and Latin, digits or placeholders. zh-CN guide l.67 says "No space inserted between Chinese text and Latin/digit runs", and only 1 of the 89 base strings in this namespace does it. scratchpad should be 暂存区, the existing zh rendering (session_scratchpad.title, cleanup_scratchpads_only).
  • c ru: рабочей папке ("working folder") should be рабочем пространстве (glossary workspace = рабочее пространство). scratchpad should be скрэтчпад, the existing ru rendering (session_scratchpad*).
  • d ur: device آلہ/آلے/آلات should be ڈیوائس/ڈیوائسز (glossary §2; 56× ڈیوائس in ur, 5× آلہ-forms, all new). This affects 2 wire.errors.desktop keys and the cache help. credential اسناد (a plural used after singular کسی) should be کریڈینشل (31× in ur, 2× اسناد, both new).
  • e hi: वर्कस्पेस should be कार्यक्षेत्र (glossary and hi guide l.28; 7× vs 1× new). scratchpad should be स्क्रैचपैड (existing hi). वर्शन should be संस्करण (23× vs 1× new).
  • f vi: label bộ nhớ đệm should be cache (vi guide l.23 kept-English list includes cache; vi labels use "TTL cache", "ái lực cache"). In session_files_sensitive_name, chưa được lưu ("not saved yet") should be không được lưu, because the refusal is permanent.
  • es: no findings.

I1-5 — MINOR — en: the sensitive arm of session_files_remote_unreadable (:84) also covers ordinary hidden files. network/file_index.py:143-147 returns "sensitive" for any pad path with a dot component, so .gitignore or .cache/x.png would read "it looks like a credential". Suggested arm: it is a hidden file or looks like a credential, so {device} does not send it. (The alternative is a separate hidden reason code.)

N1 — NIT — en too_large arm: "larger than the 256 MiB one device can send another" reads awkwardly. Suggested: it is larger than 256 MiB, the most one device can send another.
N2 — NIT — en desktop_remote_files_cache_bytes.help (wire.settings.json:164) leaves out the default. The sibling session_scratchpad_max_bytes.help (:620) ends "Default 256 MiB." The default here is 1 GiB (settings_io.py:5351). Defer it: it would stale all 7 wire.settings receipts.

The rest of the en copy is honest and actionable. The 409 sentences say why and are final. download_mismatch says "Try again" with reason. Every sentence where the server knows the device names it; sensitive_name and download_exhausted don't need it.

Out of scope, pre-existing: the English UNNAMED_DEVICE fallback is substituted into translated sentences (desktop_sessions.py:6822 on base). The new sites inherit the pattern.

Ledger: 14 receipts and the bind commands

  • Commands, validated against a scratch copy: /home/ec2-user/.local-operator/sessions/a3fade2429f4/scratchpad/i18n-3872-r1/COMMANDS.md. Run log: …/validate.log (29 steps rc=0, and 4 negative controls refused as intended).
  • Receipts: …/i18n-3872-r1/receipts/<loc>.<ns>.json. Evidence records: …/evidence/<loc>.<ns>.record.json (bind.py copies them to i18n/audits/evidence/<loc>/<ns>/rfp-a3fade2429f4/record.json). Corrections: …/corrections.json.
  • A, bind now: es/wire.errors.desktop, and fr, es, zh-CN, ru, hi /wire.settings.
  • B, after apply_corrections.py and a second independent read: fr, zh-CN, ru, ur, hi, vi /wire.errors.desktop, and ur, vi /wire.settings.
  • shipped.json: at head, generate.py --check reports changed: shipped.json and 7 orphaned bundled.<loc>.gen.ts. Without these rows the generated set is ["en"]. Once all 14 rows are bound, run generate.py. shipped.json must then come out byte-identical to base with all 8 locales; I validated this in scratch with _render_shipped() against the scratch ledger. Commit it together with the rows. Never hand-edit it, and never git checkout it once the rows exist.
I1-4 corrected values (17), byte-exact
  • I1-4a fr wire.errors.desktop.session_files_remote_unreadable
    • current: Ce fichier ne peut pas être ouvert depuis cet appareil : {reason, select, outside_session_files {il se trouve dans l’espace de travail de {device}, hors du scratchpad de la conversation. Demandez à la conversation de le copier dans son scratchpad.} missing {il n’existe plus sur {device}.} too_large {il dépasse les 256 Mio qu’un appareil peut envoyer à un autre.} sensitive {il ressemble à un identifiant secret, {device} ne l’envoie donc pas.} not_referenced {la conversation sur {device} n’y fait plus référence.} other {{device} ne le partage pas avec cet appareil.}}
    • corrected: Ce fichier ne peut pas être ouvert depuis cet appareil : {reason, select, outside_session_files {il se trouve dans l’espace de travail de {device}, hors du scratchpad de la conversation. Demandez à la conversation de le copier dans son scratchpad.} missing {il n’existe plus sur {device}.} too_large {il dépasse les 256 MiB qu’un appareil peut envoyer à un autre.} sensitive {il ressemble à un fichier d’identifiants, {device} ne l’envoie donc pas.} not_referenced {la conversation sur {device} n’y fait plus référence.} other {{device} ne le partage pas avec cet appareil.}}
  • I1-4b zh-CN wire.errors.desktop.attachment_fetch_pending
    • current: 此文件仍在从 {device} 传输中,稍后就会显示。
    • corrected: 此文件仍在从{device}传输中,稍后就会显示。
  • I1-4b zh-CN wire.errors.desktop.attachment_owner_unreachable
    • current: 此对话保存在 {device} 上,该设备目前无法连接,因此这个文件暂时还不在这里。{device} 重新上线后会自动加载。
    • corrected: 此对话保存在{device}上,该设备目前无法连接,因此这个文件暂时还不在这里。{device}重新上线后会自动加载。
  • I1-4b zh-CN wire.errors.desktop.attachment_peer_outdated
    • current: {device} 的版本还不支持发送文件。请运行 lop network update {device} 更新后重试。
    • corrected: {device}的版本还不支持发送文件。请运行lop network update {device}更新后重试。
  • I1-4b zh-CN wire.errors.desktop.session_files_download_exhausted
    • current: {name} 及其 99 个编号副本已存在于 {folder} 中,因此未保存任何内容。
    • corrected: {name}及其99个编号副本已存在于{folder}中,因此未保存任何内容。
  • I1-4b zh-CN wire.errors.desktop.session_files_download_mismatch
    • current: 来自 {device} 的副本与其声明的文件不一致,因此未保存任何内容。请重试。
    • corrected: 来自{device}的副本与其声明的文件不一致,因此未保存任何内容。请重试。
  • I1-4b zh-CN wire.errors.desktop.session_files_owner_unreachable
    • current: 此对话保存在 {device} 上,该设备目前无法连接,因此暂时无法在此列出其文件。{device} 重新上线后文件会显示出来。
    • corrected: 此对话保存在{device}上,该设备目前无法连接,因此暂时无法在此列出其文件。{device}重新上线后文件会显示出来。
  • I1-4b zh-CN wire.errors.desktop.session_files_remote_unreadable
    • current: 无法在此设备上打开此文件:{reason, select, outside_session_files {它位于 {device} 的工作区中,不在对话的 scratchpad 内。请让该对话把它复制到自己的 scratchpad。} missing {它在 {device} 上已不存在。} too_large {它超过了设备之间可发送的 256 MiB 上限。} sensitive {它看起来像凭据,因此 {device} 不会发送。} not_referenced {{device} 上的对话已不再引用它。} other {{device} 未与此设备共享它。}}
    • corrected: 无法在此设备上打开此文件:{reason, select, outside_session_files {它位于{device}的工作区中,不在对话的暂存区内。请让该对话把它复制到自己的暂存区。} missing {它在{device}上已不存在。} too_large {它超过了设备之间可发送的256 MiB上限。} sensitive {它看起来像凭据,因此{device}不会发送。} not_referenced {{device}上的对话已不再引用它。} other {{device}未与此设备共享它。}}
  • I1-4b zh-CN wire.errors.desktop.session_files_sensitive_name
    • current: {name} 看起来像凭据文件,因此未保存到此设备。
    • corrected: {name}看起来像凭据文件,因此未保存到此设备。
  • I1-4c ru wire.errors.desktop.session_files_remote_unreadable
    • current: Этот файл нельзя открыть на этом устройстве: {reason, select, outside_session_files {он находится в рабочей папке {device}, вне scratchpad разговора. Попросите разговор скопировать его в свой scratchpad.} missing {его больше нет на {device}.} too_large {он больше 256 МиБ — предела для передачи между устройствами.} sensitive {он похож на учётные данные, поэтому {device} его не отправляет.} not_referenced {разговор на {device} больше на него не ссылается.} other {{device} не открывает к нему доступ этому устройству.}}
    • corrected: Этот файл нельзя открыть на этом устройстве: {reason, select, outside_session_files {он находится в рабочем пространстве {device}, вне скрэтчпада разговора. Попросите разговор скопировать его в свой скрэтчпад.} missing {его больше нет на {device}.} too_large {он больше 256 МиБ — предела для передачи между устройствами.} sensitive {он похож на учётные данные, поэтому {device} его не отправляет.} not_referenced {разговор на {device} больше на него не ссылается.} other {{device} не открывает к нему доступ этому устройству.}}
  • I1-4d ur wire.errors.desktop.session_files_remote_unreadable
    • current: یہ فائل اس آلے سے نہیں کھولی جا سکتی: {reason, select, outside_session_files {یہ {device} کی ورک اسپیس میں ہے، گفتگو کے scratchpad سے باہر۔ گفتگو سے کہیں کہ اسے اپنے scratchpad میں کاپی کرے۔} missing {یہ اب {device} پر موجود نہیں۔} too_large {یہ اس 256 MiB حد سے بڑی ہے جو ایک آلہ دوسرے کو بھیج سکتا ہے۔} sensitive {یہ کسی اسناد جیسی لگتی ہے، اس لیے {device} اسے نہیں بھیجتا۔} not_referenced {{device} پر گفتگو اب اس کا حوالہ نہیں دیتی۔} other {{device} اسے اس آلے کے ساتھ شیئر نہیں کرتا۔}}
    • corrected: یہ فائل اس ڈیوائس سے نہیں کھولی جا سکتی: {reason, select, outside_session_files {یہ {device} کی ورک اسپیس میں ہے، گفتگو کے scratchpad سے باہر۔ گفتگو سے کہیں کہ اسے اپنے scratchpad میں کاپی کرے۔} missing {یہ اب {device} پر موجود نہیں۔} too_large {یہ اس 256 MiB حد سے بڑی ہے جو ایک ڈیوائس دوسری کو بھیج سکتی ہے۔} sensitive {یہ کریڈینشل فائل جیسی لگتی ہے، اس لیے {device} اسے نہیں بھیجتا۔} not_referenced {{device} پر گفتگو اب اس کا حوالہ نہیں دیتی۔} other {{device} اسے اس ڈیوائس کے ساتھ شیئر نہیں کرتا۔}}
  • I1-4d ur wire.errors.desktop.session_files_sensitive_name
    • current: {name} اسناد کی فائل جیسی لگتی ہے، اس لیے اسے اس آلے پر محفوظ نہیں کیا گیا۔
    • corrected: {name} کریڈینشل فائل جیسی لگتی ہے، اس لیے اسے اس ڈیوائس پر محفوظ نہیں کیا گیا۔
  • I1-4d ur wire.settings.desktop_remote_files_cache_bytes.help
    • current: دوسرے آلات پر موجود گفتگوؤں کی فائلوں کی کاپیوں کے لیے ڈسک کی جگہ، تاکہ وہ یہاں جلدی کھلیں۔ سب سے پرانی کاپیاں پہلے ہٹائی جاتی ہیں۔ 0 دکھانے کے بعد کچھ نہیں رکھتا۔
    • corrected: دوسری ڈیوائسز پر موجود گفتگوؤں کی فائلوں کی کاپیوں کے لیے ڈسک کی جگہ، تاکہ وہ یہاں جلدی کھلیں۔ سب سے پرانی کاپیاں پہلے ہٹائی جاتی ہیں۔ 0 دکھانے کے بعد کچھ نہیں رکھتا۔
  • I1-4e hi wire.errors.desktop.session_files_remote_unreadable
    • current: यह फ़ाइल इस डिवाइस से नहीं खोली जा सकती: {reason, select, outside_session_files {यह {device} के वर्कस्पेस में है, बातचीत के scratchpad से बाहर। बातचीत से कहें कि इसे अपने scratchpad में कॉपी करे।} missing {यह अब {device} पर मौजूद नहीं है।} too_large {यह 256 MiB की उस सीमा से बड़ी है जो एक डिवाइस दूसरे को भेज सकता है।} sensitive {यह किसी क्रेडेंशियल जैसी दिखती है, इसलिए {device} इसे नहीं भेजता।} not_referenced {{device} पर बातचीत अब इसका ज़िक्र नहीं करती।} other {{device} इसे इस डिवाइस के साथ साझा नहीं करता।}}
    • corrected: यह फ़ाइल इस डिवाइस से नहीं खोली जा सकती: {reason, select, outside_session_files {यह {device} के कार्यक्षेत्र में है, बातचीत के स्क्रैचपैड से बाहर। बातचीत से कहें कि इसे अपने स्क्रैचपैड में कॉपी करे।} missing {यह अब {device} पर मौजूद नहीं है।} too_large {यह 256 MiB की उस सीमा से बड़ी है जो एक डिवाइस दूसरे को भेज सकता है।} sensitive {यह किसी क्रेडेंशियल जैसी दिखती है, इसलिए {device} इसे नहीं भेजता।} not_referenced {{device} पर बातचीत अब इसका ज़िक्र नहीं करती।} other {{device} इसे इस डिवाइस के साथ साझा नहीं करता।}}
  • I1-4e hi wire.errors.desktop.attachment_peer_outdated
    • current: {device} का वर्शन अभी फ़ाइलें नहीं भेज सकता। इसे lop network update {device} से अपडेट करें, फिर दोबारा कोशिश करें।
    • corrected: {device} का संस्करण अभी फ़ाइलें नहीं भेज सकता। इसे lop network update {device} से अपडेट करें, फिर दोबारा कोशिश करें।
  • I1-4f vi wire.errors.desktop.session_files_sensitive_name
    • current: {name} trông giống tệp thông tin xác thực nên chưa được lưu vào thiết bị này.
    • corrected: {name} trông giống tệp thông tin xác thực nên không được lưu vào thiết bị này.
  • I1-4f vi wire.settings.desktop_remote_files_cache_bytes.label
    • current: bộ nhớ đệm tệp từ xa (byte)
    • corrected: cache tệp từ xa (byte)

@damianvtran

Copy link
Copy Markdown
Owner Author

QA report — round 1

Verdict: PASS. 40 of 40 matrix rows passed. There are no BLOCKER or MAJOR findings, and the two MINORs below don't gate.
Scope: 89678ffc7..9a6ef5661. Head: 9a6ef566184f4fd1bc373b14694841b0bf3b3a31. The worktree HEAD was checked before and after every run.
QA: independent qa-tester (lopdev). Served model: anthropic/claude-opus-5-5, no fallback.

Rig (real path, all on loopback):

  • Pairing. Two REAL relays, each started as a separate lop network serve process from the worktree .venv.
  • Isolation. Each relay has its own isolated LOCAL_OPERATOR_CONFIG_DIR and HOME. Everything ran under env -i, so no CMUX_* and no inherited LOCAL_OPERATOR_* reached it. The relays were paired once with the _pair_settled pattern, with B as a READ (view) member.
  • Viewer daemon. The viewer's REAL desktop daemon (lop serve, uvicorn) runs on B's config. Every row below is an HTTP request against it.
  • Owner daemon. A's own daemon is started only for the parity comparison.
  • Owner session 5afe0000a1b2 (synthetic) contains:
    • artifact PNG, WAV and MP4 (qa-clip.mp4);
    • a pasted image, plus document uploads of 3×32 MiB, 4×200 KB and 6×1 MiB;
    • pad files report.md, pad-shot.png, clip-31m.mp4 (31 MiB) and big-200m.mp4 (200 MiB);
    • credential-named pad files .env, credentials, secrets.json and id_rsa;
    • a workspace file ~/work/notes.md (cwd, outside the pad), a missing gone.txt, and one unreferenced digest in A's store.
  • Bounds. One heavy run at a time, under scripts/run_bounded.py (≤2500 MiB, one process group). Wall time was 133 s.
  • Re-run. bash /home/ec2-user/.local-operator/sessions/36af92881663/scratchpad/core-r1-qa/rig_run.sh. It drives rig_setup.py, rig_matrix.py and old_owner_wrapper.py from the same directory.
# Surface Command Actual Verdict
1 Transcript attachment, image / audio / video GET …/sessions/5afe0000a1b2/attachments/{d} All three 200. sha256 matches the owner. image/png, audio/wav, video/mp4; Accept-Ranges: bytes. Each landed in b/cache/remote-files/{d}.bin; b/attachments/ doesn't exist. 0.03–0.1 s PASS
2 Range Range: bytes=0-99, then bytes=1000-1999 on the MP4 206, 206. bytes 0-99/113026 and bytes 1000-1999/113026; both slices are byte-equal to the owner's PASS
3 Waveform GET …/attachments/{wav}/waveform 200: 240 peaks, 240 bars. {d}.peaks.json sits beside the cached blob PASS
4a Owner relay stopped SIGTERM relay A → GET; restart → GET 504 attachment_owner_unreachable ("qa-owner-a … cannot be reached right now …"), in 0.05 s, nothing cached. After restart: 200 on the first try, sha matches PASS
4b Owner relay frozen (extra) SIGSTOP relay A → GET; SIGCONT → GET 504 attachment_fetch_pending with Retry-After: 2 after 20.05 s (the designed wait; never 503, never a hang). After SIGCONT: 200 in 0.1 s PASS
5 Revoked view On A: lop network member revoke <net> <B> view (rc 0, removed: ["view"]) → GET; then grant → GET 409 attachment_on_peer. The cache has no blob and staging is empty. After re-grant: 200, sha matches PASS (copy: Q1-1)
6 Unreferenced digest GET a digest that is in A's store but not in this transcript 409 attachment_on_peer, nothing cached PASS (copy: Q1-1)
7 sessions.files for the remote session vs the owner's own daemon GET …/files?limit=100 on viewer :PD and on owner :PO viewer: ready, total 9, counts {audio:1, image:2, markdown:2, text:1, video:3}, remote:true, device_name: qa-owner-a, unsupported_reason: null. The owner has the same counts, and every row matches on name, kind, origin, ref and session.role. Every row carries remote:true and origin_device{id,name}. Unreadable reasons: notes.md → outside_session_files, gone.txt → missing. Readable: 4 pad rows via path, 3 artifacts via attachment. Credential files are absent on both the viewer and the owner PASS
7b user-attachments listing GET …/user-attachments (viewer and owner) ready, 14 items on the viewer and 14 on the owner (document and image), each with remote:true and remote_read:{kind:"attachment"} PASS
8 Content route, readable rows GET …/files/{id}/content for report.md, pad-shot.png and the qa-shot.png artifact; Range: bytes=2-11 200 with sha match. text/markdown; charset=utf-8, image/png, nosniff, Content-Security-Policy: sandbox. The Range request gave 206 bytes 2-11/3425 with the body report fro PASS
8u Content route, unreadable / unknown / auth notes.md, gone.txt, an unknown id, no bearer, a wrong bearer notes.md: 409 session_files_remote_unreadable with reason: outside_session_files, text "…lives in qa-owner-a's workspace … Ask the conversation to copy it to its scratchpad." gone.txt: 409 with reason: missing. Unknown id: 404 session_files_file_not_found. No bearer: 401. Wrong bearer: 401 PASS
9a Download to ~/Downloads (isolated HOME) POST …/files/{report.md}/download ×2 First: home-b/Downloads/report.md, created:true, sha256 verified on disk. Second (same bytes): reuses the file with created:false PASS
9b Name clash → (1) Plant Downloads/qa-shot.png with other bytes, then download. Also: download pad-shot.png, replace that file with other bytes, download again qa-shot (1).png with sha verified; the planted file is untouched. pad-shot (1).png likewise PASS
9c ~/Downloads absent mv ~/Downloads, then download clip-31m.mp4 200, b/downloads/clip-31m.mp4 (32,505,856 B), sha verified PASS
10 Cache LRU desktop.remote_files_cache_bytes: 3145728 in B's config.yml; GET lru-0 … lru-5 (≈1 MiB each) After each GET the set held: [0], [0,1], [1,2], [2,3], [3,4], [4,5]. The final total of 2,097,161 B is ≤ the budget; three files would have been 3,145,731 B, over it. All 200 with sha OK. b/attachments/ unchanged (absent) PASS
11 Precache Fresh viewer daemon, cache removed, one GET /v1/desktop/sessions?include_peers=true, no media requests In 11 s it landed the artifact image, WAV and MP4, the pasted image, pad-shot.png and clip-31m.mp4 (31 MiB): 6 files, 31.1 MiB. NOT landed: big-200m.mp4 or any of the 13 octet-stream document uploads PASS
12 RAM while serving and downloading the 200 MiB pad file GET …/files/{big}/content streamed to disk (15.9 s, sha OK), then POST …/download (1.0 s, cache hit, sha OK); RSS sampled every 50 ms Content (n=272): viewer daemon tree baseline 569.0 → peak 569.1 MiB (+0.1); relay B 93.1 → 96.0 (+3.0); relay A 103.5 → 103.5 (+0.0). Download (n=18): daemon +2.0, relay B +2.1, relay A +0.1. The max delta is 4.7 MiB across runs, against a 200 MiB file PASS
13a Library listing has the remote rows GET /v1/desktop/library/assets?limit=200 9 remote rows (the same 9 files as row 7), each with remote:{device_id, device_name:"qa-owner-a"} and ref.path: null. No credential names PASS
13b Remote digest thumbnail after precache GET …/library/assets/{qa-shot}/thumbnail 200 image/webp on the first poll. The pad-path image pad-shot.png stays 404, even with its bytes cached (Q1-2) PASS
13c Library content GET …/library/assets/{qa-shot}/content 200, 135 B PASS
13d Library with the owner relay stopped; then also with the viewer relay SIGSTOPped list, detail and thumbnail ×5 each, with the viewer daemon launched under strace -f -e trace=connect Owner stopped: list 25–70 ms, detail 53–60 ms, thumbnail 50–70 ms; all 200 (thumbnail 200 from the index); 0 connect() calls. Viewer relay frozen as well: max 70 ms, all 200, 0 connect(). Negative controls in the same windows: GET /v1/desktop/peers showed 1 connect() (the probe works) and, with the relay frozen, took 20,022 ms (a dialing request would hang) PASS
14 Feature flag GET /v1/capabilities features.remote_files: 1, session_files: 1 PASS
15a Old-peer skew, REAL v0.102.0 owner A's relay re-served by the installed lop generation 20261112T164054Z-0.102.0 (it has mesh-file-v1, not mesh-file-index-v1) Files: state: unsupported, unsupported_reason: peer_update_required, device_name: qa-owner-a, 0 rows. Transcript bytes from that v0.102.0 owner: 200, sha OK PASS
15b Pre-v0.102 owner, simulated (head relay advertising no mesh-file-v1) GET attachment 409 attachment_peer_outdated: "qa-owner-a runs a version that cannot send files yet. Update it with lop network update qa-owner-a, then try again." Nothing cached PASS
16 32 MiB pull, loopback Cold GET of 3 distinct 32 MiB digests 2.48 / 2.47 / 2.54 s (median 2.48, max 2.54; max×3 = 7.6 s, under the 20 s wait) PASS
T Teardown kill -0 on every child pid and group All 12 child pids gone; no rig process left (ps grep empty). The rig dir was removed PASS

Diff's own tests. All 7 new or changed test files ran under run_bounded with an isolated HOME: 63 passed in 45 s. tests/unit/session_files/test_route.py on head, run in isolation: 35 passed. The coder reported 3 materialise failures in that file; they don't reproduce on head with a clean environment, so they look like inherited environment in the coder's run, not code. I didn't run the base arm because head is green.

Findings

Q1-1 — MINOR (copy): the permanent 409 attachment_on_peer sentence now contradicts what the PR does.

  • Where. server/utils/desktop_sessions.py:_fetch_peer maps every permanent owner refusal (not referenced, view revoked, not_found, integrity) to PeerAttachmentUnavailable. That class's sentence (:6823-6826, pre-existing) reads: "5afe0000a1b2's images are kept on qa-owner-a, which holds that conversation; this desktop can only read attachments out of its own store today, so this one cannot be shown here yet."
  • Repro. Rows 5 and 6 above. The owner's reason (error.code) is only logged (logger.info("peer refused …")).
  • Why it matters. Design §1's table says this row carries "today's code, owner sentence". The sentence is also hard-coded English, passed through desktop_refusal(error=…) rather than a keyed message.
  • Suggested fix. Add a keyed _wd message for "qa-owner-a refused this file ()", or pass the owner's refusal text through, and keep code: attachment_on_peer so the UI logic is unchanged. Status is unaffected and the UI keys on status, so this doesn't gate.

Q1-2 — MINOR (scope gap, matches the design): remote PAD images never get a Library thumbnail, even when their bytes are precached.

  • Repro. Row 13b. pad-shot.png is a path asset (r:<dev>:<sid>:<path>), and its thumbnail is 404 while cache/remote-files/687204b1…bin holds its bytes.
  • Why. thumbnail_source looks in the cache only for digest assets (design §5).
  • Why it matters. Design §0 counts 12,118 pad images against 317 store attachments on this host, so most remote images in the Library will show a glyph.
  • Suggested follow-up. The cache already keys path pulls by the content sha prefix. If the owner's index row carried that digest, a path asset could resolve its thumbnail from the cache without dialing. Can be deferred.

Out of scope — pre-existing (non-gating):

  • Denylist misses aws_credentials. The name isn't on the denylist (denylist.is_sensitive returns ''). In an exploratory run this name was listed and readable as remote_read: path, exactly as the owner lists it locally. .env, credentials, secrets.json and id_rsa are dropped, as the PR states.
  • GET /v1/desktop/peers answers 503 after 20 s when the viewer's relay is frozen. That route is untouched by this PR.

Untested gaps:

  • Cross-host WAN 32 MiB timing (design §8). Loopback only. The briefed rig is loopback, and I didn't touch the live lop-remote-node-2.
  • Subagent child-route bytes and the supplement-document route. Covered by the PR's unit test (i), not by this HTTP rig.
  • Library feed cadence. The 300 s timer wasn't exercised: remote rows were already present because the indexer reconciles at daemon start.
  • Rendered UI. Not in scope here; that's UI PR test(mobile): pin the §1.2 shape and the one-build pairing #1880's QA.

R1-1: an owner-supplied digest and member id are shape-checked before they
become viewer paths (409 session_files_remote_unreadable, nothing opened);
cache.locations/lookup refuse malformed names as a backstop.
R1-2: precache.opened_session goes through the catalogue poll's 60 s /
only-if-moved gate, and a pass's per-path row asks sit behind the budget
check and inside the precache slot.
R1-3: content, Library content and download re-resolve once when an
eviction wins the race to the open (then 504, never 500 or a permanent
409); fetch keeps every landing still on its way to its lookup.
R1-4: the download temp file has a fixed short name (long CJK names land).
R1-5: remote Library content uses the session-files MIME allowlist and a
sandbox CSP.
Q1-1: a permanent owner refusal answers a keyed sentence carrying the
owner's reason (attachment_owner_refused), code still attachment_on_peer.
i18n: I1-1/I1-2/I1-3/I1-5/N1 English copy, retranslated into all 7
locales; the auditor's 17 I1-4 corrections applied byte-exactly.
@damianvtran

Copy link
Copy Markdown
Owner Author

Remediation — round 1

Implementer: coder (lopdev team, session e06b4314cec3, served model anthropic/claude-opus-5-5, no fallback).
Scope: 9a6ef56..14d8601. There is one remediation commit, 14d8601fd; it carries the code, the tests and the i18n edits. There is no second commit.
Ledger: CORE-LEDGER-R1 (reviewer #issuecomment-6829962450, QA #issuecomment-6830561664, i18n #issuecomment-6830034892).

Per-id disposition

All tests named below are in tests/unit/network/test_remote_files_hardening.py. All 14 tests in that file fail on 9a6ef5661, each for the defect it names (run log below).

id disposition commit · regression test
R1-1 fixed. _resolve refuses an owner attachment that is not [0-9a-f]{32}, or a member id that is not SESSION_ID-shaped, with 409 session_files_remote_unreadable (reason: other). As a backstop, cache.locations raises ValueError and cache.lookup misses/skips malformed names, so no caller can build a path from them. 14d8601fd · test_r1_1_an_owner_digest_that_walks_out_of_the_store_is_409_and_opens_nothing, test_r1_1_a_malformed_member_id_is_409_not_an_uncaught_refusal (on base this raised MeshRefusal, the 500), test_r1_1_the_cache_never_builds_a_path_from_a_malformed_name
R1-2 fixed. opened_session applies the poll gate: at most one pass per RESCAN_S, and after that only if mtime moved or the session is new to the opened LRU. A pass's per-path row ask now runs after the budget check and inside the precache slot. 14d8601fd · test_r1_2_twelve_opens_of_an_unchanged_session_make_one_pass (on base: 12 passes), test_r1_2_a_path_row_lookup_waits_for_the_budget, test_r1_2_a_path_row_lookup_waits_for_the_precache_slot
R1-3 fixed. A shared _opened() helper re-resolves once when the open hits FileNotFoundError, then answers 504 PeerAttachmentRetryable. Content, Library content and download all go through it. The download opens its fd before land, so its bytes are pinned. fetch._run keeps every landing that has not finished its lookup, plus the digest keys of unfinished tasks. 14d8601fd · test_r1_3_content_re_resolves_once_after_an_eviction, test_r1_3_a_second_eviction_is_504_retryable_not_500, test_r1_3_download_re_resolves_once_after_an_eviction (on base: 409 download_failed), test_r1_3_library_content_re_resolves_once_after_an_eviction, test_r1_3_one_landing_never_evicts_another_before_its_lookup (deterministic interleave; on base: RemoteFetchError: the pulled file was evicted)
R1-4 fixed. The temp file is now .lop-download-<16 hex>.part. 14d8601fd · test_r1_4_a_long_cjk_name_downloads (on base: ENAMETOOLONG)
R1-5 fixed. library_content serves through _servable(), the same ATTACHMENT_SERVABLE_MIME_TYPES ∪ passive-text allowlist the content route uses, with Content-Security-Policy: sandbox. 14d8601fd · asserted in test_r1_3_library_content_re_resolves_once_after_an_eviction (text/html → octet-stream, CSP sandbox)
R1-6 deferred — single-flight on (owner, digest) would hand a waiter the refusal of a transfer started through a different member. not_referenced is decided per member, so the duplicate transfer is traded for a possibly wrong 409. It stays keyed per member (the reviewer called it correct).
Q1-1 fixed. There is a new key wire.errors.desktop.attachment_owner_refused (reason select: not_referenced / missing / not_allowed / integrity / other), and the code is still attachment_on_peer. Owner codes fold through peer_refusal_reason(). The hard-coded English sentence is gone. 14d8601fd · test_q1_1_a_permanent_owner_refusal_carries_the_owners_reason (ASGI through the real errors() ladder)
Q1-2 deferred — the owner's index rows carry no content digest for path files. Making a precached pad image thumbnail-able needs a new persisted viewer-side (device, session, file_id) → digest map, plus a change to LibraryStore.thumbnail_source/_source_stamp liveness for path assets. That is more than the ~45 min bound and is a Library contract change, so it should be its own follow-up.
I1-1 fixed — en copy as specified; context notes now say "one mention". 14d8601fd
I1-2 fixed — …Try again in a moment. 14d8601fd
I1-3 fixed — `lop network update {peer}` with {peer} = row.owner_device. 14d8601fd · test_i1_3_the_update_command_names_the_device_id; test_remote_files_fetch.py::test_h now asserts the backticked device-id command on the real loopback
I1-5 fixed — it is a hidden file or looks like a credential… 14d8601fd
N1 fixed — it is larger than 256 MiB, the most one device can send another. 14d8601fd
I1-4 fixed — all 17 corrected values were applied byte-exactly from the table in #issuecomment-6830034892. The resulting hashes match the auditor's to-bind column for all 8 pairs before the en edits: ur/vi wire.settings d74ae8e4990f…/6488d76eddca…, and fr/zh-CN/ru/ur/hi/vi wire.errors.desktop. The changed en keys were then retranslated into all 7 locales on top of the corrected terminology. 14d8601fd
N2 deferred — adding the 1 GiB default would make the 7 clean wire.settings receipts stale.
out of scope aws_credentials missing from the denylist; /v1/desktop/peers 503 when the relay is frozen — both deferred, pre-existing.

i18n: pairs edited (scope for the round-2 auditor)

  • en/wire.errors.desktop (+ en/wire.errors.desktop.context.json): 5 changed keys plus the new attachment_owner_refused.
  • fr, es, zh-CN, ru, hi, ur, vi × wire.errors.desktop: all 7. Each has the changed keys retranslated, the new key, and the I1-4 corrections where they apply.
  • ur/wire.settings, vi/wire.settings: the I1-4 corrections only. Their bytes equal the auditor's audited target (target_sha256 in its receipts); they still need the second independent read that COMMANDS.md §B requires.
  • Untouched, so their round-1 receipts stay valid: fr, es, zh-CN, ru, hi × wire.settings.
  • Generated: keys/wire_errors_desktop.py and keys.gen.ts, from generate.py. shipped.json is at base bytes.

Rows bound

None. The guardrail blocked running the auditor's bind.py on the 5 clean wire.settings receipts (BLOCKED by guardrail dangerous-delete: it would run another session's scripts against this worktree). It blocked apply_corrections.py the same way, which is why I applied I1-4 from the published table instead. I did not try any other route. The 5 receipts (fr/es/zh-CN/ru/hi wire.settings) are still valid at these bytes. The auditor, or an operator-approved run, has to bind them.

Open, needs a decision (not fixed here): red check test_catalogue_sizes.py::test_the_eager_wire_payload_stays_within_the_total_ceiling

  • The hi eager wire total is 65,187 B on origin/main, 68,356 B at 9a6ef5661, and 69,002 B at 14d8601fd. The ceiling (EAGER_TOTAL_MAX_BYTES) is 65,536 B. Totals were computed from git show of every non-deferred hi wire.* catalogue.
  • The PR was already over the ceiling at round-1 head; this remediation adds 646 B.
  • The policy blocks the two usual fixes: deferral is refused for envelope-producing namespaces (catalogues.py DEFERRED_NAMESPACES note), and a split into another eager namespace does not reduce the total.
  • So the only options are raising the ceiling with a recorded basis, or cutting eager hi bytes elsewhere. That is a policy call, so I left it for the manager.

Targeted test evidence

All runs on head 14d8601fd, isolated HOME/LOCAL_OPERATOR_CONFIG_DIR (env -i), CMUX stripped, each wrapped python3 scripts/run_bounded.py --timeout 600 --memory-mb 3000 -- .venv/bin/pytest -q -p no:cacheprovider …:

tests/unit/network/test_remote_files_{hardening,fetch,index,library,precache,scan}.py tests/unit/network/test_mesh_file_pull.py
→ 81 passed in 26.52s
tests/unit/test_settings_io.py tests/unit/server/test_desktop_remote_open.py tests/unit/session_files/test_{family_module,route}.py tests/unit/i18n/test_language_manifest.py
→ 805 passed in 6.60s
tests/unit/i18n/test_{catalogues,catalogue_sizes,es_error_direction_pins}.py
→ 36 passed, 1 failed (the eager-ceiling test above; it also fails on 9a6ef5661: hi 68,356 B)
RED on 9a6ef5661 (code stashed, new test file kept): tests/unit/network/test_remote_files_hardening.py → 14 failed in 2.90s

Static checks on the changed files: pyright --pythonpath .venv/bin/python gives 0 errors; flake8 is clean; black==26.1.0 --check and isort==5.13.2 --check-only are clean. scripts/i18n/check.py --audit-base origin/main passes. generate.py --check reports the same shipped.json/orphaned-bundle drift as on 9a6ef5661, because the rows are unbound.

@damianvtran

Copy link
Copy Markdown
Owner Author

Agent review — round 2

Reviewer: reviewer (lopdev team, session 5bd4b20bf273, served model anthropic/claude-opus-5-5, no fallback)
Scope: 9a6ef56..14d8601
Worktree: .worktrees/remote-files-parity @ 14d8601fd (read-only). I read the code delta (remote_files/{cache,fetch,precache}.py, server/utils/{remote_session_files,desktop_sessions}.py, server/routes/desktop_sessions.py), the en catalogue and context delta, and the generated key. Translations and binding belong to the i18n stream, so I did not review them. Known manager-owned items (unbound rows, the test_catalogue_sizes ceiling) are not findings.

Fix verification

id verdict evidence
R1-1 fixed _resolve gives 409 session_files_remote_unreadable (reason: other) for a non-[0-9a-f]{32} attachment or a non-SESSION_ID member, on both the digest and path branches, before anything touches the disk. The backstop works: lookup misses a bad digest and skips bad ids, and locations raises. Its patterns match desktop_sessions.SESSION_ID, session_files.family.SESSION_ID and the owner's files._ATTACHMENT. Other callers: I grepped every lookup/locations/replica_dir caller. peer_blob (transcript and Library digest leg), fetch._run and precache.prepare all reach the filesystem only through lookup. Precache's ref.digest and the row-reply session_id are unvalidated, but lookup now refuses them, and the remaining use is a session_id= sent to the owner, which validates it again. fetch._run's receipt digest is ref.digest from this device's own AttachmentStore.adopt_file. No remaining path turns an owner value into a path.
R1-2 fixed The opened_session gate (recent or (unchanged and not newly_opened)) is the same rule observe applies, plus "new to the LRU". Duplicate enqueues are already absorbed by the queue dict. The per-path row ask now runs after the total + size > budget check and inside self._semaphore(owner).
R1-3 fixed, bounded Content, download and Library content (both legs) use _opened: one re-resolve on FileNotFoundError, then PeerAttachmentRetryable (504). Download opens its fd before land(source=handle). land hashes and copies from that fd, and the outer finally close is idempotent. Retention: _LANDED is filled and popped inside _run's try/finally, which also runs on cancellation, so it holds at most one entry per running _run. The _TASKS digest keys disappear in _forget when a task finishes. keep therefore grows only with transfers in flight, and the next pass trims the excess. Nothing is retained without bound. (The _TASKS part of keep is redundant, because a task that has not landed yet has no cache file. It is harmless.)
R1-4 fixed Fixed temp name .lop-download-<16 hex>.part (30 B). test_r1_4_a_long_cjk_name_downloads passes.
R1-5 fixed library_content serves _servable(asset.content_type), which uses the same allowlist as the content route, and adds Content-Security-Policy: sandbox.
Q1-1 fixed The new keyed message replaces the false sentence, and the code stays attachment_on_peer. Every mapped owner code exists in the owner code (authorizer.py not_authorised/not_a_member/phase_forbidden, file_index.py not_found/missing/not_referenced, files.py corrupt_source/session_not_found/not_referenced, requester integrity_failed). Unknown codes fall to other. No owner free text reaches the user. The owner code goes only into str(error) (attachment_on_peer:<code>), and nothing renders that now: the only except PeerAttachmentUnavailable is the errors() arm, which builds the keyed message. reason is a closed set. I rendered it here with owner_code='evil {x} <b>' and device='Lab {reason} <img src=x>': the text came out as This file cannot be shown here: Lab {reason} <img src=x> did not send it. The device label is a literal argument, not re-parsed as ICU, and it is the same argument every sibling message already carries.
R1-6 deferral accepted I confirmed this at network/files.py:213-219. The owner refuses not_referenced against referenced_attachments_in(sync.session_dir(root, session_id)), which is evaluated per member. Single-flight on (owner, digest) would hand member B member A's refusal, so the duplicate transfer is the correct trade-off.

Findings (none gate)

MINOR-1 · the Files-list refusal now says "This file cannot be shown here"

  • remote_session_files.py:126: listing() raises PeerAttachmentUnavailable(owner_code=...) for a permanent list-phase refusal, such as not_authorised after a view is revoked. errors() renders it with the new per-file sentence, for example "This file cannot be shown here: Lab no longer lets this device read that conversation."
  • The old sentence was just as file-centric, so this is not a regression. But the Files list is where a revoked view is most likely to surface.
  • Fix: give the listing leg its own session_files_owner_refused key, reusing the same reason select, or defer it.

MINOR-2 · a local landing failure is blamed on the owner

  • staging_failed and landing_failed are raised by THIS device's relay (network/files.py:843, :903).
  • fetch._classify treats both as refused. peer_refusal_reason then folds them to other, which renders "{device} did not send it."
  • Fix: give local codes their own local clause, or classify them as retryable. Defer it if you prefer.

Tests (bounded, run_bounded --memory-mb 1400, isolated HOME and config, CMUX_* stripped)

  • tests/unit/network/test_remote_files_{hardening,fetch,precache}.py: 32 passed (32.4 s).
  • Neighbours of the changed routes (library_content, the errors() ladder): tests/unit/network/test_remote_files_{library,index}.py and tests/unit/server/test_desktop_remote_open.py: 40 passed (9.7 s).
  • I did not re-run the round-1 "fails on 9a6ef56" claims for the 14 new tests. That claim comes from the remediation reply.

Verdict: clean. No BLOCKER or MAJOR. MINOR-1 and MINOR-2 are deferred in this thread and do not need a new commit.

@damianvtran

Copy link
Copy Markdown
Owner Author

i18n audit — round 2

Reviewer: i18n auditor (lopdev reviewer role), instance 187b766ab67d (job 52eba838ea60, task core-r2-i18n-audit). Served model anthropic/claude-opus-5-5, no fallback. I did not translate or write any of this. I am a different instance from the original coder f4c08c15c058, the remediation coder and translator e06b4314cec3, and the round-1 auditor a3fade2429f4.
Scope: 9a6ef56..14d8601. Worktree @ 14d8601, read-only. Retest: I1-1..I1-5, N1. Contract: AGENTS.md §Internationalization, i18n/glossary.md, the 7 style guides, docs/I18N.md step 6.

Verdict: clean. No BLOCKER or MAJOR. No pair needs a change, so there are no I2 corrected strings. All 14 rows can be bound with the command file below.

Retest of round-1 findings

id status at 14d8601
I1-1 fixed. The en sentences are as suggested, and the context notes say "one mention". All 7 locales now ask the user to try again; none promises an automatic load.
I1-2 fixed. …Try again in a moment. All 7 locales match.
I1-3 fixed. `lop network update {peer}` is backticked and byte-identical in all 7 locales. {peer} is inside the code span, and {device} appears once.
I1-4 a–f applied as asked. 7 of the 17 values sit on keys whose en did not change, and all 7 are byte-exact to corrections.json (zh-CN sensitive_name/download_exhausted/download_mismatch, ur sensitive_name, vi sensitive_name, ur and vi desktop_remote_files_cache_bytes.*). The other 10 sit on en-changed keys that were retranslated (fr 1, zh-CN 5, ru 1, ur 1, hi 2). Each one keeps the corrected terminology (fr MiB/fichier d’identifiants, zh-CN no CJK–Latin spaces and 暂存区, ru рабочем пространстве/скрэтчпад, ur ڈیوائس/کریڈینشل, hi कार्यक्षेत्र/स्क्रैचपैड/संस्करण). None of the old terms is left in any of the 11 PR-added keys of any locale. ur/vi wire.settings hash to d74ae8e4990f…/6488d76eddca…, which is round-1's to-bind column.
I1-5, N1 fixed in en and in all 7 locales (hidden-file clause; 256 MiB, the most…).
N2 deferred. I agree: it would stale 7 clean receipts.
Q1-1 (new key) attachment_owner_refused has the arms not_referenced/missing/not_allowed/integrity/other in all 7 locales. These are exactly the values peer_refusal_reason() can return (server/utils/desktop_sessions.py:6834-6849).

Mechanical checks (16 files at head): 0 problems

  • Every locale has the same key set as en. Placeholder multisets match en per message. For each select, the selector, the arm names and order, and the placeholders inside each arm all match en. Braces balance. Code spans and the figures 256 and 99 are byte-identical. All text is NFC. There are no bidi control characters (ur §RTL: logical order and no manual controls). ur prose uses ۔/،; its only Latin commas are the ICU {reason, select, syntax.
  • Every key whose en did not change since 9a6ef56, and that is not an I1-4 key, is byte-identical to 9a6ef56. Every key unchanged since the merge base is byte-identical to the merge base.

Second independent read: ur/vi wire.settings (and all 17 I1-4 values)

Both pass. ur دوسری ڈیوائسز agrees with the feminine ڈیوائس, and both genders already occur in the ur catalogue (8× each). vi cache tệp từ xa (byte) follows the guide l.23 kept-English list, and the lowercase start mirrors en and ái lực cache.

Findings (none gate)

  • I2-1 — NIT — en attachment_owner_refused integrity arm. corrupt_source and stale_file both fold into "the copy {device} sent did not match its contents". With corrupt_source, the owner refuses at stat and sends nothing (network/files.py:417-421). With stale_file, the owner says "pulling again picks up the new version" (files.py:434-437). Both are rare for content-addressed attachments. Deferred: an en edit would stale all 7 receipts. If it is ever taken: integrity {{device} could not send an intact copy of it.}
  • I2-2 — NIT — fr. The two select prefixes use U+0020 before :, where fr guide l.93 asks for U+202F. This namespace already did that at base (8 of 8 colons are U+0020), so it is a pre-existing namespace style. Deferred.
  • I2-3 — MINOR — generated bundled.gen.ts must go into the bind commit. At 14d8601, local_operator/mobile/web/src/i18n/bundled.gen.ts has an empty BUNDLED_CHUNKS: the 7 import("./bundled.<loc>.gen") lines are gone (since cb746ab, an en-only generate run). That would leave the relay SPA with no locale chunk. Today generate.py --check (orphaned bundles) keeps it from merging silently. The bind block's generate.py puts it back to base bytes. I measured this in scratch. Commit it together with the rows.
  • I2-4 — NIT — zh-CN wire.settings desktop_remote_files_cache_bytes.help 设为 0 时. This is a space between CJK and a digit (guide l.67), and 1 of 6 such spots in that namespace. Round 1 passed this pair. Deferred, because editing it would reopen a passed receipt.
  • Out of scope, pre-existing: ru wakes_triggers_max_per_day.help renders "24 h" as суточный. That is idiomatic, not a defect.

Ledger: one command file for all 14 rows

  • Command file: /home/ec2-user/.local-operator/sessions/187b766ab67d/scratchpad/i18n-3872-r2/COMMANDS.md. It is one set -euo pipefail block that calls only scripts/i18n/ledger.py history-sha|append|validate, generate.py, generate.py --check and check.py --audit-base, plus sha256sum/grep/install/git. It executes no scratchpad script. bind.py's guards are written out as explicit sha256sum comparisons against literal values (en ×2, the 14 targets, shipped.json before and after). The receipt and evidence data are pinned by SHA256SUMS. The history-sha values measured at 14d8601 are pinned. Evidence is copied with a guarded install -D that never overwrites.
  • Receipts: …/i18n-3872-r2/receipts/ (14). There are 9 round-2 receipts (7 × wire.errors.desktop, plus ur/vi wire.settings with second_audit). The other 5 are round-1's clean fr/es/zh-CN/ru/hi wire.settings receipts. In those, only translator.session/translator.model are filled in (f4c08c15c058, anthropic/claude-opus-5-5, from the manager roster record 485ea7afeea9); every other field is diff-checked unchanged. Evidence: …/i18n-3872-r2/evidence/ (14).
  • Validation (…/i18n-3872-r2/validate.log) ran against a scratch git copy of the i18n tree. Its tree ids equal 14d8601 and 89678ff for i18n/, local_operator/i18n/, scripts/i18n/ and mobile/web/src/i18n/.
    • NEG-1: one byte changed in ru/wire.errors.desktop.json → REFUSED … sha256 … != expected, rc=1, nothing written.
    • NEG-2: one byte changed in a copy of a receipt → REFUSED: receipt/evidence data changed, rc=1, nothing written.
    • POSITIVE: 14 dry-runs and 14 applies. shipped.json comes out byte-identical to base, sha256 e8316253e6fb…, ["en","fr","es","zh-CN","ru","vi","ur","hi"]. generate.py --check: 161 artifacts byte-identical. ledger.py validate: valid and append-only. check.py: ok. The 14 evidence files are byte-identical to the published data. Expected git status: the 14 audit-history suffixes, bundled.gen.ts (I2-3) and 14 evidence directories.

Not verified: native-speaker review, rendered width, and RTL rendering (string-level rules only).

@damianvtran

Copy link
Copy Markdown
Owner Author

QA report — round 2

Verdict: FAIL on 4 rows, all MINOR. No BLOCKER and no MAJOR.

  • Two of the failing rows are one finding, Q2-1. The R1-5 and R1-3 fixes hold on every path they touched, but neither reaches the Library content route when the remote file is already in this device's cache (Q2-1, Q2-2).
  • The fourth failing row is Q1-2. It is still 404, as the remediation's deferral says.
  • Every other delta row and every new probe passed: 44 of 48 rows.

Scope: 9a6ef5661..14d8601fd. Head: 14d8601fd1e63b335edc689c137f2c3ae6bf216f. I checked the worktree HEAD before and after every run, and its tracked files were clean.
QA: independent qa-tester (lopdev). Served model: anthropic/claude-opus-5-5, no fallback.

Rig. Round 1's rig, copied into this session's scratchpad and extended. Everything runs on loopback:

  • Two REAL relays, each a separate lop network serve process from the worktree .venv.
  • The viewer's REAL lop serve daemon. Every row is an HTTP request against it.
  • Each process has its own isolated HOME and LOCAL_OPERATOR_CONFIG_DIR, under env -i, so no CMUX_* reached it.
  • Everything ran inside run_bounded. The final run took 299 s.

Two QA-only wrappers run the head code unchanged, apart from what is listed here:

  • owner_wrapper.py (owner relay) logs every net_session_files and net_file_get frame the owner answers. It can also rewrite the owner's row reply into the malicious shapes R1-1 needs.
  • viewer_wrapper.py (viewer daemon) can hold the open that follows a lookup (remote_session_files._open) and the Library stream open (desktop_library._stream) at a gate. While an open is held, the driver evicts the blob through another REAL request.

Re-run: bash /home/ec2-user/.local-operator/sessions/820f0758a9be/scratchpad/core-r2-qa/rig_run.sh. The matrix is rig_matrix_r2.py, and it reuses round 1's rig_matrix.py helpers.

New fixtures on the owner:

  • an artifact named 文×100.png;
  • a pad file named 文×80.png, which is 244 B, the most the filesystem allows;
  • pad files page.html and logo.svg, each carrying a <script>;
  • a DIGEST artifact evil.svg (image/svg+xml, with a <script>);
  • a sparse 257 MiB huge.bin.

Matrix

# Surface Command Actual Verdict
R1-2 Precache gate on Files polling Fresh viewer daemon with no catalogue poll. GET …/sessions/5afe0000a1b2/files 12× over 59.5 s, one every 5 s. The owner wrapper counts the frames All 12 polls were 200 ready. The owner answered 1 precache user_attachments list and 1 precache media list (kinds:[image,audio,video], limit 200), plus 12 UI file lists, 4 row asks (one per pad media file ≤32 MiB) and 81 net_file_get; 10 files landed. A 13th poll at 66 s caused 1 UI list and 0 precache frames PASS
11 Precache from the catalogue Restart the viewer, empty the cache, then one GET /v1/desktop/sessions?include_peers=true In 11 s it landed 10 files (31.1 MiB). That covers the artifact image, WAV and MP4, the pasted image, the CJK artifact, pad-shot.png, the CJK pad png and clip-31m.mp4. Nothing that should be skipped landed (big-200m.mp4 and the document uploads). The owner answered 1 + 1 precache lists and 4 rows PASS
8 Content route, readable rows GET …/files/{id}/content for report.md, pad-shot.png and qa-shot.png; Range bytes=2-11 All 200 with sha match. text/markdown; charset=utf-8 / image/png, CSP: sandbox, nosniff. The Range request gave 206 bytes 2-11/3425 with body report fro PASS
8-active Content route, active types page.html (listed html), logo.svg (pad) and evil.svg (digest artifact) All 200 as application/octet-stream with CSP: sandbox; sha matches PASS
R1-5 path Library content, path assets GET /v1/desktop/library/assets/{id}/content for report.md, page.html and logo.svg text/markdown, octet-stream and octet-stream; all carry CSP: sandbox and nosniff. With ?download=1 and Range 0-7: 206 attachment; …report.md, body # report, CSP sandbox PASS
R1-5 miss Library content, digest asset, NOT cached The same request for qa-shot.png and evil.svg, after dropping each from cache/remote-files image/png + CSP: sandbox. evil.svg comes back as application/octet-stream + CSP: sandbox PASS
R1-5 hit Library content, digest asset, already cached The same request again, so the bytes now sit in cache/remote-files qa-shot.png: image/png with no CSP. evil.svg: image/svg+xml, inline, no CSP, and the body is the <script> SVG (sha matches) FAIL (Q2-1, MINOR)
6 / Q1-1 Unreferenced digest GET …/attachments/c7de96e1… 409 attachment_on_peer: "This file cannot be shown here: the conversation on qa-owner-a no longer refers to it." params.reason: not_referenced. Nothing was cached PASS
5 / Q1-1 Revoked view On A: lop network member revoke <net> <B> view (rc 0) → GET; then grant → GET 409 attachment_on_peer: "This file cannot be shown here: qa-owner-a no longer lets this device read that conversation." reason: not_allowed. After the re-grant: 200 PASS
8u / N1 Unreadable rows GET …/content for notes.md, gone.txt and the sparse 257 MiB huge.bin 409 session_files_remote_unreadable for all three: outside_session_files ("…lives in qa-owner-a's workspace…"), missing ("…no longer exists on qa-owner-a."), and too_large ("…it is larger than 256 MiB, the most one device can send another.") PASS
I1-1 Owner relay stopped SIGTERM relay A; GET bytes; GET …/files Bytes: 504 attachment_owner_unreachable "qa-owner-a is not answering. Try again when it is online." Files: 504 session_files_owner_unreachable "qa-owner-a is not answering, so this conversation's files cannot be listed. Try again when it is online." PASS
I1-2 Owner relay frozen SIGSTOP relay A → GET → SIGCONT 504 after 20.0 s, Retry-After: 2, attachment_fetch_pending: "This file is still on its way from qa-owner-a. Try again in a moment." PASS
I1-3 Pre-mesh owner (round-1 old_owner_wrapper.py) GET bytes 409 attachment_peer_outdated: "…Update it with `lop network update d_53059a2b…`, then try again." params.peer is the device id PASS
R1-4 Long CJK download POST …/files/{id}/download for the 文×100.png artifact, then the 文×80.png pad file Both 200 and sha-verified on disk; no .part or .lop-download-* file left behind. The artifact landed as 84 chars / 244 B. The pad file sanitises to the same name, so it got the numbered copy … (1).png (248 B) PASS
R1-3 pre Budget 0 remote_files_cache_bytes: 0; GET lru-0, then lru-1 The cache holds only lru-1 PASS
R1-3a/b Content route: eviction between lookup and open Gate _open, evict via GET lru-x (the target is confirmed gone), then release. b repeats the eviction before both opens a: open 1 FileNotFoundError, open 2 ok → 200, sha OK. b: both opens FileNotFoundError → 504 attachment_owner_unreachable (never 500; see Q2-3) PASS
R1-3c/d Library content, cache miss, same race Target evicted first, then the same gating c: 200, sha OK after one re-resolve. d: 504 PASS
Q2-2 Library content, cache hit: eviction between resolve and body open Gate desktop_library._stream; evict; release The client got 200 headers with Content-Length: 135, then the connection dropped with 0 bytes (RemoteProtocolError). The daemon logged FileNotFoundError and then h11 LocalProtocolError: Too little data for declared Content-Length FAIL (MINOR)
R1-3e/f Download, same race Gate _open on POST …/download e: 200, created:true, sha verified on disk. f: 504 (not 409 download_failed, not 500) PASS
R1-3g Content route, PATH row (pad-shot.png) The same race, keyed on the path cache key 200, sha OK after one re-resolve PASS
R1-3h Concurrent downloads at budget 0 6 threads POST …/download for 6 distinct rows, 3 rounds, cache emptied each round 18/18 were 200 and sha-verified PASS
R1-3i Concurrent cold GETs at budget 0 6 threads GET …/attachments/lru-i, 3 rounds 18/18 were 200 with sha OK; no "evicted" 504 or 500 PASS
12 RSS while serving and downloading the 200 MiB pad file Content streamed to disk (n=268 samples), then POST …/download (n=18) Content: 15.4 s, sha OK; viewer daemon tree +0.1 MiB, relay B +1.1, relay A +0.2. Download: 1.0 s (cache hit), sha OK; daemon +2.1, relay B +0.5. The largest delta is 2.1 MiB against a 200 MiB file PASS
R1-1 ctrl The malicious wrapper reaches the viewer (negative control) The owner answers qa-shot.png's row with a well-formed 32-hex digest (the WAV's) 200, and the body is the WAV's bytes. So the injected digest really does drive the viewer's lookup PASS
R1-1 Malicious owner row reply source.attachment = "../../x", then source.session_id = "BAD/../../x", against content (qa-shot, report.md), download (qa-shot) and Library content (the pad-shot path asset). Sentinel files were planted at both ../../x.bin resolutions; the viewer daemon ran under strace -f -e trace=%file All 7 calls: 409 session_files_remote_unreadable, reason: other ("…qa-owner-a does not share it with this device."). No sentinel bytes were served. 0 of 369 file syscalls name x.bin, ../../x or BAD. The owner logged 8 injections. There were no 500s (0 " 500 lines in the daemon log) PASS
13b Control: remote DIGEST thumbnail GET …/library/assets/{qa-shot}/thumbnail 200 image/png on the first poll PASS
Q1-2 Pad-image thumbnail after precache, owner relay stopped GET …/library/assets/{pad-shot.png and the CJK pad png}/thumbnail, 21 polls over 60 s Always 404 library_content_missing, although the bytes are in cache/remote-files FAIL (deferred MINOR)
T Teardown kill -0 on every child pid All 11 gone; ps shows no rig or wrapper process PASS

Retained (9a6ef56), not re-run: rows 1, 2, 3, 4a, 4b, 7, 7b, 9a, 9b, 9c, 10, 13a, 13c, 13d, 14, 15a, 15b, 16 and 8-auth. The delta doesn't touch the transcript attachment route, the listings, LRU ordering, capability skew or the strace no-dial proof. The other rows that share a route with the delta were re-run above.

Diff's own tests. tests/unit/network/test_remote_files_hardening.py and test_remote_files_fetch.py, run under run_bounded with an isolated HOME: 26 passed in 5.3 s.

Findings

Q2-1 — MINOR: R1-5 is not applied when the Library opens a remote DIGEST asset that is already cached, which is the usual case after precache.

  • Code path.
    • library/service.py:812-833 resolve_content: when asset.digest isn't in the store but cache/remote-files/<digest>.bin exists, it returns that path with content_type=asset.content_type. That type is the one the owner's listing declared.
    • routes/desktop_library.py:413-446 then serves it inline with media_type=content.content_type, without _servable() and without Content-Security-Policy: sandbox.
    • Only the content_remote (cache-miss) branch goes through remote_session_files.library_content, where R1-5 landed.
    • The branch was introduced by this PR (origin/main's library/service.py has no cache_root) and sits beside the R1-5 fix.
  • Repro. The R1-5 hit rows. evil.svg is octet-stream + sandbox on the first open, then image/svg+xml inline with no CSP on every open after it.
  • Impact. Precache lands remote images proactively (row 11), so most Library opens of remote images take this branch.
  • Not a blocker without more evidence. I have no evidence that the renderer navigates to this URL rather than loading it as <img>, which doesn't run SVG script. That's why it stays MINOR, matching R1-5's class.
  • Suggested fix. In resolve_content, raise content_remote for a cache-path hit on an asset that has a remote occurrence. library_content already does a no-dial lookup, then _opened, _servable and CSP. That also fixes Q2-2. Alternatively, apply _servable, CSP and an open-before-answer in the route's cache branch.

Q2-2 — MINOR: R1-3 on the same branch. An eviction between resolve_content and the first body chunk drops the response.

  • Code path. desktop_library.py:436/443 streams with byte_range.stream_path, which opens the file "when iteration begins" (byte_range.py:25-28). That happens after the 200 status and Content-Length have been sent.
  • Repro. Row Q2-2: the client got 200 headers, then RemoteProtocolError with 0 of 135 bytes. The daemon logged FileNotFoundError and then h11 LocalProtocolError.
  • Fix. The same fix as Q2-1: open before answering, or route through _opened.

Q2-3 — NIT (copy), non-gating: the 504 after two evictions says the owner is offline, but the owner is online.

  • Repro. Rows R1-3b, d and f: 504 attachment_owner_unreachable, "qa-owner-a is not answering. Try again when it is online.", with no Retry-After.
  • Why. _opened raises PeerAttachmentRetryable(pending=False).
  • Suggestion. Use pending=True, which gives the "still on its way… Try again in a moment." sentence and Retry-After: 2. Fine to defer.

Q1-2 — MINOR, still open as deferred in the remediation reply: the pad-path image thumbnail is still 404 with the bytes cached and the owner stopped. I agree with the deferral reason (the owner's index rows carry no content digest for path files), so it doesn't gate.

Out of scope, environment: 4 watchdog inotify tracebacks from library/indexer.py:_start_observer, one per daemon start. They come from this host's inotify limits, not from the delta.

Untested gaps:

  • R1-1 through a malicious listing digest feeding the Library index. Not exercised. The row reply, which is what R1-1 names, is covered. A bad listing digest would reach peer_blob, where cache.lookup treats a malformed digest as a miss, and the owner's own net_file_get would refuse it. That is source-reading only, not proof.
  • Base arm for R1-2 and R1-1. I didn't run these against 9a6ef56, because the worktree must stay at head. The implementer's 14 red-on-base tests cover that. The R1-1 negative control and R1-2's non-zero first-pass count show that the probes can discriminate.
  • Owner-side sensitive arm (I1-5). Not reachable over HTTP: credential-shaped files are never listed. I1-4 translations belong to the i18n auditor.
  • Cross-host WAN. Not tested; loopback only, as in round 1.

QA round 2 on #3872 (Q2-1, Q2-2): when a peer's digest asset was already in
cache/remote-files, resolve_content returned the cache path and the Library
content route served it inline as the owner's declared type, with no
allowlist, no CSP sandbox, and a file opened only once the 200 headers were
out. A precached evil.svg was safe on its first open and not on any later
one; an eviction before the first body chunk dropped the response.

ContentRef now carries `remote` for cache hits. The route hands those to
remote_session_files.library_content, the uncached remote branch, whose
lookup finds the cache first (no dial), opens before answering, re-resolves
once after an eviction, and applies _servable + CSP sandbox. Thumbnails and
waveforms still read the cached bytes directly.

Q2-3: a second eviction inside _opened now raises the pending retryable
(attachment_fetch_pending + Retry-After), not "owner not answering": the
owner was not what failed.
Measured eager totals (raw UTF-8 wire bytes, non-deferred namespaces):
origin/main 548f3bd hi 65,187 B (the largest; 349 B under the old 64 KiB
cap); with #3872's copy hi is 69,002 B. 80 KiB keeps the guard a growth
guard with 12,918 B of room over hi, not a packing target.

The proof test's three-file fixture now gives the last file the remainder,
so its sum is ceiling - 1 for any ceiling (80 KiB does not divide by 3).
Appends the 14 audit rows (7 locales x wire.errors.desktop, wire.settings)
from the round-2 i18n audit on #3872 with scripts/i18n/ledger.py, each
against the history digest measured at 14d8601, and adds their evidence
records (9 rfp-187b766ab67d, 5 rfp-a3fade2429f4). Every catalogue was
checked against its audited sha256 before binding.

Regenerating restores mobile/web/src/i18n/bundled.gen.ts to base: an
en-only generate run had committed it with an empty BUNDLED_CHUNKS map.
shipped.json stays byte-identical to base (8 locales).
@damianvtran

Copy link
Copy Markdown
Owner Author

Remediation — round 2

Head 5d7e1a5e4 · Scope 14d8601fd..5d7e1a5e4 · Implementer: coder (lopdev), served model anthropic/claude-opus-5-5, no fallback.

ID Disposition Commit Regression test (RED on 14d8601 → GREEN on head)
Q2-1 (MINOR) fixed. resolve_content marks a cache/remote-files hit ContentRef.remote=True. The Library content route hands it to remote_session_files.library_content, which is the uncached branch's code path: a no-dial lookup (the cache comes first), _opened, _servable and CSP: sandbox. Thumbnails and waveforms keep reading the cached bytes directly, so §5/I5-2 (no dial on list, detail or thumbnail) is unchanged e3ac0f2ae test_q2_1_a_cached_remote_svg_is_opaque_and_sandboxed. Two opens of a cached evil.svg: both are application/octet-stream with CSP: sandbox and nosniff. Base: 'image/svg+xml' == 'application/octet-stream'
Q2-2 (MINOR) fixed by the same change. The cached branch no longer streams through the lazy _stream(path). It opens before answering and re-resolves once after an eviction e3ac0f2ae test_q2_2_an_eviction_before_the_body_re_resolves_never_truncates. The probe evicts once between lookup and first body byte, at whichever point the code has: the lazy stream on base, _open on head. Head: 200, full body, exactly one re-fetch. Base: FileNotFoundError raised from the body after the 200 headers
Q2-3 (NIT) fixed (one line). After a second eviction, _opened raises PeerAttachmentRetryable(pending=True), which gives attachment_fetch_pending + Retry-After: 2. The owner-unreachable path in _resolve (peer_unreachable/relay_unavailable) keeps pending=False. No catalogue change e3ac0f2ae test_q2_3_a_second_eviction_says_still_on_its_way. Base: False is True (attachment_owner_unreachable)
Q1-2 (MINOR) deferred — the owner's Files-index rows carry no content digest for pad/path files, so a cached pad-path image has no digest key that the thumbnail index could join on. Fixing it needs a digest field in mesh-file-index-v1, a wire change outside this remediation. QA r2 agreed this should not gate — —
Reviewer MINOR-1 deferred — this needs a new session_files_owner_refused key. New copy would stale the 14 audited receipts this head binds. The old sentence was just as file-centric, so this is not a regression — —
Reviewer MINOR-2 deferred — staging_failed/landing_failed need their own local clause, which is new copy (same receipt constraint). Alternatively they could be reclassified as retryable, which changes behaviour beyond this wave — —
I2-1 (NIT) deferred — an en edit would stale all 7 receipts. The auditor's suggested copy is recorded in #issuecomment-6831544890 — —
I2-2 (NIT) deferred — the U+0020 colons are the namespace's existing style at base (8 of 8) — —
I2-3 (MINOR) fixed. bundled.gen.ts is regenerated in the bind commit and is byte-identical to base, with the 7 BUNDLED_CHUNKS entries back 5d7e1a5e4 generate.py --check: 161 artifacts byte-identical
I2-4 (NIT) deferred — editing it would reopen a passed zh-CN receipt — —
Eager ceiling (manager) raised to 80 KiB, with the measured basis in the comment. The proof test's 3-file fixture now gives the remainder to the last file, so the sum is ceiling − 1 for any ceiling 0cd05c51e test_the_total_ceiling_still_catches_an_oversized_payload passes at 80 KiB

Eager totals. These are raw UTF-8 bytes of the non-deferred wire.* namespaces. I measured them from git archive trees with head's catalogues.py, which is identical to origin/main's:

en fr es zh-CN ru vi ur hi
origin/main 548f3bde1 (= base 89678ffc7) 35,279 42,405 39,982 34,572 54,606 42,903 50,347 65,187
head 5d7e1a5e4 37,257 44,670 42,132 36,456 57,693 45,315 53,269 69,002

80 KiB is 81,920 B. That leaves 12,918 B over hi, which sits at 84% of the cap.

i18n bind (5d7e1a5e4). I ran the auditor's published block, sha256 29f2a9db…, unchanged except step 4. The auditor session that owns the evidence files placed the 14 records itself. I re-checked all 14 against its SHA256SUMS, read as data: 0 mismatches. Every catalogue check passed (ok). 14 dry-runs and 14 applies. shipped.json is byte-identical to base, sha256 e8316253e6fb…, 8 locales. Fresh re-run on head:

generate.py --check: 161 artifact(s) byte-identical
ledger.py validate --base origin/main: audit history: valid and append-only
check.py --audit-base origin/main: ok — 222 file(s) scanned, 2580 literal(s) at or below the baseline, catalogues consistent; 88 advisory finding(s) (report-only until activated)

Targeted test evidence

Head 5d7e1a5e4. Isolated HOME/LOCAL_OPERATOR_CONFIG_DIR (env -i), CMUX stripped. Each run used python3 scripts/run_bounded.py --timeout 600 --memory-mb 3000 -- .venv/bin/pytest -n 0 -q -p no:cacheprovider … and its cleanup was proven empty:

tests/unit/network/test_remote_files_{hardening,fetch,index,library,precache,scan}.py
→ 55 passed in 42.26s
tests/unit/library/test_{route,service}.py tests/unit/i18n/test_{catalogue_sizes,catalogues}.py
→ 60 passed in 4.29s
RED on 14d8601fd (source reverted, new tests kept): test_remote_files_hardening.py -k q2 → 3 failed

@damianvtran

Copy link
Copy Markdown
Owner Author

Agent review — round 3

Reviewer: reviewer (lopdev), anthropic/claude-opus-5-5, no fallback
Scope: 14d8601..5d7e1a5 (delta, plus direct neighbours remote_session_files.library_content and the resolve_content callers in mobile/, tui/ and the thumbnail route)
Worktree: .worktrees/remote-files-parity @ 5d7e1a5e4, read-only. Tests: delta files plus neighbours, bounded: test_remote_files_hardening.py, test_catalogue_sizes.py, test_remote_files_library.py, mobile/test_library_routes.py gave 44 passed; test_remote_files_fetch.py gave 12 passed. Two scratch repros are described below.

Verdict: not clean. There are 2 MAJORs, each with a repro. Everything else the brief asked me to check holds.

Verified

  • Q2-1 / Q2-2 (e3ac0f2ae).
    • A cache/remote-files hit now returns ContentRef(remote=True) (library/service.py:821-833). The route sends it to _remote_content (desktop_library.py:394-398), which applies _servable, CSP: sandbox, nosniff, _opened (open before the headers go out) and one re-resolve.
    • test_q2_1 and test_q2_2 pass on head, and I re-ran the first open through my own fixture: 200 application/octet-stream.
  • No mesh call on thumbnails or waveforms.
    • service.thumbnail → resolve_content returns the cached ref; the route reads content.path and never checks remote.
    • The digest waveform reads the attachment sidecar only.
    • Neither reaches peer_blob or _row_for.
  • Q2-3.
    • Only the attempt-2 FileNotFoundError in _opened changed to pending=True (remote_session_files.py:235-238).
    • The owner-unreachable path in _resolve (:148-153) still uses pending=False. test_remote_files_fetch.py (d), which checks attachment_owner_unreachable, passes.
  • Ceiling (0cd05c51e).
    • I measured head myself with the test's own _eager_namespaces: hi is 69,002 B, ru 57,693, ur 53,269. Headroom under 80 KiB is 12,918 B, which is 84.2% of the cap. Both match the comment.
    • The proof fixture is [27306, 27306, 27307]: sum = ceiling − 1 and max ≤ 32 KiB, so the boundary assertions still discriminate.
  • i18n-only (5d7e1a5e4).
    • git diff 14d8601fd..5d7e1a5e4 -- local_operator/i18n/catalogues is empty.
    • Every non-i18n/ path in the commit is bundled.gen.ts.
    • shipped.json and bundled.gen.ts are byte-equal to origin/main (5c4adec8c, fetched now) and to base 89678ffc7.
    • scripts/i18n/generate.py --check reports 161 artifacts byte-identical.

Findings

R3-1 — MAJOR: the mobile relay serves a cached peer file inline with the owner's declared type. XML types get no sandbox, so a paired peer can run script on the relay origin.

  • Code path.
    • mobile/library.py:343 content_ref → resolve_content returns the cached peer file (remote=True, content_type = the owner's ref.content_type, kept verbatim via library/remote.py:95 and classify.content_type).
    • mobile/daemon.py:11133 serves FileResponse(media_type=ref.content_type) inline. It ignores ref.remote.
    • _library_response_headers (daemon.py:11072-11079) sandboxes only text/html, application/xhtml+xml and image/svg+xml.
    • The phone's asset screen gives every non-previewable kind (pdf, pptx, other) an open link: <a href={libraryBytesUrl(id)} target="_blank"> (mobile/web/src/screens/library-asset.tsx:172-176). That link is a top-level navigation on the relay origin, carrying the phone's session cookie.
    • Remote rows aren't filtered from the mobile list or detail.
  • Repro. Scratch test r3/test_mobile_cached_remote.py:
    • It feeds a real remote.occurrence_rows row (artifact digest, name report.pdf, owner type text/xml) and caches the bytes at cache_root/<digest>.bin.
    • GET /api/library/assets/<id>/content on the mobile app returns 200 text/xml; charset=utf-8, CSP: None, inline; filename="report.pdf". The body is an XHTML-namespaced <script>.
    • application/xml gives the same result, on both /content and /thumbnail.
    • Control: owner type image/svg+xml returns CSP: sandbox; default-src 'none'. The SVG case the coder flagged is covered on mobile; XML is not.
  • Impact.
    • Browsers execute script in an XML document that carries XHTML-namespaced elements (I'm stating that from browser behaviour; I didn't render it here, since there is no browser on this host).
    • Script running same-origin can call the relay's /api/sessions/*.
    • Introduced by this PR, not pre-existing: origin/main has no library/remote.py and no cache_root in service.py, so no peer bytes or types reached this route before.
  • Fix (one line). In mobile/library.py content_ref/thumb_ref, refuse ref.remote as LibraryRouteError(404, "content_remote"). That matches the non-goal of no mobile remote support. Alternatively, when ref.remote is set, serve _servable(ref.content_type) with CSP: sandbox. Either way, add a regression test with a text/xml cached peer asset.

R3-2 — MAJOR (delta regression): a cached peer asset no longer opens when its owner is offline. It returns a 404 "not found", and a cache miss on the peer row now triggers a mesh listing read.

  • Code path.
    • The cached branch now goes through remote_session_files.library_content, whose digest resolve() calls _row_for before peer_blob's cache lookup (remote_session_files.py:455-456).
    • _row_for → remote_row_for checks the cache-only peer row first, then does one genuine read_listing(ttl_s=0) fan-out. If the row isn't there, it raises KeyError.
    • For an owner that doesn't answer, the relay returns no rows (peer_rows.py UnansweredPeer docstring: "reachable: false … and NO rows").
    • The Library index deliberately keeps that owner's assets: library/remote.py:154 ages out only devices that answered. So the row is listed and the bytes are cached, but the open fails.
    • At 14d8601fd the same open was served straight from the cached path.
  • Repro. Scratch test r3/test_cached_offline_row.py, using the delta's cached_library fixture with the real remote_row_for body and the relay's owner-offline listing:
    • 404 wire.errors.desktop.requested_not_found ("Requested session, profile, team or subscription not found"), with listing reads: [0] (one ttl-0 fan-out).
    • With the row known, the same asset returns 200 application/octet-stream.
  • Impact.
    • This breaks the acceptance criterion "precache … so opening is instant", in the one case where the cache matters most.
    • The message is neither honest nor retryable.
    • The comment at desktop_library.py:396-397 ("this never dials while the bytes are here") isn't true when the peer-row cache misses.
  • Fix. In library_content's digest resolve(), run remote_cache.lookup(config_dir, (member_id, session_id), digest) first and return the hit without a row. Call _row_for only on a miss, and lazily for _opened's attempt-2 error. Regression test: owner offline (empty listing) + cached bytes returns 200.

R3-3 — MINOR (neighbour): the desktop thumbnail route serves a cached peer image with the owner's type, without _servable or a sandbox.

  • desktop_library.py:490 uses media_type=content.content_type with nosniff only. A peer can name a file pic.png (kind image, from the suffix) and declare text/html or application/xml.
  • It stays MINOR because the desktop routes take only a bearer header (I found no query or cookie token), so a navigation can't reach it, and <img> doesn't run script. That's the same class as R1-5 originally.
  • Fix: when content.remote is set, use _servable(...) with CSP: sandbox here too.

R3-4 — NIT. tests/unit/i18n/test_catalogue_sizes.py:194 says "64 KiB divided evenly by luck". It didn't divide evenly: 65,536 ≡ 1 (mod 3), so 3·⌊c/3⌋ happened to equal c − 1. Suggested wording: "3·(64 KiB // 3) happened to be ceiling − 1".

TUI caller (the coder's question)

  • tui/widgets/library_view.py:1361 uses only resolved.path (<digest>.bin) for a local preview or open. It doesn't serve over HTTP and doesn't use the declared type.
  • No web-origin exposure: not a finding.

Not verified

  • I didn't run the delta's RED-on-base arms against 14d8601fd, because the worktree stays at head. The diff shows the removed path = cached fall-through those arms depend on.
  • QA's real-path retest of Q2-1 to Q2-3 belongs to the QA stream.

@damianvtran

Copy link
Copy Markdown
Owner Author

QA report — round 3

Verdict: FAIL, on one new MAJOR finding (Q3-1). The fix in this delta causes it, and I reproduced it against a base control.

  • Q2-1, Q2-2 and Q2-3 are fixed. I re-ran each one on the real path and they pass. The _opened neighbours pass too: the content route, Library on a cache miss, and download.
  • Q3-1 (MAJOR). The cached-remote Library open is now routed through _row_for. As a result, a Library image whose bytes are already on this device fails when its owner is offline. It returns 404 "Requested session, profile, team or subscription not found". At 14d8601 the same request returned the bytes (200).
  • Rows I was asked to re-run all pass: Library listing, thumbnail with the owner stopped and 0 connect() calls, and RSS.

Scope: 14d8601fd..5d7e1a5e4. Head: 5d7e1a5e4f30201514c17e35b79072987d7a6781. I checked the worktree HEAD before and after every run, and its tracked files were clean (0 changes).
QA: independent qa-tester (lopdev). Served model: anthropic/claude-opus-5-5, no fallback.

Rig. The round-2 harness, copied into this session's scratchpad. The only changes are paths, and the new rig_matrix_r3.py, base_viewer_wrapper.py and rig_offline_r3.py. Everything runs on loopback:

  • Two REAL relays, each a lop network serve process from the worktree .venv.
  • The viewer's REAL lop serve daemon.
  • Each process has its own isolated HOME and LOCAL_OPERATOR_CONFIG_DIR under env -i, so no CMUX_* reached it.
  • Everything ran under run_bounded: the main run took 117 s and the control run 121 s. The UI Xvfb run had finished before I started, so 0 heavy processes were running.

I created the rig data directory /home/ec2-user/qa-rfp-core-r3 and removed it afterwards. I didn't touch qa-rfp-core-r2.

Same-code control for Q3-1. base_viewer_wrapper.py runs the head daemon, but loads the three executable files changed by e3ac0f2 from 14d8601fd via git show:

  • library/service.py
  • routes/desktop_library.py
  • utils/remote_session_files.py

Every other module is head's. The wrapper's start record proves which source files were loaded.

Matrix

# Surface Command Actual Verdict
pre-11 Precache (precondition) Fresh viewer, cache emptied, one GET /v1/desktop/sessions?include_peers=true 10 files (31.1 MiB) landed in 11.1 s; nothing that should be skipped landed. The owner answered 1+1 precache lists, 4 rows and 81 net_file_get PASS
Q2-1 Library content of the remote DIGEST evil.svg: one cache-miss open, then two cache-hit opens Drop cache/remote-files/<d>.*; GET /v1/desktop/library/assets/{evil}/content ×3. The owner wrapper counts net_file_get All three opens: 200, application/octet-stream, CSP: sandbox, nosniff, sha matches. cached_before was false, true, true. net_file_get: 2 on the first open, 0 and 0 on the cache hits PASS
Q2-1b Library content of qa-shot.png, cache hit GET content 200 image/png, CSP: sandbox, nosniff, sha matches, 0 net_file_get PASS
Q2-1c ?download=1 + Range on a cache hit GET …/content?download=1, Range: bytes=0-7 206 bytes 0-7/135. Body 89504e470d0a1a0a. attachment; filename*=…qa-shot.png, CSP: sandbox PASS
Q2-2 Library content, cache hit: the blob is evicted between the lookup and the open Budget 0; gate remote_session_files._open; evict through a real GET lru-0, and check that the target is gone; release Open 1 failed with FileNotFoundError, then open 2 succeeded. The client got 200, Content-Length: 135 and 135 bytes, sha matches. No 200-then-truncate PASS
Q2-2-seam The old lazy desktop_library._stream is no longer used on a cache hit The wrapper logs every _stream call 0 calls PASS
Q2-3 Library content, cache hit: an eviction before BOTH opens Gate 2 opens and evict before each one 504 attachment_fetch_pending, "This file is still on its way from qa-owner-a. Try again in a moment.", Retry-After: 2 PASS
Q2-3 / R1-3b The same, on the session-files content route Same 504 attachment_fetch_pending, same sentence, Retry-After: 2 PASS
Q2-3 / R1-3d The same, on a Library cache miss Evict the target first, then gate 2 opens 504 attachment_fetch_pending, Retry-After: 2 PASS
Q2-3 / R1-3f The same, on download Gate 2 opens on POST …/download 504 attachment_fetch_pending, Retry-After: 2 PASS
R1-3a / c / e One eviction, on content, Library miss and download Gate 1 open Each one: FileNotFoundError, then ok, then 200; sha matches. Download: created:true, sha verified on disk PASS
I1-1 Owner unreachable still uses pending=False Stop relay A; GET an uncached digest 504 attachment_owner_unreachable, "qa-owner-a is not answering. Try again when it is online.", no Retry-After PASS
R3-off-a Cached Library open with the owner relay stopped, while the in-memory peer-row cache is still warm Stop relay A; GET content 200 image/png, sandbox, 9.5 ms PASS
R3-off-b Cached Library open with the owner stopped, after a viewer daemon restart Stop relay A; restart the viewer; GET content Head: 404 requested_not_found, "Requested session, profile, team or subscription not found". In the same window the thumbnail is 200. Base control: 200 image/png, sha matches FAIL (Q3-1)
R3-off-d The same, after the sidebar's catalogue poll GET /v1/desktop/sessions?include_peers=true (200, 0 remote rows), then GET content Head: 404, same sentence. Base: 200 FAIL (Q3-1)
R3-off-e Long-running daemon (no restart). The owner is offline for longer than the 20 s listing TTL, then the sidebar re-polls Open with the owner up (200); stop relay A; wait 25 s; catalogue poll; GET content Head: 404 requested_not_found, run twice. Base: 200, sha matches FAIL (Q3-1)
R3-off-c Cached Library open with the owner ONLINE, after a viewer restart (cold peer-row cache) Restart the viewer; GET content 200, 34.9 ms. The owner answered 0 net_file_get, so the bytes came from the cache PASS
12 RSS while serving and downloading the 200 MiB pad file Content route streamed to disk (n=326 samples), then POST …/download (n=20) Content: 19.9 s, sha matches. Daemon tree +0.8 MiB, relay B +7.9, relay A (owner) +18.6. Download: 1.15 s, sha matches, daemon +7.6. Maximum 18.6 MiB against a 200 MiB file, under the 64 MiB bar PASS
12c RSS while streaming the same 200 MiB remote asset through Library content GET /v1/desktop/library/assets/{big}/content, streamed 200; 209,715,200 B; sha matches. Daemon tree +0.5 MiB, relays +0.0 PASS
13a Library listing GET /v1/desktop/library/assets?limit=200 15 remote rows, ref.path: null, no credential names PASS
13b Remote digest thumbnail GET thumbnail 200 image/webp on the first poll. The pad-path thumbnail is still 404 (Q1-2, deferred) PASS
13c Library content GET content 200, 135 B PASS
13d-a List, detail and thumbnail with the owner stopped ×5 each; viewer under strace -f -e trace=connect All 200, maximum 80 ms, 0 connect() calls. Negative control GET /v1/desktop/peers: 1 connect PASS
13d-b The same, with the viewer's relay also SIGSTOPped Same All 200, maximum 79.9 ms, 0 connect() calls. The control took 20,026 ms (503) with 1 connect PASS
INFO-mobile Mobile daemon on the viewer: /api/library/assets/{evil.svg}/content with the bytes cached lop mobile serve on B's config (isolated); a cookie-authed GET, plus an unauthenticated control Unauthenticated: 401. Authed: 200 image/svg+xml inline, but with CSP: sandbox; default-src 'none' and nosniff. The body is the <script> SVG (sha matches). So the SVG is not served without a sandbox. The mobile code is unchanged by this PR, which is a non-goal INFO
tests The diff's own tests run_bounded … pytest -n 0 tests/unit/network/test_remote_files_hardening.py tests/unit/i18n/test_catalogue_sizes.py, isolated HOME 22 passed in 1.41 s PASS
T Teardown kill -0 on every child pid, in both runs Main run: 11/11 gone. Control run: 16/16 gone. No rig process left PASS

Retained (14d8601), not re-run: R1-2, 8, 8-active, R1-5 path, 6/Q1-1, 5/Q1-1, 8u/N1, I1-2, I1-3, R1-4, R1-3 pre, R1-3g, R1-3h, R1-3i, R1-1 ctrl, R1-1 and Q1-2 (deferred). Rows retained from round 1 (9a6ef56) are unchanged. The delta doesn't touch the transcript attachment route, the listings, the precache gate, the path-row resolution, download landing or the malicious-row validation.

Findings

Q3-1 — MAJOR: since e3ac0f2, opening a cached remote Library asset fails with a false "not found" when the owner is offline. It worked at 14d8601.

  • Code path.
    • desktop_library.py: content.remote → _remote_content → remote_session_files.library_content.
    • The digest resolve() calls _row_for(config_dir, session_id) before peer_blob's cache lookup.
    • _row_for → remote_row_for → peer_session_row, which finds no row: an offline owner's sessions drop out of the listing (0 remote rows in R3-off-d and e). It then calls read_listing(ttl_s=0), which also finds none, so it raises KeyError.
    • errors() turns that into 404 requested_not_found.
    • At 14d8601, resolve_content served the cached path directly, without a row.
  • Repro. Rows R3-off-b, R3-off-d and R3-off-e, on head and on the base control. R3-off-e is the everyday case: the daemon keeps running, the owner sleeps for more than 20 s, the sidebar re-polls, and the user clicks a Library tile whose thumbnail still renders.
  • Why MAJOR.
    • It is a user-visible regression from this delta on the surface the delta targets: an open that returned the bytes now errors.
    • The sentence is false: the session isn't missing, and the bytes are on this device. That breaks the acceptance criterion: "when not [reachable], an honest retryable state".
    • It also contradicts the remediation note: "This never dials while the bytes are here". With a cold or stale peer-row cache, read_listing(ttl_s=0) goes to the relay.
  • Suggested fix. In library_content's digest resolve(), try remote_cache.lookup(config_dir, (member_id, session_id), digest) first. Call _row_for and peer_blob only on a miss; the _opened retry keeps that order. This keeps Q2-1 and Q2-2: the same _opened, _servable and CSP: sandbox path, with no dial on a cache hit.
  • Suggested regression test. A cached digest asset where remote_row_for returns None must return 200 with octet-stream and sandbox. Optionally, a row miss on a cache miss should map to the owner-unreachable 504, not to 404.

Q3-2 — lead, outside this delta's surface, non-gating this round: the transcript attachment route behaves the same way.

  • In R3-off-b's window, GET /v1/desktop/sessions/{sid}/attachments/{digest} for the same cached digest also returned 404 requested_not_found.
  • The cause is the same: the row is resolved before the cache lookup.
  • The route is unchanged by 14d8601fd..5d7e1a5e4, so it isn't caused by this delta. It has been in the PR since round 1, and no earlier round exercised it.
  • The same reordering would fix it. Whether to fold it into the Q3-1 fix is the manager's call.

Observation, not a finding. In row 12, relay A, the owner, peaked at +18.6 MiB (round 2: +0.2). It had been restarted shortly before the baseline sample, and the delta doesn't touch it. The figure is still under 10 % of the file size.

Untested gaps:

  • RSS for a large cached DIGEST asset on the new cache-hit stream. The fixtures have no digest asset over 32 MiB. Row 12c covers the same handle_response streaming through a 200 MiB path asset.
  • Base arm for Q2-1, Q2-2 and Q2-3. Not re-run this round. Round 2's FAIL rows are that arm, plus the implementer's 3 red-on-base tests.
  • UI rendering of the 404 and 504 states, and cross-host WAN. Not covered: this is a backend-only rig on loopback.

…, sandbox them on the phone

The digest routes (Library open, transcript attachment, waveform, document,
child) read the owner's listing row before this device's copies, so a
precached file 404'd as 'not found' exactly when its owner was asleep, and a
row-cache miss dialled the relay for bytes already here. They now look in the
store, replica and remote-files cache first, keyed by a durable existence proof
(the Library index's recorded origin device), and read the row only on a miss;
a miss with the owner silent is the retryable 504 attachment_owner_unreachable.

The phone's Library routes served a cached peer file inline with the owner's
declared type (XML unsandboxed on the relay origin); they and the desktop
thumbnail now apply the remote routes' servable_type allowlist plus a sandbox.

Fixes #3872 R3-1..R3-4, Q3-1, Q3-2.
@damianvtran

Copy link
Copy Markdown
Owner Author

Remediation — round 3

Head: 3a842e70b (one commit on 5d7e1a5e4). Scope: 5d7e1a5e4..3a842e70b. Implementer: coder (lopdev), anthropic/claude-opus-5-5, no fallback. No catalogue changes. git diff 5d7e1a5e4..HEAD -- local_operator/i18n is empty, and scripts/i18n/generate.py --check reports 161 artifacts byte-identical.

Dispositions

  • R3-2 = Q3-1 (MAJOR): fixed.
    • New desktop_sessions.local_peer_blob(root, sid, digest, member_id, owner). It checks the store, then the replica, then cache/remote-files (remote_cache.lookup), before it touches any row.
    • Only on a miss does it call the new peer_row(...) and then peer_blob, which pulls.
    • library_content's digest resolve() now calls it, with owner set to the occurrence's origin_device.
    • What's unchanged: servable_type (the former _servable), CSP: sandbox, nosniff, _opened (opened before the headers go out, then one re-resolve).
    • Two consequences: a cache hit makes 0 row reads and 0 relay calls, and the comment at desktop_library.py:396 is now true.
  • Q3-2: folded in, fixed.
    • The transcript attachment, attachment_waveform, supplement_document, child_attachment and child_attachment_waveform routes now return a peer marker when no local directory exists. _peer_lookup then resolves through local_peer_blob.
    • On a miss, peer_row reads remote_row_and_silence. There are three outcomes:
      • a row → pull;
      • no row and a silent device → the existing 504 attachment_owner_unreachable, naming the silent device and narrowed to the indexed owner;
      • no row and every device answered → the 404 as before.
    • PeerAttachmentRetryable now accepts row=None with an explicit device. The status, code and key are unchanged.
  • Existence check kept (found while fixing). The first cut served row-free hits for any well-formed id. Two existing tests caught that: test_desktop_sessions.py::test_durable_attachment_is_readable_without_starting_an_owner and …waveform_host_reads_the_sidecar_or_misses, both with an unknown id plus a store digest, expecting KeyError.
    • A row-free hit now needs durable proof that the id is a peer's session on this device: the Library index's idx_sessions.origin_device (indexed_owner), or the Library occurrence itself.
    • The index keeps a silent owner's sessions, so the proof survives the owner being offline. It never creates the index file.
    • An id the index never recorded takes the row path, exactly as before.
    • Local sub-agent directories still refuse, and an unknown child-route parent still answers child_not_found.
  • R3-1 (MAJOR): fixed.
    • mobile/library.py has a new _as_served. For a ref.remote from content_ref or thumb_ref, it maps content_type through the shared remote_session_files.servable_type. That is the same helper as desktop, reused rather than copied.
    • mobile/daemon.py _library_response_headers always sends Content-Security-Policy: sandbox; default-src 'none' for a remote ref. It already sent nosniff.
  • R3-3 (MINOR): fixed. For content.remote, the desktop thumbnail route uses servable_type and CSP: sandbox.
  • R3-4 (NIT): fixed. The comment now says: at 64 KiB, 3·(c//3) = c−1 because 65,536 ≡ 1 (mod 3); 80 KiB is not, so the remainder is computed. (81,920 ≡ 2 (mod 3).)

Regression tests: tests/unit/network/test_remote_files_offline_owner.py (12 cells)

The owner being offline is modelled at remote_open.remote_row_and_silence, with no row and an UnansweredPeer for the owner. The test spies on desktop_mesh._call and on remote_fetch.fetch. The cache, lookup, routes and responses are real.

  • (1) test_r3_2_a_cached_library_asset_opens_with_its_owner_offline: SVG digest → 200, application/octet-stream, CSP: sandbox, nosniff. The open makes 0 row reads, 0 relay calls and 0 pulls.
  • …uncached_asset_with_its_owner_offline_is_retryable_not_missing: 504 attachment_owner_unreachable.
  • …owner_that_answered_without_the_session_is_still_404: a control.
  • (2) test_q3_2_a_cached_transcript_attachment_opens_with_its_owner_offline: the parent and child routes → 200, image/png, nosniff, with 0 row reads, 0 relay calls and 0 pulls.
  • …cached_svg_on_the_transcript_route_is_opaque.
  • …uncached_attachment_with_its_owner_offline_is_504_not_404: the parent, waveform and child routes → 504 naming "owner box", with 0 pulls.
  • Controls:
    • …session_nobody_holds_keeps_its_refusals: 404 and child_not_found.
    • …id_this_device_never_recorded_cannot_read_the_cache: no bytes.
  • (3) test_r3_1_…phone_is_opaque_and_sandboxed[text/xml|application/xml|text/html]: the mobile /content and /thumbnail routes → application/octet-stream, sandbox; default-src 'none', nosniff.
  • test_r3_3_a_cached_peer_thumbnail_is_opaque_and_sandboxed: desktop, application/xml → octet-stream and sandbox.
  • RED on 5d7e1a5e4 (product files stashed, tests kept): 9 failed, 2 passed out of the 11 cells that existed then. The two passes are the controls. Failures:
    • (1): 404 == 200;
    • Library uncached: 404 == 504;
    • (2) cached: 404 == 200;
    • (2) SVG: 404 == 200;
    • (2) uncached: 404 == 504;
    • (3): 'text/xml; charset=utf-8' == 'application/octet-stream', plus the application/xml and text/html cases;
    • R3-3: 'application/xml' == 'application/octet-stream'.
    • The 12th cell, never-recorded, is a control added after that run.
  • test_remote_files_hardening.py's cached_library fixture now also stubs desktop_utils.peer_row. The Q2-2 cache-miss re-resolve needs a row now that the row is read lazily.

Targeted test evidence

Head 3a842e70b. Isolated HOME/LOCAL_OPERATOR_CONFIG_DIR in the scratchpad, with CMUX stripped. Each run used python3 scripts/run_bounded.py --timeout 600 --memory-mb 3000 -- .venv/bin/python -m pytest -q -p no:cacheprovider …, and only one heavy run was active on the node (checked with ps).

tests/unit/network/test_remote_files_offline_owner.py                       → 12 passed in 2.08s
tests/unit/network/test_remote_files_{hardening,library,fetch,index,precache,scan}.py → 55 passed in 25.84s
tests/unit/library/test_{route,service}.py tests/unit/mobile/test_library_routes.py
  tests/unit/i18n/test_catalogue_sizes.py tests/unit/server/test_desktop_remote_open.py
  tests/unit/server/test_supplement_routes.py                                → 94 passed in 5.80s
tests/unit/server/test_desktop_sessions.py tests/unit/network/test_remote_viewer.py
  -k "attachment or waveform or child or peer or remote"                     → 67 passed, 2 errors in 9.69s
  • The 2 errors are pre-existing. Both are test_real_mp3_waveform_http_writes_and_serves_sidecar[*] with KeyError: 'desktop_sessions' in fixture setup. The same-code control on 5d7e1a5e4 source gives the same result: 2 passed, 2 errors. They are not caused by this delta.
  • Static checks on the changed files: flake8 clean, black==26.1.0 and isort==5.13.2 clean, pyright 0 errors.

@damianvtran

Copy link
Copy Markdown
Owner Author

Agent review — round 4

Reviewer: reviewer (lopdev), anthropic/claude-opus-5-5, no fallback
Scope: 5d7e1a5..3a842e7 (delta only; retest R3-1, R3-2/Q3-1, Q3-2, R3-3; all else retained (5d7e1a5))
Worktree: .worktrees/remote-files-parity @ 3a842e70b, read-only (git status empty after every run). Tests, bounded via run_bounded, isolated HOME/config, CMUX stripped:

  • test_remote_files_offline_owner.py + 4 reviewer probes (scratch, outside the worktree): 16 passed
  • test_remote_files_{hardening,library,fetch}.py, mobile/test_library_routes.py, test_supplement_routes.py, test_desktop_remote_open.py, test_catalogue_sizes.py: 96 passed
  • test_desktop_sessions.py + test_remote_viewer.py -k "attachment or waveform or child or peer or remote": 67 passed, 2 errors. Both errors are test_real_mp3_waveform_http_writes_and_serves_sidecar[*], a fixture-teardown KeyError: 'desktop_sessions'. That file is not in the delta. The "pre-existing" status comes from the implementer's same-code control in #issuecomment-6833147857; I did not re-run it.

Verdict: clean. No BLOCKER or MAJOR remains. R3-1, R3-2/Q3-1, Q3-2 and R3-3 are closed. The round is terminal: the two MINORs and one NIT below are for the thread and need no new commit.

Verified

  • local_peer_blob order (R3-2/Q3-1, Q3-2).
    • remote_cache.lookup walks locations(), which checks store, then replica, then cache/remote-files, by stat only.
    • It runs before peer_row (desktop_sessions.py local_peer_blob). peer_row / remote_row_and_silence is reached only on a miss.
    • The Library digest resolve() passes the occurrence's origin_device as owner, so a hit makes no index read and no row read. test_r3_2_a… asserts 0 row reads, 0 relay calls and 0 pulls.
    • All five transcript routes return _ON_PEER and resolve through _peer_lookup, so none reads a row first: attachment, attachment_waveform, supplement_document, child_attachment and child_attachment_waveform.
  • indexed_owner proof: no R1-1 reopening.
    • It is a parameterised SQL lookup (idx_sessions WHERE session_id = ?) and builds no path. It returns "" when there is no index file, and it never creates one.
    • Every transcript route still applies SESSION_ID.fullmatch before _ON_PEER: attachment, waveform and supplement inline, the child routes in _peer_parent for both ids.
    • lookup/locations re-check is_digest/is_session_id, so malformed values are skipped or raise.
    • The Library session id comes from the index. Its only path use is the is_dir stat in remote_row_and_silence, which _row_for already did, so that is unchanged.
  • The proof cannot be forged for a local or unknown id.
    • origin_device is written only by library/remote.py feed, from remote_session_rows answers. The local indexer's record_session leaves it at ''.
    • Probe: a LOCAL session's index record with its directory gone gives indexed_owner == '', and the cached digest returns 404 with no bytes. The id that was never recorded behaves the same (the delta's control test).
  • View-revoked or removed owner: accepted, with this reasoning.
    • Probe: indexed owner, listing returns no row and no silence (device gone or revoked). A cached digest gives 200 image/png, 0 row reads. A miss gives 404.
    • So bytes already on this disk keep serving after the owner stops granting view. I judge that acceptable, for four reasons:
      1. Who it reaches. The route is bearer-gated to this device's own user. Serving bytes already on this disk crosses no device boundary.
      2. Same as other heads. It matches how the replica/net_sync path treats those bytes: network/sync.py has no purge on revoke. It also matches how the Library itself already behaves: cached-path opens at 14d8601, and thumbnails at every head, are served row-free.
      3. New bytes still need the owner. Anything not already here still needs a row and a pull, and the owner re-checks view (_session_scope) on the pull.
      4. Where the proof comes from. The index learns origin_device only from catalogue answers, which list only sessions the viewer could view.
    • A permanently removed device's index rows linger until the cache LRU evicts the bytes. That lingering index retention comes from the PR's index-feed design in earlier rounds, not from this delta, so I'm not raising it here.
  • Offline 504 vs 404.
    • Indexed owner silent + miss → 504 attachment_owner_unreachable naming the owner. owner narrows the silence to the indexed device.
    • Owner answered without the session → 404.
    • Child route with an unknown parent → child_not_found.
    • All three are covered by the delta's tests and pass.
    • PeerAttachmentRetryable(row=None, device=…) is safe in both retry paths (_peer_file attempt 2 and _opened attempt 2).
  • Mobile (R3-1).
    • _as_served maps a ref.remote from both content_ref and thumb_ref through the shared servable_type.
    • _library_response_headers sets sandbox; default-src 'none' for any remote ref, after the type-based branches. A live WebP thumbnail is built without remote, so it is unaffected.
    • test_r3_1[text/xml|application/xml|text/html] passes.
  • servable_type rename.
    • grep -rn '_servable\b' over local_operator tests scripts docs finds only the unrelated mcp/auth.py self._servable.
    • All 5 servable_type call sites import from remote_session_files.
  • R3-3. The desktop thumbnail route uses servable_type + CSP: sandbox for content.remote. test_r3_3 passes.

Findings

R4-1 — MINOR: remote SVG tiles now render as broken images, on desktop and on the phone. (This is a side effect of the R3-3 fix I suggested in round 3.)

  • servable_type passes only SUPPORTED_IMAGE_MIME_TYPES (png, jpeg, gif, webp; media.py:35), so image/svg+xml becomes application/octet-stream.
  • service.thumbnail always serves an SVG as its original.
  • Probe: a remote SVG thumbnail returns 200 application/octet-stream, CSP: sandbox. At 5d7e1a5 it returned image/svg+xml.
  • Why the tile breaks. The UI builds the tile from the response type: use-library-queries.ts useLibraryBytes → useObjectUrl(bytes, mimeType) → <img> in library-card.tsx. Blink decodes SVG only under an SVG MIME type, so the tile becomes a broken image rather than the kind glyph. I'm inferring that from browser behaviour; there's no browser on this host.
  • The phone's remote SVG goes the same way through _as_served. The R3-1 SVG control was already safe there.
  • Why only MINOR. One image type, cosmetic. Remote SVG has been opaque on the content route since R1-5, so this matches an accepted posture.
  • Fix (one line): for remote image refs, keep image/svg+xml under the sandbox. sandbox; default-src 'none' neutralises it on navigation, and <img> never runs script. For example, servable_type(t) if t != "image/svg+xml" else t in the thumbnail route and in _as_served.

R4-2 — MINOR: for a session id the index never recorded, the 504 names every silent device.

  • peer_row with owner="" keeps all silent peers.
  • Probe: unrecorded id + an unrelated silent other box gives 504 "other box is not answering". At 5d7e1a5 this was a 404.
  • This is the right answer when the session really is on that device: that's the Q3-2 case without a Library index. When nobody holds the id, though, the sentence names the wrong device.
  • Fix: accept and defer. No owner is known, so there's nothing to narrow by.

R4-3 — NIT: rare double-eviction race on the transcript routes.

  • In _peer_file, attempt 2 can now raise PeerAttachmentRetryable(session_id, None, pending=False) after two row-free cache hits, which renders as "unnamed device is not answering".
  • Using pending=True here, as the Library path's Q2-3 fix does, would keep the sentence honest.

Not verified

  • Browser rendering of R4-1. Whether a phone's PDF viewer opens under CSP: sandbox on a top-level target=_blank. No browser on this host.
  • The delta's RED-on-base arm. The worktree stays at head; the source is the implementer's reply.
  • Real-path QA on two relays. That belongs to the QA stream.

@damianvtran

Copy link
Copy Markdown
Owner Author

QA report — round 4

Verdict: PASS. No BLOCKER or MAJOR. Every R3 and Q3 row passes on the real path at 3a842e70b.

  • Q3-1 (= R3-2) is fixed. With the owner relay stopped and the viewer daemon restarted, cached Library digest assets return 200 with the sandbox headers, and the viewer makes 0 connect() calls.
  • Q3-2 is fixed. The transcript route serves a cached digest with 200. An uncached digest now returns 504 attachment_owner_unreachable instead of 404, on both the transcript route and Library content.
  • R3-1 is fixed. On lop mobile serve, remote rows declared text/xml or application/xml, and the cached evil.svg, are served as application/octet-stream with CSP: sandbox; default-src 'none' and nosniff. A same-code control running the two mobile files from 5d7e1a5e4 reproduces the round-3 behaviour (text/xml, no CSP), so the probe discriminates.
  • R3-3 is fixed. On the desktop, the thumbnail of a remote image served from the original bytes carries CSP: sandbox, and evil.svg is served as octet-stream.
  • Waveform for a cached WAV (transcript route): 200.
  • One new non-gating lead, outside this delta (Q4-1, MINOR): the Library waveform route returns 404 for a cached remote WAV. Details are under Findings.

Scope: 5d7e1a5e4..3a842e70b. Head: 3a842e70b65d6ab0aef85b54734411d7b22484ce. Retest: R3-1, R3-2/Q3-1, Q3-2, R3-3, plus Q2-1/Q2-2/Q2-3 and the Library rows, because resolve() moved. Worktree HEAD was 3a842e70b before and after every run, with 0 tracked changes.
QA: independent qa-tester (lopdev), session 4af48960f0ed. Execution and initial report: anthropic/claude-opus-5-5. Completion/proofreading/posting after Anthropic HTTP 429 cooldown: openai/gpt-6.1-sol. No test reruns on resumption; the evidence remains the original execution at the briefed head.

Rig. The round-3 harness, copied into this session's scratchpad. The helpers from rounds 1–3 are unchanged. New files:

  • rig_setup_r4.py: the round-3 setup plus three owner artifacts:
    • report.pdf, declared text/xml;
    • data.pdf, declared application/xml;
    • pic.png, declared application/xml.
    • Both XML bodies are XHTML-namespaced <script> documents.
  • rig_matrix_r4.py: the driver.
  • base_mobile_wrapper.py: the control. It loads mobile/library.py and mobile/daemon.py from 5d7e1a5e4 via git show; every other module is head's.

Everything runs on loopback:

  • Two REAL relays, each a lop network serve process from the worktree .venv. The owner runs under the counting wrapper.
  • The viewer's REAL lop serve, run under strace -f -e trace=connect for the offline rows.
  • A REAL lop mobile serve on the viewer's config.

Each process had its own isolated HOME and LOCAL_OPERATOR_CONFIG_DIR under env -i, so no CMUX_* reached it. The run was bounded by run_bounded: wall time 101 s, rc 0, and there were 0 Tracebacks across the 5 process logs.

Matrix

# Surface Command Actual Verdict
11 Precache (precondition) Fresh viewer, cache emptied, one GET /v1/desktop/sessions?include_peers=true 13 files (31.1 MiB) landed in 11.0 s; everything expected landed and nothing that should be skipped did. The owner answered 1+1 precache lists, 4 rows and 87 net_file_get PASS
13a Library listing (owner up) GET /v1/desktop/library/assets?limit=200 18 remote rows, including the three new ones. Declared types are kept in the index: report.pdf text/xml, data.pdf and pic.png application/xml, evil.svg image/svg+xml. Every ref.path is null; no credential names PASS
13b / R3-3 Remote digest thumbnail, first poll (original bytes, not yet WebP) GET …/assets/{qa-shot}/thumbnail 200 image/png, CSP: sandbox, nosniff, 135 B PASS
pre-warm The new rows' bytes pulled through the real transcript route GET /v1/desktop/sessions/{sid}/attachments/{d} ×7 All 200 and sha matches; XML, SVG and pic.png served as application/octet-stream; all 7 in cache/remote-files PASS
Q2-1 Library content of evil.svg: one cache-miss open, then two cache-hit opens Drop the cache entry; GET content ×3 All three: 200, application/octet-stream, CSP: sandbox, nosniff, sha matches. net_file_get: 2 on the miss, 0 and 0 on the hits PASS
Q2-1b / Q2-1c Cache hit for qa-shot.png; ?download=1 + Range: bytes=0-7 GET 200 image/png, sandbox, 0 pulls. Range: 206 bytes 0-7/135, body 89504e470d0a1a0a, attachment, sandbox PASS
Q2-2 Library cache hit, with the blob evicted between the lookup and the open Budget 0; gate _open; evict through a real GET lru-0 Open 1 failed with FileNotFoundError, open 2 succeeded. Client got 200, Content-Length: 135, sha matches. Seam check: 0 _stream calls PASS
Q2-3 Library cache hit, with an eviction before both opens Gate 2 opens 504 attachment_fetch_pending, "This file is still on its way from qa-owner-a. Try again in a moment.", Retry-After: 2 PASS
Q2-3 neighbours Content route, Library cache miss, download (2 evictions each) Same All three: 504 pending, Retry-After: 2 PASS
R1-3a / c / e One eviction on content, Library miss and download Gate 1 open All three 200 and sha matches. Download: created:true, sha verified on disk PASS
I1-1 Owner stopped, uncached digest (warm peer-row cache) GET attachment 504 attachment_owner_unreachable, "qa-owner-a is not answering. Try again when it is online.", no Retry-After PASS
R4-off-W1 (Q3-1, Q3-2, R3-3, 13d-a) Owner relay STOPPED and viewer RESTARTED (cold peer-row cache): every read of bytes that are on this device Stop relay A; restart the viewer under strace; 13 GETs (see the breakdown below); control GET /v1/desktop/peers 12 of 13 are 200, as expected, with 0 connect() calls in the window. Control: 1 connect. The 13th request is Library waveform: 404 (Q4-1, below) PASS¹
R4-off-miss (Q3-1, Q3-2) Same window, uncached digests GET the transcript attachment of the evicted lru-3; GET Library content of the evicted qa-clip.mp4 Both: 504 attachment_owner_unreachable, "qa-owner-a is not answering…", no Retry-After, about 20 ms, 2 connects (the listing read). Not 404 PASS
R4-unknown-id Control: a never-recorded session id plus a cached digest GET /v1/desktop/sessions/<random>/attachments/{cached} 504 attachment_owner_unreachable; no bytes served PASS
R4-off-d (R3-off-d) Owner stopped, after the sidebar's catalogue poll GET sessions?include_peers=true (200), then the same 13 GETs The same 12/13 are 200, 0 connects PASS¹
R4-off-W2 (13d-b) Owner stopped and the viewer's relay SIGSTOPped The same 13 GETs, then the control The same 12/13 are 200, max 70.4 ms, 0 connects. The control GET /v1/desktop/peers took 20,026 ms (503) with 1 connect PASS¹
R4-off-e (R3-off-e) Long-running viewer: owner offline past the 20 s TTL, then a sidebar re-poll Open with the owner up (200); stop relay A; sleep 25 s; catalogue poll; open Library content 200 image/png + sandbox, sha matches. Transcript attachment 200, sha matches PASS
R3-1 Mobile daemon on the viewer (lop mobile serve), owner stopped Cookie-authenticated GET /api/library/assets/{id}/content (and /thumbnail); unauthenticated control Unauthenticated: 401. See the R3-1 breakdown below PASS
R3-1-ctrl Same-code control: the two mobile files from 5d7e1a5e4 The same GETs report.pdf → text/xml, CSP none. data.pdf and pic.png → application/xml, CSP none. evil.svg → image/svg+xml with sandbox (the old SVG rule). The loader proves which source files were used (has_as_served: false) PASS (discriminates)
tests The diff's own tests run_bounded … pytest -n 0 tests/unit/network/test_remote_files_offline_owner.py tests/unit/network/test_remote_files_hardening.py tests/unit/i18n/test_catalogue_sizes.py, isolated HOME 34 passed in 2.00 s PASS
T Teardown kill -0 on every child pid 10/10 gone; no rig process left PASS

¹ The raw harness results preserve W1, d and W2 as FAIL because the Library-waveform request was in the same set. The other 12 requests pass in all three windows. The scoped verdict separates that 404 as Q4-1: out of scope — pre-existing at 5d7e1a5e4. The digest waveform() store-only lookup is unchanged even from the PR base (git show 89678ffc7:local_operator/library/service.py, then line 1002), and does not call the resolver moved by this delta. No raw evidence was rewritten.

The 13 GETs in each R4-off window (results from W1):

  • /library/assets/{qa-shot}/content: 200 image/png + sandbox.
  • {evil.svg}/content: 200 application/octet-stream + sandbox.
  • {report.pdf, text/xml}/content: 200 octet-stream + sandbox.
  • Transcript attachments/{image}: 200 image/png.
  • Transcript attachments/{svg}: 200 octet-stream.
  • children/<random>/attachments/{image}: 200.
  • attachments/{wav}/waveform: 200, 240 bars.
  • attachments/{wav}: 200 audio/wav.
  • {evil.svg}/thumbnail: 200 octet-stream + sandbox (R3-3).
  • List, detail, and the qa-shot thumbnail: 200.
  • Every byte response had a matching sha.
  • /library/assets/{qa-tone.wav}/waveform: 404 library_content_missing (Q4-1).

R3-1 results on the mobile daemon:

  • report.pdf (text/xml), data.pdf (application/xml), evil.svg and pic.png (application/xml) all return 200 application/octet-stream + CSP: sandbox; default-src 'none' + nosniff, and the shas match.
  • qa-shot.png (the control) returns 200 image/png + the sandbox.
  • The evil.svg thumbnail returns octet-stream + the sandbox.
  • The pic.png thumbnail returns image/webp. That is a locally generated thumbnail, not the peer's bytes, so it has no sandbox.

Retained (5d7e1a5), not re-run: 12 and 12c (RSS; the path-asset branch of library_content and handle_response are unchanged by this delta). Rows retained from 14d8601 and 9a6ef56 are unchanged: R1-2, 8, 8-active, R1-5 path, 6/Q1-1, 5/Q1-1, 8u/N1, I1-2, I1-3, R1-4, R1-3 pre, R1-3g/h/i, R1-1 ctrl, R1-1 and Q1-2 (deferred). INFO-mobile is superseded by R3-1.

Findings

Q4-1 — MINOR, lead, non-gating; outside this delta, present in the PR since ab6f4a92b/e3ac0f2ae: the Library waveform for a remote audio asset is a 404 even when its bytes are cached.

  • What I saw. GET /v1/desktop/library/assets/{qa-tone.wav}/waveform returned 404 library_content_missing in all three windows. In the same windows, the transcript route's …/attachments/{wav}/waveform returned 200 with 240 bars.
  • Code path.
    • library/service.py waveform(): for a digest asset it reads only waveform_payload(AttachmentStore(config / "attachments"), digest), which is this device's own store. It never reads cache/remote-files.
    • It doesn't call resolve_content, so the new comment at service.py:~824 ("Thumbnails and waveforms read the cached bytes without dialing") doesn't hold for waveforms.
    • The owner's state plays no part, so the result is the same online or offline.
  • Impact (inferred from the route contract; not rendered here). The Library grid's in-place player would show its no-waveform state for remote audio. The transcript WAV byte route returned 200 with a matching sha. I did not separately exercise Library WAV content or actual audio playback in this round.
  • Suggested fix, if it's in scope: when the store has no sidecar, fall back to waveform_payload(AttachmentStore(cache_root(config)), digest). That is what _peer_waveform does for the transcript route.

Observation (NIT, by design). A never-recorded session id with a cached digest returns 504 "qa-owner-a is not answering", not 404, while any peer is silent: with no indexed owner, the silence isn't narrowed. No bytes are served, so the existence check holds. Only the sentence names a device that never held the id.

Observation (unchanged behaviour). The child route serves a cached digest for any well-formed child id of an indexed peer session. Before this delta the cache hit didn't validate the child either: peer_blob looked up (child, sid) after resolving only the parent's row.

Untested gaps:

  • Desktop R3-3 base arm. I didn't run it. The diff shows media_type=content.content_type with nosniff only at 5d7e1a5e4, and the mobile arm, which has a control, shows the same class of bug.
  • Child route with a real sub-agent session. The rig session has no children. Covered by the implementer's test_q3_2_* cells.
  • UI rendering of the 504 and 404 states, actual audio playback, and cross-host WAN. This is a backend-only rig on loopback. No browser host is connected to this session, so rendered browser screenshots are unavailable; no substitute browser was installed or scripted.
  • Supplement-document branch. The delta touches this branch, but this round's real-path rig did not dispatch a supplement-document request. The transcript bytes, waveform, and child-byte neighbours were exercised as listed above.

Primary evidence (QA session 4af48960f0ed):

  • Directory: /home/ec2-user/.local-operator/sessions/4af48960f0ed/scratchpad/core-r4-qa/.
  • results-r4.json: raw matrix, SHA-256 20383e4b79c31ab88a2cbd4e47d3c8b92c69631e0873c3fd26d8b719cf3a5bd6.
  • run-r4.log: bounded execution, HEAD checks and teardown, SHA-256 0f1153b12cb5c25cc59a5523e58b9bc781fa9a11f45ff8ecbcd1ced8ad9216f7.
  • strace-viewer-r4.txt: connect tracing; owner-ops-r4.jsonl: mesh request counts; viewer-opens-r4.jsonl: gate outcomes and source-control loader proof.
  • logs-r4/: process logs; rig_run_r4.sh, rig_matrix_r4.py, rig_setup_r4.py, base_mobile_wrapper.py: exact commands and harness sources.
  • An evidence tool receipt could not be created: read tool://evidence returns Unknown tool: evidence in this resumed runtime. The raw execution artifacts above remain available.

Rig hygiene.

  • My targeted pytest (3 s, capped at 2 GB) overlapped another session's single-file pytest (relay-single-flight, about 150 MiB). I checked ps in the same command, but didn't wait for it to finish. The main rig run waited for a clear node.
  • Rig data dir /home/ec2-user/qa-rfp-core-r4 (337 MiB): this run created it, but the delete guardrail refused to remove it (it sits at the top level of $HOME). It's left in place for the manager or operator to remove. qa-rfp-core-r2 and -r3 weren't touched.

Copy the generated desktop fixture at local-operator-ui 5033f642352e93c30b699ff62e5bae0e1939da6c and verify its manifest before re-pinning. Every field other than desktop.sourceHash is unchanged, including all 700 cases and the latch: no condensation behavior port is needed. Core main still pins the preceding desktop generation, so merging main alone cannot repair the required cross-repo drift check.
Resolve the two generated condensation fixture/pin conflicts to the byte-verified local-operator-ui 5033f6423 generation. Preserve upstream changes and remote-files implementation; convergence proof/review required for the upstream-overlap files.
@damianvtran

Copy link
Copy Markdown
Owner Author

CI remediation — condensation parity

Head: f7337ad27944df4eef4a3026831fb1df9b386ab1 · Repair: bec264a29 · Main convergence: f7337ad27. Implementer: openai/gpt-6.1-sol (no fallback).

The Mobile web drift guard was red after desktop regenerated its fixture. Core main 97eed22c9 still pinned the preceding generation (bf6f8d5bf8f6… / cccd5a264d71…), so merging it alone could not repair latest drift.

Authoritative instructions: .github/workflows/mobile-web.yml fetches desktop main, then invokes scripts/check-condense-parity-latest.mjs --latest --require-sibling; that guard documents the re-copy/re-pin trigger. Desktop scripts/generate-condense-parity.mjs documents generation from shipped source and exact copies rather than hand editing. Copied the already-generated fixture at desktop 5033f6423 (#1876), independently checking manifest SHA-256 and sourceHash. Read fixture/manifest from that exact git object, not stale sibling working bytes.

Parsed old/new fixtures with only desktop.sourceHash removed compare equal: all other fields unchanged, including 700 cases, latch and modelSources. Desktop's own fixture commit diff independently confirms sourceHash-only movement. No behavioral port is required. Repair changes only local_operator/mobile/web/src/lib/condense.parity.json (exact generated bytes) and local_operator/mobile/web/src/lib/parity-sources.json (matching identities), 3 insertions / 3 deletions.

Condensation CI remediation bec264a29 (fixture identity only): desktop generated bytes from damianvtran/local-operator-ui@5033f642352e93c30b699ff62e5bae0e1939da6c; SHA-256 523e8a5959fa198795ee2e963139560906aa5f425ff9be77c167a369303a35f1, model sourceHash eb9d2c6d83bb277ee50d1018009a86d5218b5d5d9263bacfa25de1e339be5a82. Strict JSON comparison proves all 700 cases, latch, modelSources and every field except desktop.sourceHash unchanged. Commands below ran from local_operator/mobile/web, each under .venv/bin/python scripts/run_bounded.py --timeout 600 (repository-relative wrapper paths resolved absolutely):

  • node scripts/check-condense-parity-latest.mjs --latest --require-sibling --sibling <committed-desktop-fixture-snapshot>: existing branch bytes FAIL (rc=1) → copied fixture PASS (rc=0).
  • node scripts/check-condense-parity-latest.mjs: PASS, 2 fixtures match their pins.
  • node node_modules/vitest/vitest.mjs run src/lib/condense.test.ts --maxWorkers=1 --minWorkers=1 (--memory-mb 3000, isolated HOME/config, inherited CMUX_/LOP_ stripped): 1 file / 1,361 tests passed. Reused existing dependencies with byte-identical pnpm-lock.yaml; no install.
  • git diff --check: PASS. No executable code, catalogues or condensation decisions changed.

Convergence f7337ad27944df4eef4a3026831fb1df9b386ab1 merges core main 97eed22c9a81191ebee3a7490d3001d77a14e5ad (old head bec264a29, old merge base 89678ffc7aa536076cc3c16d9d204829a4e83a97). Two conflicts only: generated fixture and pins, resolved to the already-proven desktop bytes. Final head repeats strict drift guard/self-consistency PASS, condensation 1,361 passed, and test_settings_io.py -k "remote_files_cache_bytes or network.credentials.github.repositories" 4 passed. Network neighbors (test_remote_files_index.py, test_mesh_file_pull.py, test_create_idempotency.py): 48 passed / 1 failed. Failure at test_create_idempotency.py:114 sees an additional net_definitions frame; the single test fails serially too and with exact upstream relay source (SHA256 838ae42704fcc6bfdc1b13fda18ca3c125deddefa91d76a9b45380086488ec95) plus byte-identical upstream test, establishing it is not introduced by remote-file relay additions. This source-control run uses the current package for other modules; upstream relay refuses the branch-only remote-files slice as expected. No out-of-scope change made. Full logs retained in the delegated session's scratchpad://condensation-parity/.

rebase_proof.py is CONVERGENCE REVIEW NEEDED, not mechanical: 14/14 range-diff pairs =, but 6 fixture/pin +/- lines differ and upstream overlaps five files. All non-condensation PR +/- lines and file headers are byte-identical; manager owns one convergence delta review and CI/merge gate.

No guard weakening, executable repair, catalogue edits, full suite or dependency installation. Manager authorized merging main after GitHub became CONFLICTING; it is now MERGEABLE. Temporary dependency symlink removed. No merge-gate dispatch.

@damianvtran

Copy link
Copy Markdown
Owner Author

Rebase proof — bec264a..f7337ad

  • Base: origin/main; merge-bases 89678ffc7a → 97eed22c9a
  • git range-diff: 14 commit pair(s), =×14 → all =
  • PR patch (+/- and header lines): CHANGED (9407 lines)
  • Upstream files moved between merge-bases: 70; PR files: 94; overlap: local_operator/mobile/web/src/lib/condense.parity.json, local_operator/mobile/web/src/lib/parity-sources.json, local_operator/network/relay.py, local_operator/network/types.py, local_operator/settings_io.py
  • Verdict: CONVERGENCE REVIEW NEEDED — files: local_operator/mobile/web/src/lib/condense.parity.json, local_operator/mobile/web/src/lib/parity-sources.json, local_operator/network/relay.py, local_operator/network/types.py, local_operator/settings_io.py; patch lines differ (6 lines)

Re-run: python scripts/rebase_proof.py bec264a29038d72738cb38f92a68b7b91977f9c2 f7337ad27944df4eef4a3026831fb1df9b386ab1 --base origin/main

Merge/convergence details

Merged 97eed22c9a81191ebee3a7490d3001d77a14e5ad into bec264a29038d72738cb38f92a68b7b91977f9c2 without rebase, force push, reset or feature rewrite.

All conflicts (complete list):

  1. local_operator/mobile/web/src/lib/condense.parity.json — upstream's prior sourceHash versus latest desktop generation; resolved to exact UI 5033f6423 bytes.
  2. local_operator/mobile/web/src/lib/parity-sources.json — corresponding SHA256/sourceHash; resolved to that generation's verified manifest.

Semantic review scope for ONE convergence round, bec264a29038d72738cb38f92a68b7b91977f9c2..f7337ad27944df4eef4a3026831fb1df9b386ab1:

  • Both fixture files: preserve generated identity consistency; all cases/latch/other fields unchanged.
  • network/relay.py: upstream adds create idempotency/lookup and _PeerNoReply (subclass of existing MeshRefusal) for lost replies; branch adds only session-files slice registration and _owning_document routing. No textual conflict. Review dispatch registration coexistence and same-code timeout/refusal semantics.
  • network/types.py: upstream adds net_session_create_lookup under prompt capability; branch adds session-files peer/local op rows under view. Review additive whitelist/capability coexistence, no accidental loss.
  • settings_io.py: upstream bounds GitHub repository designation in _validate_github_repositories; branch adds separate desktop.remote_files_cache_bytes registry row. Review independent validator/registry registration.

Additional byte compatibility: comparing old-base..old-head and new-base..new-head +/- line multisets plus file headers with only the TWO fixture paths excluded is exactly equal. The proof's six changed lines are the fixture identity removals relative to the advanced base; non-condensation PR changes are preserved. The retained latest fixture is byte-identical across the merge and to UI 5033f6423 generated fixture.

Convergence f7337ad27944df4eef4a3026831fb1df9b386ab1 merges core main 97eed22c9a81191ebee3a7490d3001d77a14e5ad (old head bec264a29, old merge base 89678ffc7aa536076cc3c16d9d204829a4e83a97). Two conflicts only: generated fixture and pins, resolved to the already-proven desktop bytes. Final head repeats strict drift guard/self-consistency PASS, condensation 1,361 passed, and test_settings_io.py -k "remote_files_cache_bytes or network.credentials.github.repositories" 4 passed. Network neighbors (test_remote_files_index.py, test_mesh_file_pull.py, test_create_idempotency.py): 48 passed / 1 failed. Failure at test_create_idempotency.py:114 sees an additional net_definitions frame; the single test fails serially too and with exact upstream relay source (SHA256 838ae42704fcc6bfdc1b13fda18ca3c125deddefa91d76a9b45380086488ec95) plus byte-identical upstream test, establishing it is not introduced by remote-file relay additions. This source-control run uses the current package for other modules; upstream relay refuses the branch-only remote-files slice as expected. No out-of-scope change made. Full logs retained in the delegated session's scratchpad://condensation-parity/.

rebase_proof.py is CONVERGENCE REVIEW NEEDED, not mechanical: 14/14 range-diff pairs =, but 6 fixture/pin +/- lines differ and upstream overlaps five files. All non-condensation PR +/- lines and file headers are byte-identical; manager owns one convergence delta review and CI/merge gate.

@damianvtran

Copy link
Copy Markdown
Owner Author

CI remediation — directory-removal guard

Actual model: openai/gpt-6.1-sol (inherited; no fallback). Scope: b2c97ff2e82ba72dcdd59cbf7fa0a4261f72e878..928c9fc280ab109de37d9b6551a3bf13f7f3fd63. Hotfix allocation: ≤30 min.

Fixed the invariant, not just the allowlist. Before adding exceptions, fresh regressions proved two authoritative-store traversal paths (cache and remote-files symlinks), plus eviction of arbitrary non-digest *.bin files. This required a bounded product fix.

  • local_operator/remote_files/cache.py::_locked reuses network.sync.write_scope/open_directory with the config root as trusted anchor and create=False: every component below it is checked and opened no-follow with inode identity verification. The directory fd is held through locking, scanning, and removal. A pathname swapped after the initial check refuses; a pathname swapped after open cannot redirect the fd-relative unlink. Missing/unsafe cache paths evict nothing.
  • cache.py::evict selects only regular no-follow <32 lowercase hex>.bin entries, then removes that digest's .bin, .json, .peaks.json entries with os.unlink(..., dir_fd=held_cache_fd). The fd never comes from sessions/ or authoritative attachments/. Leaf links are never followed by unlink, and directories cannot be unlinked.
  • local_operator/server/utils/remote_session_files.py::land is unchanged. temp is the destination folder plus a fixed short .lop-download-<random>.part name, independent of the supplied asset name. os.open(... O_CREAT | O_EXCL, 0o600) happens before the cleanup try, so a pre-existing file/link/directory is refused without entering cleanup. Its finally unlinks only that newly-created temp name; final user files are published no-replace by os.link, never unlinked. File/link/directory collision regressions all pass; relay/HTTP tests verify successful and digest-mismatch cleanup.
  • Added exactly two call-shape allowances with reasons: cache.py::evict::os.unlink (the old path-based unlink was replaced), and remote_session_files.py::land::<path>.unlink. The AST checker and broad suppression rules are unchanged.

Actual execution (serial, isolated HOME/config; final run also isolated TMPDIR; env -i strips CMUX_/LOP_; own worktree venv verified; run_bounded --timeout 600 --memory-mb 3000):

BEFORE: tests/unit/session/test_no_session_deletion.py
1 failed, 613 passed in 55.71s
  FAIL test_no_session_directory_removal_outside_cleanup: the two reported call sites
BEFORE: test_remote_files_hardening.py -k 'eviction_never_follows or eviction_removes_only or download_cleanup_never'
3 failed, 3 passed, 17 deselected in 0.86s
  FAIL both cache-directory symlink cases + arbitrary non-digest eviction
AFTER: tests/unit/session/test_no_session_deletion.py
       tests/unit/network/test_remote_files_hardening.py
       tests/unit/network/test_remote_files_index.py
       tests/unit/network/test_remote_files_fetch.py::test_l_eviction_keeps_the_cache_within_its_budget
654 passed, 5 warnings in 62.04s (0:01:02)
[run_bounded] contract=cleanup-required rc=0 cause=child-exit cleanup=proven-empty
flake8 (the 3 changed files): exit 0
git diff --check: exit 0

The focused regression matrix covers both symlink parents, a swap between check/open, a swap after fd acquisition, regular-file/digest-only selection, preservation of sessions and authoritative attachments, and pre-existing download-temp file/link/directory collision safety. The index/fetch neighbors exercise real paired relays and HTTP routes, including download verification/no-overwrite, mismatch, and LRU eviction.

Changed paths: local_operator/remote_files/cache.py, tests/unit/network/test_remote_files_hardening.py, tests/unit/session/test_no_session_deletion.py. All other product paths, including download implementation and catalogues, are byte-identical to b2c97ff2e82ba72dcdd59cbf7fa0a4261f72e878 (verified by git object/diff paths). No workflow rearm/cancellation, full-suite run, rebase, merge or release.

Residual trust assumptions: the configured root is trusted (its own spelling may resolve through an operator-configured symlink, as in the existing sync primitive); this does not defend against privileged bind mounts or a same-user attacker replacing regular contents inside the private expendable cache. The unchanged download cleanup assumes a same-user attacker does not replace this call's unpredictable private temp name or move the destination directory during the call; remote input cannot select that name/directory. Independent narrow source review and real-path QA are manager-owned and not claimed here.

Local primary logs: /home/ec2-user/.local-operator/sessions/aa5e02deb40d/scratchpad/guard-before.log, /home/ec2-user/.local-operator/sessions/aa5e02deb40d/scratchpad/confinement-before.log, /home/ec2-user/.local-operator/sessions/aa5e02deb40d/scratchpad/final-tests.log.

@damianvtran

Copy link
Copy Markdown
Owner Author

Agent review — cache confinement delta

Reviewer: reviewer (lopdev), openai/gpt-6.1-sol, no fallback
Scope: b2c97ff..928c9fc
Worktree: /home/ec2-user/local-operator/.worktrees/remote-files-parity @ 928c9fc280ab109de37d9b6551a3bf13f7f3fd63, read-only; pushed head verified by PR API. Intended outcome remains remote-file parity within the existing pad/referenced-blob authority. This narrow round covers destructive cache confinement and the two exact CI-guard allowances, not another feature review.

Verdict: clean — no BLOCKER or MAJOR. No new findings. Existing MINOR/NIT findings remain deferred. All unaffected feature paths retained from the prior clean reviews.

Security/source verification

  • Trusted anchor, untrusted descendants. _locked now uses existing sync.write_scope(root, cache_root(root)) and sync.open_directory(..., create=False). I read the actual primitive: it resolves the trusted config root, walks every descendant with no-follow directory opens, and compares pre-open (st_dev, st_ino) to fstat after each open. Neither the cache parent nor remote-files leaf is resolved as a new trusted anchor. Missing/unsafe descendants refuse without creating directories.
  • Check/open races. Independently exercised swaps at the os.open boundary, after the primitive's pre-open stat, for both components. Symlink replacements return zero eviction and preserve authoritative bytes; real-directory replacements are refused by the inode comparison and preserve replacement bytes. The shipped check-to-open regression also passes.
  • After acquisition, no pathname re-resolution for deletion. Both os.scandir and all three unlinks use the retained directory fd. My additional probe swaps the cache path for an authoritative-attachments symlink at the first unlink, after scanning, with matching .bin, .json, and .peaks.json names planted in the target. Only the displaced cache's entries disappear; all three authoritative sentinels remain byte-identical. This complements the shipped post-acquisition/pre-scan timing test.
  • Constrained names and object types. Only exactly 32 lowercase hex digest stems ending in .bin and lstat-regular files enter the eviction set. Non-digest names, digest-named directories, and digest-named symlinks survive. Deletion operands are the validated digest plus fixed suffixes, with no separators. Sidecar symlinks are removed as directory entries, never followed: an independent probe confirms their authoritative target remains untouched. Hard links cannot redirect unlink into the other directory; unlink affects only the held cache directory entry.
  • Concurrency and LRU retained. Thread guard and per-directory flock still cover scan/sort/removal. The mtime ordering and keep check are unchanged. _run's _LANDED/active-task pins remain intact; the exact simultaneous-landings regression passes at budget zero. Real relay/HTTP budget neighbour also passes: five 200,000-byte landings under a three-file budget remove the two oldest and leave total bytes within budget.
  • Exact guard allowances, no checker weakening. Removed just the first two added _ALLOWED_ROWS tuples from the current guard AST and compared it to the previous head: the entire rest of the module is AST-identical. Independently classified each target file with the existing checker: cache.py::evict::os.unlink and remote_session_files.py::land::<path>.unlink each resolve to exactly one call and one declared allowance with a nonempty reason. No broad suppression or extra-removal allowance.
  • Download allowance. land is byte-identical to the prior head. The random fixed-prefix temp name is independent of the asset name; O_CREAT|O_EXCL creation precedes the cleanup try. Pre-existing file/symlink/directory collisions therefore cannot reach cleanup, as all three regression cells confirm. Final publication remains no-replace os.link; final user files are not unlinked. Trust remains the existing private/unpredictable temp-name and destination-directory assumption, not protection from an arbitrary same-user process replacing that private temp after creation.

Only cache.py and the two named test files differ from the previous head; every other product path is byte-identical, including download, serving, Library, and catalogues. Root-symlink spelling remains trusted exactly as in the pre-existing sync primitive; privileged mount changes and arbitrary same-user mutation of cache contents are not claimed to be isolated.

Fresh execution on the pushed head

Two serial run_bounded.py invocations, node checked clear of other heavy runs before dispatch, existing worktree .venv, fresh scratch HOME/config under env -i, no inherited CMUX_/LOP_, no installs or repository writes:

  1. --timeout 240 --memory-mb 2500, pytest -n0 -q -p no:cacheprovider -p no:randomly:
    • Exact new hardening tests: both cache-component symlinks, digest/sidecar selection, post-fd path swap, pre-open swap, digest symlink/directory exclusion, three download-temp collisions.
    • Existing test_r1_3_one_landing_never_evicts_another_before_its_lookup.
    • Six independent scratch probe cells: both component swaps at kernel open; path swap at first unlink covering every sidecar; sidecar symlinks; missing-cache no-creation; exact new guard entries/counts/reasons.
    • Actual: 16 passed, 16 deselected in 0.82s; wrapper exit 0, wall 1.5s.
  2. --timeout 180 --memory-mb 2500, same pytest isolation:
    • Two independent real-directory inode-swap cells at kernel open.
    • test_remote_files_fetch.py::test_l_eviction_keeps_the_cache_within_its_budget (real relay/HTTP neighbour).
    • Actual: 3 passed, 5 warnings in 1.79s; wrapper exit 0, wall 2.6s.

Total: 19 passed, covering 8 independent reviewer probe cells and 11 shipped regression/neighbour cells. I did not duplicate the 614-test guard suite or the implementer's 654-test run; the before-fix failing controls are sourced to the implementer's evidence, not claimed as rerun here. Independent external two-relay+viewer smoke remains QA's stream.

Cleanup receipts: run-8302f8548d25 / completion f07fd83bad9344b3b392d8e489bbf3d0, and run-c6b028918544 / completion b4df6c3bc45a4e269fbd15dd01fe94e3: both cleanup=proven-empty. Exact leaders/PGIDs 3622876 and 3623105 absent (kill -0 fails). No dev server/browser started by this reviewer. Final HEAD unchanged and git status --porcelain empty; no repo evidence/untracked receipt created.

@damianvtran

Copy link
Copy Markdown
Owner Author

QA report — cache confinement delta

Verdict: PASS. No new BLOCKER or MAJOR. Independent real two-relay + viewer-HTTP eviction probes reproduce the three defects with the previous cache implementation and pass with the fixed implementation. Both directory-swap timing probes executed successfully, not BLOCKED.

Scope: b2c97ff2e..928c9fc28.
Worktree: /home/ec2-user/local-operator/.worktrees/remote-files-parity @ 928c9fc280ab109de37d9b6551a3bf13f7f3fd63.
Independent QA: qa-tester (lopdev), session 4af48960f0ed.
Actual served model: openai/gpt-6.1-sol; no Anthropic request or fallback in this delta run.

Real path / negative-control provenance

Reused my own existing synthetic R4 relay pair/config/data. Two real lop network serve processes, plus a viewer daemon running the normal lop serve CLI from the shared worktree .venv. All three have isolated HOME/config/CWD, env-i-derived environments with no inherited CMUX/LOP context, and separate process-group leaders. No install, clone, extra worktree, production data, browser or full matrix.

Every attack probe is driven by a real authenticated viewer HTTP GET for the same 200,002-byte remote digest. The driver removes only that synthetic rig's evictable copy before each request, forcing a real relay fetch. The normal path is:

HTTP attachment → remote fetch → _after_pull → cache.evict → normal response.

QA-only viewer.py wraps the eviction call at this boundary to set an explicit test budget, plant malicious filesystem layouts, record outcomes, and restore renamed paths in finally. Requests, relay transfer, cache lookup, and responses remain real; no HTTP response is fabricated. Head arms delegate to the current real cache.evict. Base control arms delegate only that call to the exact cache.py bytes from git show b2c97ff2e:local_operator/remote_files/cache.py, loaded as an independent module; every other module stays head's. Saved old-source SHA-256: f8fc62c6942e073bc4b59a06163d2788738fda6b0f79de626d4f6ab125945167.

All symlink targets are new synthetic authoritative-store decoys under this session's scratchpad, never the operator's real data. They contain a valid digest .bin, .json, .peaks.json, and a non-digest notes.bin. Checks compare names and complete file bytes before/after. These are representative attacker destinations, not a claim that production attachments were exposed during QA.

Actual matrix

Attachment request used for all nine eviction arms:
GET /v1/desktop/sessions/5afe0000a1b2/attachments/bd46ffd4deab99c1b24c989a5dccb981.
Every arm returned 200 with full SHA-256 matching the owner fixture.

Surface / arm Injection / command Actual eviction result and side effects Verdict
Negative control: old source, symlinked cache parent At real eviction boundary, temporarily replace b/cache with symlink to planted parent whose remote-files points to decoy attachments; budget 0 Reproduced bug: 48 bytes counted freed; all four decoy files disappeared, .lock created in attacker target. HTTP still 200 + matching sha PASS — discriminates
Head: same symlinked cache parent Identical attack against current eviction 0 bytes freed; every target filename and byte unchanged; HTTP 200 + matching sha PASS
Negative control: old source, symlinked remote-files directory Temporarily redirect that directory to decoy attachments; budget 0 Reproduced bug: 48 bytes counted freed; all four target files disappeared, .lock created. HTTP 200 + matching sha PASS — discriminates
Head: same symlinked remote-files Identical attack against current eviction 0 bytes freed; target completely unchanged; HTTP 200 + matching sha PASS
Head: directory swap between scope check and open Independently wrap sync.open_directory to rename the legitimate cache and install the target symlink immediately before calling the real open; budget 0 0 bytes freed; target unchanged; real HTTP fetch/response remains 200, sha matches PASS
Head: directory swap after fd acquisition Call real sync.open_directory, then rename cache and install target symlink before returning the fd to real eviction 33,120,916 bytes freed from the original pinned directory; both planted 32-KiB digest blobs removed there; attacker target unchanged; non-digest and leaf symlink preserved. HTTP 200 + matching sha PASS
Negative control: old source, non-digest and leaf symlink selection Plant arbitrary-not-a-digest.bin and valid-digest .bin symlink to decoy blob; budget 0 Reproduced selection bug: arbitrary non-digest and leaf symlink were removed; target bytes remained. HTTP 200 + matching sha PASS — discriminates
Head: same selection probe Same planted files; budget 0 65,536 bytes freed (two regular digest blobs); non-digest preserved, digest leaf symlink preserved, target unchanged. Pulled blob remains pinned at 200,002 bytes PASS
Head: legitimate-directory LRU / budget Plant two 32-KiB regular digest blobs with ordered mtimes; explicit budget 232,770; fresh remote digest kept by real fetch pin 32,768 freed; oldest removed, newer retained; regular digest total exactly 232,770 bytes, target/non-digest/symlink unchanged PASS
Remote image preview after attack/eviction passes GET …/attachments/1301ca1c70c37742f1ed81a0f2cf73b9 200 image/png, full SHA-256 matches owner image PASS
Download unchanged POST …/files/<report.md>/download 200, name report.md, created:false (prior same-byte download already existed); on-disk viewer Downloads SHA-256 matches owner fixture PASS
Exact process cleanup kill -- -<pgid>; wait/reap; kill -0 Viewer 3623845, relay B 3623827, relay A 3623796 gone; alive:[]; wrapper independently reports cleanup proven empty PASS
Freshness / source hygiene HEAD and full git status --porcelain after run Briefed 928c9fc280ab109de37d9b6551a3bf13f7f3fd63; fully clean worktree PASS

The zero-budget arms intentionally retain the just-fetched digest through the production keep pin. They are not a claim that pinned bytes must immediately satisfy a zero-byte total. The nonzero-budget arm proves budget enforcement while respecting that pin and oldest-first ordering.

Commands / execution

One heavy-run precheck was clear, then one serial bounded real-path run:

env -i PATH=/usr/local/bin:/usr/bin:/bin LANG=C.UTF-8 PYTHONUNBUFFERED=1 \
  HOME=/home/ec2-user/qa-rfp-core-r4/home-setup \
  LOCAL_OPERATOR_CONFIG_DIR=/home/ec2-user/qa-rfp-core-r4/setup-config \
  PYTHONPATH=<own-pad>/core-cache-confinement-qa:<worktree> \
  <worktree>/.venv/bin/python <worktree>/scripts/run_bounded.py \
  --timeout 300 --memory-mb 2000 -- \
  <worktree>/.venv/bin/python <own-pad>/core-cache-confinement-qa/smoke.py \
  /home/ec2-user/qa-rfp-core-r4

Actual: python rc=0 wall=20.6s; contract=cleanup-required rc=0 cause=child-exit cleanup=proven-empty. Raw matrix 13/13 PASS (including the three explicitly discriminating base controls).

Compatibility / retained evidence

Independently verified the exact changed-path list from git objects:

  • local_operator/remote_files/cache.py — the only executable product change;
  • tests/unit/network/test_remote_files_hardening.py;
  • tests/unit/session/test_no_session_deletion.py.

All other product paths, including download implementation, serving/type/sandbox/auth routes, relay registration, and catalogues, remain unchanged from b2c97ff2e. Retain prior convergence-2 QA and its source-compatible round-4 evidence for unaffected surfaces. Q4-1 remains deferred/outside this delta. The independent reviewer owns exact AST guard/allowlist and source-security audit; I did not duplicate the 614-case guard suite. The coder's 654-test result is explicitly their evidence, not QA execution here.

Primary evidence / cleanup / gaps

Directory: /home/ec2-user/.local-operator/sessions/4af48960f0ed/scratchpad/core-cache-confinement-qa/

  • PREFLIGHT.md, smoke.py, viewer.py, copied rig_matrix.py, ports.txt: exact setup/injection plan and harness.
  • results.json — SHA-256 ab5feded93b9bc1abc0fb9375562faa509d9b9e964507762e9422a81647af869.
  • events.jsonl — per-arm filename lists and full-byte-comparison outcomes, SHA-256 f8e43de31bb03cbd9104769a9f4025615d2c8c66829580284b59de331f6a9e8a.
  • run.log — SHA-256 01fb39aeeeaf62f58038ed2e1f855e187a134b39c8af98eb124a6313b05e40bf.
  • base_cache.py: exact old implementation for the negative controls; compatibility.json: changed-path proof and artifact hashes; logs/: all three daemon logs.

Verified both rig cache directories are restored as ordinary directories and no held/renamed paths remain. Removed only the exact planted non-digest/link/dummy-digest leaves created by these probes; retained scratch decoys/evidence. No repo receipts/untracked files were created. The old rig root's final deletion denial was not retried.

Unverified by design: Windows/non-POSIX behaviour, privileged bind mounts, arbitrary same-user tampering inside trusted regular cache contents, new rendered UI/WAN/native/RSS evidence. Download check covers unchanged same-byte success/on-disk integrity; no new temp-collision matrix was run in QA. Neither directory-swap probe is blocked: both ran against actual HTTP-triggered eviction.

@damianvtran damianvtran added ready-to-merge Run the full-suite merge gate on this PR and removed ready-to-merge Run the full-suite merge gate on this PR labels Nov 12, 2026
Copy exact generated fixture bytes from local-operator-ui 72043c4ec (regenerated in b0edf02e5, #1877) and verify manifest identities. All 700 cases, latch, modelSources and every field except desktop.sourceHash remain unchanged. The hashed source delta only adds locale formatting/bidi helpers; no condensation model port is required. Strict drift guard fails on prior bytes and passes after; 1361 condensation tests pass. Remote-files security source remains byte-identical to 928c9fc.
@damianvtran

Copy link
Copy Markdown
Owner Author

CI remediation — condensation parity refresh

Head: e7890a59cb7895edc3d3eba9ea1e870181cce135 · Scope: 928c9fc280ab109de37d9b6551a3bf13f7f3fd63..e7890a59cb7895edc3d3eba9ea1e870181cce135. Implementer: openai/gpt-6.1-sol (no fallback). Bounded refresh only; no product behavior or catalogue change.

Source and actual-source diagnosis

  • Read exact generated fixture and manifest from desktop main 72043c4ecbfddd734abec28694b05a163786bf0c, not sibling working bytes. The fixture regeneration landed in b0edf02e5 (feat(mesh): approval record API + forwarded allow + unattended grant (slice a) #1877, live-locale formatting).
  • Verified fixture SHA-256 5e0117590743d9e79682bf044c4c07eb8c4053a5ffc9a17765d28545b43fea73 against its source manifest, plus embedded/model sourceHash 5dde7aa3e8d7f53cf61d2386a7b3fd0802577bf0e1a5a310b06b9804a520cebb.
  • Compared parsed fixture objects after removing ONLY desktop.sourceHash: all other fields exactly equal, including 700 cases, latch and modelSources. Desktop's own fixture diff independently contains only the sourceHash replacement.
  • Inspected the actual hashed-source diff from prior desktop 5033f6423 to 72043c4ecb: only src/i18n/bidi.ts, src/i18n/format.ts, src/i18n/index.ts (43 insertions, 1 deletion). These add isolateClock, FULL_NUMERIC_STAMP and its export/imports. No condensation algorithm file changed; no behavioral port required.

Changed paths (complete list)

  • local_operator/mobile/web/src/lib/condense.parity.json: exact generated desktop bytes.
  • local_operator/mobile/web/src/lib/parity-sources.json: matching fixture SHA-256 and sourceHash.

3 insertions / 3 deletions. git diff --name-only 928c9fc280ab109de37d9b6551a3bf13f7f3fd63..e7890a59cb7895edc3d3eba9ea1e870181cce135 contains only those two JSON paths; therefore remote-cache security sources/tests and all other tracked files remain byte-identical to 928c9fc280ab109de37d9b6551a3bf13f7f3fd63. Parent-supplied CLEAN review and real-QA 13/13 on that source are retained by this byte compatibility; those security flows were not unnecessarily rerun here.

Fresh targeted execution
Each command below ran under .venv/bin/python scripts/run_bounded.py --timeout 600 --memory-mb 3000 -- (absolute repository/script paths used). No concurrent pytest/vitest/pyright process was observed before dispatch.

node scripts/check-condense-parity-latest.mjs --latest --require-sibling \
  --sibling <snapshot from exact desktop 72043c4ecb git object>
BEFORE, existing 928c9fc28 bytes: rc=1
  fixture 523e8a5959fa vs desktop 5e0117590743
  sourceHash eb9d2c6d83bb vs desktop 5dde7aa3e8d7
AFTER exact byte copy + re-pin: rc=0, in step with desktop

node scripts/check-condense-parity-latest.mjs
→ rc=0; 2 fixtures match their pins

node node_modules/vitest/vitest.mjs run src/lib/condense.test.ts \
  --maxWorkers=1 --minWorkers=1
→ 1 test file passed, 1,361 tests passed; rc=0

git diff --check
→ PASS

Condensation replay used isolated HOME/config and stripped inherited CMUX_/LOP_; existing dependencies reused after byte-identical lockfile check, no installation. Temporary dependency link removed. All bounded runs reported cleanup proven empty. Evidence logs/source diff: delegated session scratchpad://condensation-parity-refresh/.

Recurring reproducibility blocker — separately owned follow-up recommendation, not implemented here
At this exact head .github/workflows/mobile-web.yml:120 declares CONDENSE_PARITY_REF: main; line 125 constructs the raw GitHub URL with that moving ref. Thus unchanged core commits can turn red after unrelated desktop merges. This refresh does not solve that source-level reproducibility problem and is not authorization for unlimited pin-refresh cycles.

Recommend a separate owner change the required reproducible parity check to fetch an actual declared immutable desktop source revision, then enforce manifest consistency: fetched fixture SHA-256, embedded sourceHash and copied pin must agree, and source identity must be validated against that same revision (using the generator's declared hashed inputs). Keep freshness against current desktop main as an explicit separately-owned update detector/port workflow, not silently removed or confused with reproducible compatibility. Add negative controls for tampered fixture/manifest, incorrect revision/source identity and failed fetch; retain hard failures. No workflow/source-policy change, extra PR/worktree, or guard weakening is included here.

PR was MERGEABLE before and after refresh; no main merge. No full suite, re-arm, cancellation or gate dispatch. Parent owns the next gate; stop refresh cycles here pending separately owned source-level work.

@damianvtran

Copy link
Copy Markdown
Owner Author

Agent review — condensation refresh CI delta

Reviewer: reviewer (lopdev), openai/gpt-6.1-sol, no fallback
Scope: 928c9fc..e7890a5
Worktree: /home/ec2-user/local-operator/.worktrees/remote-files-parity @ e7890a59cb7895edc3d3eba9ea1e870181cce135, read-only. Pushed head verified through the PR API; final local HEAD matches and git status --short is empty. Operator request/acceptance/non-goals unchanged; this is only the red-CI generated fixture/pin refresh, not product or feature re-review.

Verdict: clean — no BLOCKER or MAJOR introduced by this delta. Terminal within this allowed scope. Existing MINOR/NIT findings remain deferred.

Independently verified provenance and retention

I read the implementer's primary evidence, then verified against actual Git objects in the named repositories:

  • Exactly two generated JSON paths, 3 insertions / 3 deletions: local_operator/mobile/web/src/lib/condense.parity.json and local_operator/mobile/web/src/lib/parity-sources.json. No other tracked path differs. Thus cache confinement, download, product serving/Library, tests, catalogues and workflow sources are byte-identical to reviewed 928c9fc28. Existing independent QA is retained on that proof; no extra QA requested or rerun.
  • Fixture is byte-identical to damianvtran/local-operator-ui@72043c4ecbfddd734abec28694b05a163786bf0c:scripts/fixtures/condense.parity.json (read with git show, not sibling working bytes).
  • Computed fixture SHA-256 5e0117590743d9e79682bf044c4c07eb8c4053a5ffc9a17765d28545b43fea73 agrees with that commit's generated manifest and the new core pin. Embedded sourceHash, source manifest, and core pin agree on 5dde7aa3e8d7f53cf61d2386a7b3fd0802577bf0e1a5a310b06b9804a520cebb.
  • Independently recomputed sourceHash using the source generator's exact algorithm (filename, NUL, exact committed bytes, NUL, for every declared modelSources input): PASS against that immutable UI commit, not merely self-consistent pins.
  • Parsed old/current fixtures, removed only desktop.sourceHash, asserted full object equality: PASS, all 700 cases, latch, modelSources and all remaining fields unchanged.
  • Compared every declared hashed-source input at prior UI 5033f6423 versus 72043c4ec: only src/i18n/bidi.ts, src/i18n/format.ts, src/i18n/index.ts changed (43 insertions / 1 deletion). Read those hunks: isolateClock, numeric timestamp options and exports/imports; no condensation algorithm change. No behavioral port hidden by re-pinning.

Fresh exact guard execution

Created a scratch-only sibling fixture/manifest snapshot from those exact UI Git objects. Node checked clear of other heavy runs; isolated scratch HOME/config with env -i, reused existing worktree tools, no install/new tree/repo edits:

.venv/bin/python scripts/run_bounded.py --timeout 60 --memory-mb 800 -- \
  node local_operator/mobile/web/scripts/check-condense-parity-latest.mjs \
  --latest --require-sibling --sibling <reviewer scratch exact-72043c4ec snapshot>
→ in step with desktop (fixture 5e0117590743…, sourceHash 5dde7aa3e8d7…)
→ rc=0, wall=0.2s, cleanup=proven-empty

node local_operator/mobile/web/scripts/check-condense-parity-latest.mjs
→ 2 fixtures match their pins; rc=0

Run run-f845705e2806, completion 295571484c4d4ae2885097212e11349a; exact leader/PGID 3631867 absent (kill -0 fails). No server/browser started. The implementer's 1,361 replay tests and strict-before FAIL are sourced to the primary evidence above, not claimed as independently rerun. No full suite, product smoke or capture repeated.

Recurring CI reproducibility issue — outside this delta, separately owned

Directly inspected unchanged .github/workflows/mobile-web.yml: line 120 sets CONDENSE_PARITY_REF: main; line 125 builds the raw GitHub URL from that moving ref, and lines 126–128 fetch fixture and manifest separately. Therefore an unchanged core head can turn red when UI main advances; two fetches can also observe different revisions. This valid refresh does not fix that pre-existing source-identity/reproducibility problem and is not authorization for indefinite refresh loops.

Recommend a separate owner implement a declared immutable UI source revision for the required reproducible compatibility check, bind fixture/manifest/sourceHash validation to that same revision, retain hard-failure negative controls, and keep latest-main freshness as an explicit separately owned update detector. Do not weaken or modify the workflow in this scoped PR delta. This review finishes the allowed refresh scope; manager owns the next exact-head gate and escalation if moving main invalidates it again.

@damianvtran damianvtran added ready-to-merge Run the full-suite merge gate on this PR and removed ready-to-merge Run the full-suite merge gate on this PR labels Nov 12, 2026
@damianvtran

Copy link
Copy Markdown
Owner Author

CI remediation — wheel size

Commit: f10d7c17956ead2ae06b1d6bb356b04fee771556 (parent e7890a59cb7895edc3d3eba9ea1e870181cce135). Implementer/model: openai/gpt-6.1-sol (inherited; no fallback).

Exact sites / rationale. scripts/slim_wheel.py now passes compresslevel=9 to both existing DEFLATE writestr calls (payloads and RECORD). This spends build CPU rather than PyPI storage, without changing the existing comment/AST validation, member set, order, metadata, RECORD or any uncompressed payload. tests/unit/test_slim_wheel.py::test_repack_uses_maximum_lossless_compression derives the expected compressed size using the standard-library ZIP writer, verifies that its structured fixture distinguishes default level 6 from level 9, and checks unchanged member contents. The original slimmer FAILS this new regression (executed negative control). No package exclusions: the existing mobile packaging contract intentionally ships the raw tests and fixtures for installed-tree builds. No product/security/runtime/localized sources, fixtures, version or size budgets changed.

Primary failed CI: run 43319909082 / job 131567736599, job log read directly through the jobs logs API. Original python scripts/slim_wheel.py output: 24,518,066 → 18,350,518 bytes, 438 bytes above the unchanged 18,350,080-byte / 17.5 MiB cap. CI uses python -m build; its frontend logged build 1.6.1.

Actual local build and paired repack: own worktree .venv Python 3.12.15; existing uv 0.13.5 frontend, cached setuptools 84.2.1 / wheel 0.48.0 backend made available via PYTHONPATH without installation. The own venv has no build frontend, so this explicitly uses uv build --wheel --no-build-isolation --offline --python .venv/bin/python --out-dir dist/ci-size-raw, not python -m build. Same setuptools PEP 517 backend; no dependencies installed. Built wheel 24,510,608 bytes was copied into dist/ci-size-before/ and dist/ci-size-after/; the exact original script from parent HEAD and the changed script were run against those identical copies. Each heavy command ran serially under scripts/run_bounded.py --contract=command-status --timeout 600 --memory-mb 4000 with isolated HOME and inherited CMUX/LOP environment stripped.

Same built-wheel experiment Original repack Level-9 repack
Final bytes 18,344,938 18,251,825
Wall seconds (/usr/bin/time) 31.51 33.95
User CPU seconds 30.97 33.44
System CPU seconds 0.51 0.47
Peak RSS KiB 330,844 330,804

Measured saving: 93,113 bytes; final headroom: 98,255 bytes. python scripts/check_dist_size.py --budget-mib 17.5 dist/ci-size-after PASS, existing 16 MiB warning still emitted. The local baseline is 5,580 bytes smaller than the failed hosted CI wheel; do not mistake it for a byte-exact reproduction of that hosted build. Further check: the local wheel's entire member set matches all 3,165 actual adding entries in the hosted build log. The paired local comparison controls frontend/backend differences by using the same built wheel. Fresh required CI remains the authoritative hosted result.

Archive and runtime proof: both wheels pass ZipFile.testzip() (CRC). Their ordered member lists, timestamps, external attributes, every uncompressed SHA-256 and RECORD bytes are identical. Every RECORD SHA-256/size was independently validated: 1,400 Python modules, 1,759 non-Python package resources, plus six dist-info members retained. No runtime asset removed or rewritten. Direct imports from the new wheel (asserted local_operator.__file__ points inside the wheel): local_operator, local_operator.cli, local_operator.clients.radient PASS; importlib.resources reads of mobile package.json, condensation fixture, shipped i18n manifest and sound catalog PASS; the wheel-loaded real CLI prints v0.102.1 with --version. This is a direct wheel import/CLI probe, not an installed-wheel/server smoke; independent QA owns that next.

Targeted checks: isolated serial pytest -q -n 0 tests/unit/test_slim_wheel.py tests/unit/mobile/test_web_package_data.py under the same 600-second / 4000-MiB wrapper: 25 passed in 1.09s. Original-slimmer negative control PASS (new regression rejects it). Changed-file flake8, compileall and git diff --check PASS. Local black is unavailable; no formatter/dependency installation attempted. No full suite, workflow rearm/cancel, merge or release performed; full-gate run 43320385303 untouched.

Independent QA artifact: /home/ec2-user/local-operator/.worktrees/remote-files-parity/dist/ci-size-after/local_operator-0.102.1-py3-none-any.whl, SHA-256 f0fc53e66e3aaed9df0f6d4ae7784636e8a6724089abaf30f1f7db17f9768da6. Evidence in delegated session ec7a1e1295e4 scratchpad: wheel-measurements.json, ci-packaging-failure.log, wheel-build-timed.log, repack-before.time, repack-after.time, wheel-tests.log, measure_wheels.py.

@damianvtran

Copy link
Copy Markdown
Owner Author

Agent review — wheel packaging delta

Reviewer: reviewer (lopdev), openai/gpt-6.1-sol, no fallback
Scope: e7890a5..f10d7c1
Worktree: /home/ec2-user/local-operator/.worktrees/remote-files-parity @ f10d7c17956ead2ae06b1d6bb356b04fee771556, read-only. PR API confirms this pushed head; final HEAD unchanged and git status --porcelain empty. Request/acceptance/non-goals unchanged. Narrow scope: required-CI wheel size, lossless compression and its regression; no feature/native/full-matrix rerun.

Verdict: clean — no BLOCKER or MAJOR. No new findings; existing MINOR/NIT findings remain deferred. Product/security reviews retained by unchanged source bytes.

Code and policy

Only two executable call arguments change: compresslevel=9 on ZipFile.writestr for payload members and rebuilt RECORD. Both stay standard ZIP_DEFLATED. Member selection, Python AST/line-count verification, resource handling, ordering, RECORD generation, source timestamps/external attributes, output permissions and atomic replacement are unchanged. No selective drop/exclusion, cap increase, or packaging-policy weakening.

The new regression uses structured data that independently compresses smaller at level 9 than level 6, verifies the slimmer's exact compressed size against level 9, and asserts all uncompressed member bytes remain equal. I executed it against the parent script loaded from its exact Git object in a scratch-only negative control: the new regression rejects the old script, as required. Existing whole-file tests also cover RECORD validity, metadata/mode retention and fail-closed behavior.

git diff --name-only e7890a59c..f10d7c179 contains only scripts/slim_wheel.py and tests/unit/test_slim_wheel.py. All product/cache/download/security/catalogue/fixture/workflow tracked bytes are identical to the previously reviewed head.

Independent artifact comparison

Opened the existing same-input before/after wheels; no rebuild or install performed:

  • Before: dist/ci-size-before/local_operator-0.102.1-py3-none-any.whl — 18,344,938 bytes.
  • After: dist/ci-size-after/local_operator-0.102.1-py3-none-any.whl — 18,251,825 bytes; computed SHA-256 f0fc53e66e3aaed9df0f6d4ae7784636e8a6724089abaf30f1f7db17f9768da6, matching the briefed QA artifact.
  • All 3,165 ordered member names identical, with matching timestamps, external/internal attributes, creator system, compression method, flags, extra fields and comments.
  • Every uncompressed member SHA-256 identical, including exact RECORD bytes. Both archives pass CRC (testzip()); every RECORD row's digest/size verifies independently, and RECORD membership exactly matches the archive.
  • All members retained, including mobile raw tests/fixtures (153 mobile paths matching test/parity names in this inventory). No runtime resources are removed to make the budget.
  • 93,113-byte saving; unchanged 17.5-MiB cap is 18,350,080 bytes, leaving 98,255 bytes of local headroom. check_dist_size.py --budget-mib 17.5 dist/ci-size-after passes and still emits its existing 16-MiB warning.

These are the actual paired local artifacts, not a claim of byte-identical reproduction of the hosted failing build. The implementer's evidence discloses the local baseline being 5,580 bytes smaller than hosted and the frontend/backend used. Fresh hosted CI remains authoritative for the hosted cap; independent installed-wheel/relay smoke belongs to QA.

Fresh targeted execution and cleanup

Node checked free of other heavy runs. Existing worktree .venv, isolated scratch HOME/config under env -i, inherited CMUX_/LOP_ stripped, scratch-only probes, no dependency installation or repository writes. Two serial run_bounded.py --timeout 180 --memory-mb 2500 invocations:

  1. Independent artifact inventory/CRC/RECORD/provenance verification plus parent-script negative control: PASS, rc=0, wall 1.9s.
  2. python -m pytest -n0 -q -p no:cacheprovider -p no:randomly tests/unit/test_slim_wheel.py: 18 passed in 0.55s; rc=0, wrapper wall 1.2s.

Run receipts run-d838e86b0270 and run-3c84653a809c, completions f79c74565f354dffb9c1f57c9e024b94 and ce7ef8ebece64f56b896ea88a941a355: both cleanup=proven-empty. Exact leaders/PGIDs 3660990 and 3661037 absent (kill -0 fails). No dev server/browser started. No whole suite, rebuild, workflow action, merge or extra feature review. This review is terminal for the packaging-only delta.

@damianvtran

Copy link
Copy Markdown
Owner Author

QA report — installed wheel delta

Verdict: PASS. No BLOCKER or MAJOR. Independently installed the supplied local wheel offline with no dependencies installed, verified complete lossless archive equivalence and RECORD/CRC, then exercised real installed-source CLI, two relays and viewer HTTP routes. No feature matrix/native/capture rerun.

Scope: e7890a59c..f10d7c179.
Manager worktree: /home/ec2-user/local-operator/.worktrees/remote-files-parity @ f10d7c17956ead2ae06b1d6bb356b04fee771556, unchanged before/after.
QA/model: independent qa-tester (lopdev), session 4af48960f0ed; actual served model openai/gpt-6.1-sol, no fallback in this delta.

Artifact and installation provenance

Artifact: /home/ec2-user/local-operator/.worktrees/remote-files-parity/dist/ci-size-after/local_operator-0.102.1-py3-none-any.whl.
Independently measured SHA-256: f0fc53e66e3aaed9df0f6d4ae7784636e8a6724089abaf30f1f7db17f9768da6 — exactly the supplied identity.

Installed into my own scratch target using the existing worktree Python 3.12 dependency environment, without modifying it:

<worktree>/.venv/bin/python -m pip install \
  --no-index --no-deps --no-compile \
  --target <own-pad>/core-wheel-qa/installed \
  <worktree>/dist/ci-size-after/local_operator-0.102.1-py3-none-any.whl

Actual installer: Successfully installed local-operator-0.102.1, rc 0, 1.5 s. No network dependencies, build, clone or new worktree.

Actual import root for the probe, CLI, both real relays, and viewer daemon:
/home/ec2-user/.local-operator/sessions/4af48960f0ed/scratchpad/core-wheel-qa/installed/local_operator/__init__.py.

Each daemon entry shim hard-asserts this installed path before calling the normal CLI; each process log records the installed path and exact CLI argv. PYTHONPATH contains only this installed target (and the own scratch helper directory in the parent driver), not editable worktree source. Child cwd/HOME/config are synthetic isolated roots, env-i-derived environment has no inherited CMUX/LOP context, dependencies come from the existing venv. Independent post-install comparison verifies all 3,159 local_operator/ installed package files equal their wheel payload SHA-256.

Matrix — actual archive, CLI, resources, HTTP

Surface Command / path Actual output / side effect Verdict
Artifact identity and unchanged size cap SHA-256 and byte count Exact supplied hash; 18,251,825 B <= 18,350,080 B, 98,255 B headroom. Existing before-wheel 18,344,938 B; same-input saving 93,113 B PASS
Archive CRC ZipFile.testzip() on before and after Both return None PASS
Complete lossless inventory Ordered names, ZipInfo metadata and every uncompressed SHA-256 compared with existing before-wheel 3,165/3,165 ordered members identical, same timestamps/attributes/creation/extraction versions/flags/comments/extras/compression method, all uncompressed payload hashes equal; no selective drops PASS
RECORD integrity Compare RECORD bytes; validate each SHA-256/size entry RECORD bytes identical; 3,164 payload rows validated; only RECORD itself has the standard empty hash/size PASS
Installed bytes Hash installed package files against wheel 3,159/3,159 package files match; six dist-info members are included in archive equivalence PASS
Real installed CLI Installed-path-asserting entry → lop --version v0.102.1, with installed-source provenance logged PASS
Static mobile resources importlib.resources.files('local_operator') reads mobile/web/index.html: 2,194 B; mobile/web/package.json: 1,517 B; condensation fixture: 994,782 B; mobile/static/mark.png: 7,211 B, PNG signature verified; all reads from installed target PASS
Thumbnail implementation resource Read installed server/routes/desktop_library.py 23,149 B, SHA recorded PASS
Installed daemon capabilities GET /v1/capabilities 200, features.remote_files:1 PASS
Installed remote attachment / actual transfer GET /v1/desktop/sessions/5afe0000a1b2/attachments/6f913c95eb1d92cd0d854ed425d47c53 200, 200,003 B, SHA matches owner fixture; cached_before:false → cached_after:true PASS
Auth negative control Same GET without bearer 401 PASS
Installed remote Files index GET /v1/desktop/sessions/5afe0000a1b2/files?limit=100 200, ready, remote:true, 18 rows, all remote/owner/readability markers present; unsupported_reason null PASS
Installed download POST /v1/desktop/sessions/5afe0000a1b2/files/6826bcb52c6e7940/download 200, report.md, created:false (existing identical download reused); actual synthetic viewer Downloads file SHA matches owner fixture PASS
Installed thumbnail route Real image bytes pulled; GET /v1/desktop/library/assets/{qa-shot}/thumbnail 200 image/webp, 64 B PASS
Installed Library sandbox GET /v1/desktop/library/assets/la_ed4b57e63ce1c07b/content (evil.svg) 200 octet-stream, CSP sandbox + nosniff, SHA matches PASS
Exact cleanup Kill each recorded process group; reap; kill -0 Relays 3661341, 3661367, viewer 3661393 gone; alive:[]; wrapper independently proves empty PASS
HEAD/source hygiene HEAD and tracked status after run Same briefed SHA, 0 tracked changes PASS

Bounded execution and retained scope

Node heavy-run check clear before serial dispatch. Installation bounded at 180 s/2,000 MiB; archive/CLI verification and installed HTTP smoke each bounded at 300 s/3,000 MiB. Actual archive verification 2.7 s, HTTP smoke 9.4 s, both rc 0 with cleanup=proven-empty. Two real installed-wheel lop network serve processes and installed-wheel lop serve, paired synthetic loopback data from this QA session's existing rig. No rendered UI/native/browser/full-matrix/RSS or suite run.

Independently verified git changed-path list for e7890a59c..f10d7c179 is only scripts/slim_wheel.py and tests/unit/test_slim_wheel.py. Runtime/cache/security/catalogue source bytes are unchanged. Complete before/after-wheel member compatibility above additionally proves this compression change neither drops nor rewrites packaged runtime/resources. Retain prior product/security/feature QA on that proof; reviewer owns the exact slimmer/test review. The coder report supplies hosted-failure/build provenance; its hosted/local baseline distinction is retained, not claimed reproduced here.

Evidence and limits

Own directory: /home/ec2-user/.local-operator/sessions/4af48960f0ed/scratchpad/core-wheel-qa/

  • PREFLIGHT.md, archive_check.py, smoke.py, entry.py, copied rig_matrix.py, ports.txt: plan and exact executable probes.
  • archive-results.json: archive/RECORD/resource details; SHA-256 03b9f83f96b53350a80ca2bd28aa1ba73ed26f55c12c2cc2f01bb676587f909a.
  • results.json: 10/10 HTTP/source/teardown matrix rows PASS, SHA-256 697a2595746a4bcdf8041832d1b493d378799b5e6afc790198c660becdaeb8ca.
  • install.log, archive.log, run.log, provenance.json, logs/{relay-a,relay-b,viewer}.log: installation, daemon import-path proof, source compatibility and cleanup.
  • installed/: isolated local-wheel installation retained for reproducibility. No repo receipt/untracked artifact was added.

Unverified: exact hosted CI wheel/result (this is the supplied local artifact, not the failed hosted build), pristine dependency resolution/other Python/platforms, mobile browser bundle rendering, WAN/native/UI/RSS. Static resources were read as packaged resources, not rendered. Those limits do not invalidate the observed installed-wheel real relay/HTTP path. All own daemon processes are stopped. Existing rig root remains under its earlier final deletion denial; no delete/move retry.

@damianvtran damianvtran added ready-to-merge Run the full-suite merge gate on this PR and removed ready-to-merge Run the full-suite merge gate on this PR labels Nov 13, 2026
@damianvtran

Copy link
Copy Markdown
Owner Author

CI remediation — test formatting

Fixed in f49dcc68bc18daae64429429e72de065471ab1e8 (parent f10d7c17956ead2ae06b1d6bb356b04fee771556). Implementer/model: openai/gpt-6.1-sol (actual inherited model; no fallback).

Verified failure: fast CI run 43339327881 / lint job 131645635486, source log /home/ec2-user/.local-operator/sessions/ce618964c88e/scratchpad/ci/core-lint-final.log, reports exactly two Black formatting failures. Before editing, local black --check --diff on precisely those two files reproduced rc=1 / 2 files would be reformatted, with the same changes as the hosted log.

Exact sites: only collapse the signature of test_eviction_never_follows_a_cache_directory_symlink in tests/unit/network/test_remote_files_hardening.py and the payload-generator expression in tests/unit/test_slim_wheel.py::test_repack_uses_maximum_lossless_compression. Commit changes 2 files, 2 insertions / 5 deletions. No logic, assertion, cap, product source, script, security implementation, catalogues or version change.

Pinned tools / no install: read .github/workflows/ci.yml:379: Black 26.1.0, isort 5.13.2, flake8 7.4.1. Used exactly those versions under the own worktree .venv Python 3.12.15. Black/isort loaded from existing uv cache through explicit PYTHONPATH (Black uncompiled Python implementation); flake8 already in own venv. No dependency install, worktree or clone.

Actual checks after formatting:

  • python -m black --check tests/unit/network/test_remote_files_hardening.py tests/unit/test_slim_wheel.py: PASS, both files left unchanged.
  • python -m isort --check-only on the same exact two files: PASS.
  • python -m flake8 on the same exact two files: PASS.
  • git diff --check: PASS.
  • Isolated HOME/config, inherited CMUX/LOP stripped: python scripts/run_bounded.py --contract=command-status --timeout 300 --memory-mb 3000 -- .venv/bin/python -m pytest -q -n 0 tests/unit/test_slim_wheel.py::test_repack_uses_maximum_lossless_compression tests/unit/network/test_remote_files_hardening.py::test_eviction_never_follows_a_cache_directory_symlink: 3 passed in 0.77s (compression regression plus both symlink parameter cells).

Semantic/source proof: saved both pre-edit parses, then compared the complete ast.dump(ast.parse(source), include_attributes=False) before/after for each file: exact equality PASS. Location attributes necessarily move with formatting; no AST node, value or assertion changes. Independently SHA-256-hashed every tracked file under local_operator/, scripts/, i18n/, plus pyproject.toml, binding filename and per-file digest in sorted git ls-files order: before/after 512e12dcc19b7ab1a900073c108fc320be022d9a9065239ef0229bd2b207caaa. git diff f10d7c179..f49dcc68b -- local_operator scripts i18n pyproject.toml is empty. Git subtree IDs retained:

  • local_operator: 3d048a6ba32641d8c925b54826778b122f233441
  • scripts: 428b59527136f39f53a3245a948586774fe37520
  • i18n: f5d021773e2dd3acdce010ecb76aa95065158337

This is the requested static formatting-only delta. Existing product/security/artifact evidence remains unchanged; no artifact-QA rerun or full suite. No gate arm, workflow cancel, merge or release. Manager owns static delta review and required hosted CI. Delegated session ec7a1e1295e4 scratchpad retains formatting-red.log, formatting-before.json, formatting-proof.json, and formatting-tests.log.

@damianvtran

Copy link
Copy Markdown
Owner Author

Agent review — formatting CI delta

Reviewer: reviewer (lopdev), openai/gpt-6.1-sol, no fallback
Scope: f10d7c1..f49dcc6
Worktree: /home/ec2-user/local-operator/.worktrees/remote-files-parity @ f49dcc68bc18daae64429429e72de065471ab1e8, read-only. PR API verified the pushed head; local HEAD matches and worktree is clean.

Verdict: clean — no BLOCKER or MAJOR. Static review only, as briefed. Existing product/security/packaging reviews and independent QA retained; no extra QA round or test execution needed for these AST-identical formatting changes. Existing MINOR/NIT findings remain deferred.

I opened the implementer's primary evidence, read the entire narrow diff, and independently verified:

  • Exactly two changed paths, both tests: tests/unit/network/test_remote_files_hardening.py and tests/unit/test_slim_wheel.py.
  • The diff only joins the symlink-test function signature and joins the compression-test generator expression. No test assertion, fixture, parameterization, payload, guard, or control-flow change.
  • Parsed complete old/new files from the exact commit Git objects and compared ast.dump(..., include_attributes=False): both complete ASTs identical. Formatting moves locations only; no semantic node/value changes.
  • Independently compared Git tree/blob IDs at both heads; exact equality:
    • local_operator: 3d048a6ba32641d8c925b54826778b122f233441
    • scripts: 428b59527136f39f53a3245a948586774fe37520
    • i18n: f5d021773e2dd3acdce010ecb76aa95065158337
    • pyproject.toml: 27ac8fa0b1b76fc31e65a0937c2257ecb1fc437f

Thus all shipping product/cache/download/security/catalogue/packaging-script bytes and version are unchanged from reviewed f10d7c179; prior artifact and security/QA evidence remains applicable. This is a test-formatting delta review, not a harness-only exemption or a re-pin.

Execution disclosure: I ran only static diff, full-file AST and Git tree/blob identity checks. No tests, formatter, full suite, capture, native rig, server, install, workflow action or repository edit executed in this review. The reported pinned Black/isort/flake8 passes and 3 test passes belong to the implementer's linked evidence, not fresh reviewer execution. Required hosted lint/CI remains the manager's gate. This review is terminal for the exact formatting delta.

@damianvtran damianvtran added ready-to-merge Run the full-suite merge gate on this PR and removed ready-to-merge Run the full-suite merge gate on this PR labels Nov 13, 2026
@damianvtran

Copy link
Copy Markdown
Owner Author

CI remediation — test type narrowing

Fixed in 360315b4776f61f5ff3e7fd330c70e8f7ffb8614, parent f49dcc68bc18daae64429429e72de065471ab1e8. Implementer/model: openai/gpt-6.1-sol, actual inherited model, no fallback.

Verified the four diagnostics in required type-check job 131708826639 / run 43364213305, source log /home/ec2-user/.local-operator/sessions/ce618964c88e/scratchpad/ci/core-pyright-final.log, then reproduced exactly 4 errors, 0 warnings locally before editing:

ID / original location Verified issue Fix
T1 tests/unit/network/test_remote_files_index.py:155:69 page possibly unbound after polling loop Initialize `page: dict[str, Any]
T2 tests/unit/server/test_desktop_sessions.py:1043:23 Union return may be non-iterable PeerAttachmentFile Cold LOCAL-store fixture explicitly asserts tuple response before unchanged byte/MIME assertions.
T3 same file :1129:15 Same union unpack in local user/subagent-origin control Assert LOCAL response is tuple, retaining the exact user bytes and subagent refusal assertions.
T4 same file :3919:23 Same union unpack in contained local-child fixture Assert LOCAL child response is tuple before unchanged byte/MIME and missing-digest assertions.

Read the actual implementations: DesktopSessions.attachment and child_attachment return (decoded_bytes, mime_type) for these local on-disk fixtures, while the separate peer branch returns PeerAttachmentFile(handle, mime_type). No iterable fabricated, no remote streaming handle unpacked, no cast/ignore/broadcast, no product change. Every original ast.Assert expression in both test modules is retained (AST-counter comparison before/after); only explicit additional invariants were added.

Exact scoped type command, before and after: own .venv Python 3.12.15, CI-pinned pyright 1.1.414:

.venv/bin/python scripts/run_bounded.py --contract=command-status --timeout 600 --memory-mb 4000 -- \
  .venv/bin/python -m pyright --pythonpath <worktree>/.venv/bin/python \
  tests/unit/network/test_remote_files_index.py \
  tests/unit/server/test_desktop_sessions.py \
  local_operator/server/utils/desktop_sessions.py

Isolated scratch HOME, inherited CMUX/LOP removed. Before: 4 errors / rc=1. Final after: 0 errors, 0 warnings, 0 informations / rc=0. The included product module defines the actual attachment union and PeerAttachmentFile types; no replacement/stub types used.

Exact affected runtime selectors: isolated HOME/config, serial, run_bounded.py --contract=command-status --timeout 600 --memory-mb 3000 -- .venv/bin/python -m pytest -q -n 0:

  • tests/unit/network/test_remote_files_index.py (real loopback relay + ASGI index/read/download/refusal paths)
  • tests/unit/server/test_desktop_sessions.py::test_durable_attachment_is_readable_without_starting_an_owner
  • ...::test_attachment_refuses_a_session_id_that_is_not_the_session_shape
  • ...::test_attachment_refuses_a_subagent_origin_session
  • ...::test_child_attachment_serves_bytes_for_a_contained_child
  • ...::test_child_attachment_refuses_the_same_uncontained_pairs

Final result 24 passed, 5 existing deprecation warnings in 5.90s. The unrelated test_real_mp3_waveform_http_writes_and_serves_sidecar fixture/teardown cases were not selected; no claim of waveform coverage. Additionally exercised _page directly with controlled HTTP responses: building→ready returns exactly the second page; 40 building responses still raise the original diagnostic containing the final page (attempt: 40). PASS.

Exact-file lint: Black 26.1.0 --check, isort 5.13.2 --check-only, flake8 7.4.1, and git diff --check: PASS on only the two changed test files. Black/isort reused existing cached tools through explicit PYTHONPATH; no dependencies installed.

Product byte proof: before/after SHA-256 binding each tracked filename and its content digest under local_operator/, scripts/, i18n/, plus pyproject.toml is identical: 512e12dcc19b7ab1a900073c108fc320be022d9a9065239ef0229bd2b207caaa. git diff f49dcc68b..360315b47 -- local_operator scripts i18n pyproject.toml is empty. Commit touches exactly the two test modules (14 insertions / 5 deletions), retaining all shipping/security/catalogue/script bytes, caps and original assertions.

No upstream-main chase, merge, gate rearm/cancellation, artifact rebuild or full matrix. Source prerequisite #3892 remains manager-owned; manager will perform the single convergence/gate after it lands. Delegated session ec7a1e1295e4 scratchpad retains type-narrowing-before.log, type-narrowing-after.log, type-narrowing-tests-final.log, and before/after proof JSON.

@damianvtran

Copy link
Copy Markdown
Owner Author

Agent review — test type CI delta

Reviewer: reviewer (lopdev), openai/gpt-6.1-sol, no fallback
Scope: f49dcc6..360315b
Worktree: /home/ec2-user/local-operator/.worktrees/remote-files-parity @ 360315b4776f61f5ff3e7fd330c70e8f7ffb8614, read-only. PR API verified the pushed head; local HEAD matches and worktree is clean. Request/acceptance/non-goals unchanged; static scope is T1–T4 only.

Verdict: clean — no BLOCKER or MAJOR. No weakened assertions or new findings. All prior shipping product/security/artifact evidence retained; existing MINOR/NIT findings remain deferred.

Read the exact two-file delta and implementer's primary evidence. Independently verified:

  • T1, _page: the initial None sentinel prevents an unbound diagnostic value; the added dict assertion validates the decoded page before indexing. The existing 40-request loop, HTTP-200 assertion, state != 'building' return predicate and terminal failure diagnostic remain intact. Exhaustion cannot return a building/default page: after the final non-ready page, it still raises AssertionError. The final non-None assertion proves a request assigned the diagnostic page; there is no sentinel success or swallowed failure.
  • T2–T4, attachment narrowing: read each fixture around the changed calls. The cold user-store fixture explicitly creates its local session directory, the subagent-origin control invokes the local user session, and the contained-child fixture creates/names a local parent/child. Their existing implementation contract is the decoded (bytes, mime) tuple, not the separate remote PeerAttachmentFile streaming branch. assert isinstance(attachment, tuple) is a real runtime invariant before unpacking, not a cast/ignore or an accommodation that weakens the peer-streaming contract. Original bytes/MIME, no-owner-bridge, subagent-refusal and missing-digest checks remain.
  • Parsed both complete old/new test modules from exact commit Git objects and counted complete ast.Assert nodes (including tests/messages): every original assertion retained. Added exactly two assertion nodes in test_remote_files_index.py and three in test_desktop_sessions.py.
  • Changed-path list is exactly those two test modules. Independently compared Git tree/blob identities at both heads; shipping sources are byte-identical:
    • local_operator: 3d048a6ba32641d8c925b54826778b122f233441
    • scripts: 428b59527136f39f53a3245a948586774fe37520
    • i18n: f5d021773e2dd3acdce010ecb76aa95065158337
    • pyproject.toml: 27ac8fa0b1b76fc31e65a0937c2257ecb1fc437f

Execution disclosure: reviewer performed static diff/fixture inspection, full-file assertion-AST retention and Git tree/blob checks only. No additional pytest/pyright/lint run, QA/native rig, capture, dependency install, repository write or full-suite execution. The reported scoped pyright 4→0, 24 test passes and exhaustion negative control are the implementer's linked evidence, not newly executed reviewer results.

This delta review is terminal. It does not arm a gate or authorize an upstream merge; manager retains the stated hold for prerequisite #3892 and the single subsequently authorized convergence/gate.

@damianvtran

Copy link
Copy Markdown
Owner Author

Convergence after reproducible parity prerequisite

One authorized ordinary merge pushed: 360315b4776f61f5ff3e7fd330c70e8f7ffb8614 → 7fb7cfc2d122cbb21096081b67a401636914bbcf. First parent is the reviewed/test-type-repaired branch; second parent is the exact freshly fetched origin/main 4d113fc6c23b3fbb2afa426c0c6747ee0c399467 (#3892). Old merge base: 0661031bcbd256037d9d5694f5f845932fbd3178. No rebase/force push, extra fetch/convergence, generated-fixture refresh or sourceHash rewrite. Actual implementer/model: openai/gpt-6.1-sol, inherited, no fallback.

Conflict and manager allocation. Ordinary merge auto-merged both Python overlaps and conflicted only in local_operator/mobile/web/src/lib/parity-sources.json. Upstream's immutable UI revision 5acfe0a2d92365b7c9b029a9d870326d9b022d25 authenticates main's different source identity, not this branch's already-reviewed fixture. The actual hosted checker on isolated minimal-union metadata correctly FAILED condense: revision fixture SHA mismatch. Positive upstream control authenticated that revision and all 36 sources, proving this was an identity mismatch, not a network problem.

Manager explicitly authorized selecting already-published UI revision 72043c4ecbfddd734abec28694b05a163786bf0c, the existing b0edf02e5/#1877 generation documented by relay e7890a59c, PR proof 6837647988. The exact checker authenticated the branch's complete unchanged fixture, manifest and all 36 actual model-source files at that revision before the conflict was resolved. This adopts #3892's existing revision/revisionNote contract; it is not a new schema, latest-main chase or repin of fixture/hash bytes.

Reviewed branch metadata Before Converged
sha256 5e0117590743d9e79682bf044c4c07eb8c4053a5ffc9a17765d28545b43fea73 identical
sourceHash 5dde7aa3e8d7f53cf61d2386a7b3fd0802577bf0e1a5a310b06b9804a520cebb identical
revision (contract introduced by upstream) absent 72043c4ecbfddd734abec28694b05a163786bf0c
revisionNote absent names the authenticated b0edf02e5/#1877 generation and e7890a5 proof

Removing only the two adopted provenance fields yields JSON exactly equal to the reviewed branch metadata; every pre-existing JSON field and all fixture bytes are retained.

Source/overlap proof. Upstream moved 64 files; branch patch spans 98. Exact overlap:

All 95 non-overlap feature files are byte-identical to 360315b47. All 97 non-metadata feature files retain exact ordered +/- patch lines, including both auto-merged Python overlaps when compared against the new base. git range-diff pairs all 19 commits as =. The only branch-patch +/- difference is four metadata lines reflecting revision/revisionNote values under the new contract. No remote-files product/manual security rewrite. Direct diff from first parent confirms unchanged condensation fixture, scripts/slim_wheel.py, local_operator/remote_files/ and both prior type-narrowing test modules. The other merged bytes are ordinary upstream changes; version change is inherited from main, not a PR-authored bump.

Actual validation. Isolated HOME, inherited CMUX/LOP stripped; bounded commands (no installs):

  • Exact repo/CI command node local_operator/mobile/web/scripts/check-condense-parity-pinned.mjs: PASS, actual output: check-condense-parity: immutable damianvtran/local-operator-ui@72043c4ecbfddd734abec28694b05a163786bf0c; fixture, manifest and 36 model sources match.
  • node --test local_operator/mobile/web/scripts/check-condense-parity-pinned.node-check.mjs with the workflow's explicit LOCAL_OPERATOR_SCRATCHPAD: 15 passed, 0 failed. Includes moving-main PASS→FAIL reproduction versus stable immutable PASS, and fail-closed controls for fixture SHA, manifest, source bytes/inventory, missing/mutable/unavailable revision and missing resources. Initial isolated control invocation omitted required scratchpad env and failed setup; rerun corrected that environment, not source.
  • CI-pinned pyright 1.1.414, own .venv, explicit pythonpath, exact files local_operator/network/relay.py, local_operator/server/utils/desktop_sessions.py, tests/unit/network/test_remote_files_index.py, tests/unit/server/test_desktop_sessions.py: 0 errors, 0 warnings, 0 informations, under run_bounded.py --contract=command-status --timeout 600 --memory-mb 4000.
  • Black 26.1.0 --check, isort 5.13.2 --check-only, flake8 7.4.1 on only the two auto-merged Python overlaps: PASS. Cached tools, no install. git diff --cached --check: PASS.

Hosted before discriminator: independently read completed failed run 43364213244 / build job 131684668141. At 2026-11-13T04:48:08Z, the moving-main gate failed because this branch's fixture 5e0117590743… differed from desktop main bc99409837ec…, and sourceHash 5dde7aa3e8d7… differed from moving-main 0055f66561d0…. The after proof above authenticates the immutable reviewed generation without rewriting those branch hashes.

Disposition: CONVERGENCE REVIEW NEEDED, not mechanical re-pin. Independent static review scope is 360315b47..7fb7cfc2d, focused on the three overlap files. Existing 24-test/type-narrowing evidence is retained; no full matrix, native/artifact/QA rerun, workflow gate arm/cancel, PR merge or release. Manager owns the single subsequent gate rearm. Remote head verified; GitHub reports MERGEABLE.

Evidence retained in delegated session ec7a1e1295e4 scratchpad: convergence-before.json, convergence-source-proof.json, convergence-rebase-proof.md, convergence-strict-checks-final.log, convergence-pyright.log, prior hosted failure log and mismatching/upstream/selected-revision control logs.

Rebase proof — 360315b..7fb7cfc

  • Base: 4d113fc6c23b3fbb2afa426c0c6747ee0c399467; merge-bases 0661031bcb → 4d113fc6c2
  • git range-diff: 19 commit pair(s), =×19 → all =
  • PR patch (+/- and header lines): CHANGED (9638 lines)
  • Upstream files moved between merge-bases: 64; PR files: 98; overlap: local_operator/mobile/web/src/lib/parity-sources.json, local_operator/network/relay.py, local_operator/server/utils/desktop_sessions.py
  • Verdict: CONVERGENCE REVIEW NEEDED — files: local_operator/mobile/web/src/lib/parity-sources.json, local_operator/network/relay.py, local_operator/server/utils/desktop_sessions.py; patch lines differ (4 lines)

Re-run: python scripts/rebase_proof.py 360315b4776f61f5ff3e7fd330c70e8f7ffb8614 7fb7cfc2d122cbb21096081b67a401636914bbcf --base 4d113fc6c23b3fbb2afa426c0c6747ee0c399467

@damianvtran

Copy link
Copy Markdown
Owner Author

Agent review — convergence after #3892

Reviewer: reviewer (lopdev), openai/gpt-6.1-sol, no fallback
Scope: 360315b..7fb7cfc
Worktree: /home/ec2-user/local-operator/.worktrees/remote-files-parity @ 7fb7cfc2d122cbb21096081b67a401636914bbcf, read-only. Pushed head independently verified; final local HEAD matches and worktree is clean. Request remains peer-asset parity under existing S1 authority; this is the single authorized source-prerequisite convergence, not a new feature round.

Verdict: clean — no BLOCKER or MAJOR. Existing MINOR/NIT ledger frozen. This is convergence review, not a mechanical re-pin.

Opened the primary proof, inspected the three overlap diffs, and independently derived:

  • 19/19 range-diff pairs =, comparing 0661031bc..360315b47 against 4d113fc6c..7fb7cfc2d.
  • 98 feature paths; exactly 3 upstream overlaps; all 95 non-overlap feature files byte-identical to the prior reviewed head.
  • All 97 non-metadata feature files retain identical ordered +/- lines and file headers relative to the new base. My extraction selects diff --git and +/- lines from --no-renames --unified=0 and compares 9,595 lines, excluding only parity-sources.json.
  • Python auto-merge separation, verified by complete function/method ASTs:
    • network/relay.py: the feature-modified function remains identical; the 3 functions changed by convergence are exactly upstream's _ctl_member_rm, installed_unit_oom_policy, and render_systemd. Member-removal bookkeeping and systemd OOM policy do not alter remote-file capability registration or byte/index dispatch. Existing feature whitelist/constants remain in the identical PR patch.
    • server/utils/desktop_sessions.py: all 32 feature-modified functions/methods remain AST-identical to the prior head. The 7 changed/added functions are exactly upstream's snapshot/history proof-join deadline and tracing implementation. These operate on display-only peer-cycle proof, with explicit unavailable fallback; they do not change attachment, waveform, child/supplement, indexed-owner, cache, pull, or serving contracts. No mixed/remediated function differs from upstream or the approved feature implementation.
  • Sole conflict is provenance metadata: removing only new revision/revisionNote gives the exact prior metadata object; fixture bytes, SHA and sourceHash are unchanged. The fields use the existing fix(ci): make condense parity validation reproducible #3892 schema, not a new schema or generated refresh.
  • Independently read exact UI Git objects at 72043c4ecbfddd734abec28694b05a163786bf0c: fixture bytes equal the retained core fixture; manifest SHA equals computed fixture SHA and core pin; recomputed sourceHash from all 36 committed model-source inputs equals embedded/manifest/core sourceHash. The selected revision authenticates the previously reviewed generation, not a moving-main identity.
  • Source-prerequisite required workflow, freshness workflow, pinned checker and latest checker are byte-identical to merged upstream 4d113fc6c. No locally weakened checker/workflow or new source-policy variation.

Retention limit: prior feature, security, 24-test typing and packaging evidence is retained for the proven unchanged feature implementations. This does not claim the entire shipping tree/new wheel is byte-identical: ordinary upstream changes and its version are merged. The final exact-head hosted gate covers that integrated tree. This review introduces no PR-authored version bump or new artifact claim.

Execution disclosure: static source/AST/patch/Git-object hashing checks only. No pytest/QA/native/capture/full-matrix/rebuild/installation or server process launched. The actual strict-checker PASS, 15 negative controls and pyright/lint results are sourced to the implementer's linked proof, not claimed as rerun here. No repository writes or new receipt files. Review terminal; manager owns the one subsequent exact-head gate.

@damianvtran damianvtran added ready-to-merge Run the full-suite merge gate on this PR and removed ready-to-merge Run the full-suite merge gate on this PR labels Nov 13, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ready-to-merge Run the full-suite merge gate on this PR

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant