Skip to content

fix(docker): reclaim sandbox token files on out-of-band removal - #3220

Merged
johntmyers merged 1 commit into
NVIDIA:mainfrom
letv1nnn:3041-token-file-not-cleaned-up/letv1nnn
Sep 10, 2026
Merged

fix(docker): reclaim sandbox token files on out-of-band removal#3220
johntmyers merged 1 commit into
NVIDIA:mainfrom
letv1nnn:3041-token-file-not-cleaned-up/letv1nnn

Conversation

@letv1nnn

@letv1nnn letv1nnn commented Sep 8, 2026

Copy link
Copy Markdown
Contributor

Summary

delete_sandbox_inner skipped per-sandbox token file cleanup on one exit path — Docker reports no container and no in-memory pending record survives — leaving the sandbox's gateway JWT on disk indefinitely. Every other exit path in that function already cleans up. This adds the missing call plus regression coverage.

Related Issue

Fixes #3041

Changes

  • crates/openshell-driver-docker/src/lib.rs: call cleanup_sandbox_token_file_for_delete in the "container gone, no pending record" branch of delete_sandbox_inner.
    • Uses _for_delete rather than _by_id because delete accepts a name with no id (require_sandbox_identifier requires only one of the two). With an empty id, sandbox_token_path resolves to <namespace>/sandbox.jwt, whose parent is the shared namespace directory. _for_delete guards on !sandbox_id.is_empty().
    • Keeps the Ok(false) return: nothing was removed from Docker, so no deletion is claimed and no Deleted watch event fires.
  • crates/openshell-driver-docker/src/tests.rs: two regression tests, plus a loopback stub that answers Docker's GET /containers/json with [] so the branch is reachable without a daemon.
  • crates/openshell-driver-docker/Cargo.toml / Cargo.lock: enable temp-env's async_closure feature. The sync with_vars cannot wrap an async test body, and the tests need XDG_STATE_HOME scoped to a tempdir.

Left alone deliberately: the adjacent summary_container_targetNone branch returns Ok(pending.is_some()) without removing the container. That one is a live container being silently abandoned, not a token leak — adding cleanup there would revoke a running sandbox's credential. Worth its own issue; stop_sandbox_inner handles the identical case with Err(Status::not_found).

Testing

  • mise run pre-commit passes
  • Unit tests added/updated
  • E2E tests added/updated (if applicable)

Checklist

  • Follows Conventional Commits
  • Commits are signed off (DCO)
  • Architecture docs updated (if applicable)

@copy-pr-bot

copy-pr-bot Bot commented Sep 8, 2026

Copy link
Copy Markdown

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

@letv1nnn
letv1nnn marked this pull request as ready for review September 8, 2026 16:51
@johntmyers johntmyers added gator:in-review Gator is reviewing or awaiting PR review feedback test:e2e Requires end-to-end coverage labels Sep 9, 2026
@github-actions

github-actions Bot commented Sep 9, 2026

Copy link
Copy Markdown

Label test:e2e applied, but pull-request/3220 does not exist yet. A maintainer needs to comment /ok to test b0118ac2927bd0dc459a417318febb41e1dad8fa to mirror this PR. Once the mirror exists, re-apply the label or re-run Branch E2E Checks from the Actions tab.

@johntmyers

Copy link
Copy Markdown
Collaborator

/ok to test b0118ac

@johntmyers johntmyers added gator:blocked Gator is blocked by process or repository gates and removed gator:in-review Gator is reviewing or awaiting PR review feedback labels Sep 9, 2026
@johntmyers

johntmyers commented Sep 9, 2026

Copy link
Copy Markdown
Collaborator

gator-agent

PR Review Status

This focused Docker lifecycle fix is project-valid through linked bug #3041. The initial independent code review found no blocking issues, and no Fern docs update is needed because the change restores internal token-file cleanup without changing a published user interface.

Action required: A maintainer must dispatch Trivy Changes for base 2ad86c1b2e3e81a26e0fd38ec860398acb31f2fc and head b0118ac2927bd0dc459a417318febb41e1dad8fa. This required workflow was added to main after the PR head diverged, so no current-head Trivy run or required status exists.

Blocking findings:

  • No blocking code findings remain

Carried findings:

  • None

Non-blocking suggestions:

  • None
Gator metadata
  • Validation: Linked bug bug(driver-docker): token file not cleaned up when container is already gone with no pending record #3041 provides a concrete Docker-driver reproduction and acceptance criteria, and this four-file patch is narrowly scoped to that defect and regression coverage.
  • Docs: Not needed; this is an internal cleanup correctness fix with no direct CLI, API, configuration, policy, or workflow UX change.
  • Checks: DCO and vouch are green. The current-head mirror exists; Branch Checks and E2E are running, Helm Lint is green after change detection, and a current-head Trivy Changes run must be manually dispatched.
  • E2E: test:e2e applied; /ok to test b0118ac2927bd0dc459a417318febb41e1dad8fa posted; mirror head verified; Branch E2E Checks run 34397594779 queued.
  • Head SHA: b0118ac2927bd0dc459a417318febb41e1dad8fa
  • Base SHA: 2ad86c1b2e3e81a26e0fd38ec860398acb31f2fc
  • Merge base SHA: 320d4ef79dd572c642133f175f12bafc20d89fd9
  • Patch ID: 75d5c6386c9d5657a4292aa914694a9de61e0d9a
  • Gator payload: 8
  • Review mode: initial
  • Previous reviewed SHA: none
  • Review budget exhausted: no
  • Maintainer decision required: no
  • Next state: gator:blocked
  • Blocked reason: trivy_workflow_dispatch_required

@johntmyers johntmyers added gator:watch-pipeline Gator is monitoring PR CI/CD status gator:blocked Gator is blocked by process or repository gates and removed gator:blocked Gator is blocked by process or repository gates gator:watch-pipeline Gator is monitoring PR CI/CD status labels Sep 9, 2026
Signed-off-by: Artem Lytvyn <alytvyn@redhat.com>
@johntmyers
johntmyers force-pushed the 3041-token-file-not-cleaned-up/letv1nnn branch from b0118ac to 097e5ba Compare September 10, 2026 16:52
@johntmyers

Copy link
Copy Markdown
Collaborator

/ok to test 097e5ba

@johntmyers johntmyers added test:e2e Requires end-to-end coverage and removed test:e2e Requires end-to-end coverage labels Sep 10, 2026
@github-actions

Copy link
Copy Markdown

Label test:e2e applied for 097e5ba. Open the existing run and click Re-run all jobs to execute with the label set. The run will execute the standard E2E suite after building the required gateway and supervisor images once. The matching required CI gate status on this PR will flip green automatically once the run finishes.

@johntmyers

Copy link
Copy Markdown
Collaborator

gator-agent

PR Review Status

The follow-up review of the rebased patch found no blocking issues. The author-only delta only relocates the existing regression tests relative to upstream additions; the token-file cleanup fix and its coverage remain intact.

Action required: Wait for current-head Branch E2E Checks run 34505726235 to finish, then use Re-run all jobs as requested by the E2E Label Help bot; the run is active and GitHub currently rejects the rerun. A maintainer must also approve and run current-head Trivy Changes run 34504799774.

Blocking findings:

  • No blocking code findings remain

Carried findings:

  • None
Gator metadata
  • Validation: Linked bug bug(driver-docker): token file not cleaned up when container is already gone with no pending record #3041 provides a concrete Docker-driver reproduction and acceptance criteria, and this four-file patch remains narrowly scoped to that defect and regression coverage.
  • Docs: Not needed; this is internal cleanup correctness behavior with no direct CLI, API, configuration, policy, or workflow UX change.
  • Checks: DCO is green, Branch Checks run 34505725330 is queued, Helm Lint is green, and Trivy Changes run 34504799774 is action-required.
  • E2E: test:e2e re-applied after /ok to test 097e5ba2a568395769acafd2f0a1f53eca496333; mirror head verified; run 34505726235 is executing, but the bot-required rerun attempt cannot be queued until the active run finishes.
  • Head SHA: 097e5ba2a568395769acafd2f0a1f53eca496333
  • Base SHA: 0357daee316f32a4d5c312174d68672cb0f4d389
  • Merge base SHA: c6c857342953c2f70a14d1ca1cef0d7b2db1ed58
  • Patch ID: 69f954d1cc14a01ece2c80f821614a45264dcb0e
  • Gator payload: 8
  • Review mode: follow_up
  • Previous reviewed SHA: b0118ac2927bd0dc459a417318febb41e1dad8fa
  • Review budget exhausted: no
  • Maintainer decision required: no
  • Next state: gator:blocked
  • Blocked reason: test_dispatch_required

@johntmyers johntmyers added gator:approval-needed Gator completed review; maintainer approval needed and removed gator:blocked Gator is blocked by process or repository gates labels Sep 10, 2026
@johntmyers
johntmyers added this pull request to the merge queue Sep 10, 2026
@johntmyers johntmyers added gator:merge-ready and removed gator:approval-needed Gator completed review; maintainer approval needed labels Sep 10, 2026
Merged via the queue into NVIDIA:main with commit 25021ee Sep 10, 2026
180 of 185 checks passed
@johntmyers

Copy link
Copy Markdown
Collaborator

gator-agent

Monitoring Complete

Monitoring is complete because this PR has merged.

Final status: The focused Docker token-file cleanup fix was reviewed with no blocking findings, the required Branch Checks, Helm Lint, Trivy Changes, and E2E gates completed successfully, and maintainer approval was present before merge.

I removed the active gator:* label because there is nothing left for gator to monitor on this PR.

Gator metadata
  • Head SHA: 097e5ba2a568395769acafd2f0a1f53eca496333
  • Gator payload: 8
  • Final state: merged

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

test:e2e Requires end-to-end coverage

Projects

None yet

Development

Successfully merging this pull request may close these issues.

bug(driver-docker): token file not cleaned up when container is already gone with no pending record

2 participants