refactor(policy)!: remove NetworkBinary harness field

[varshaprasad96]

Summary

Remove the deprecated NetworkBinary.harness field before 0.1.0 while preserving advisor-origin provenance at the endpoint/rule boundary.

Related Issue

Closes #3054

Changes

• Reserve protobuf field number 2 and name harness, update generated Go bindings, and remove runtime, SDK, and profile uses.
• Reject legacy YAML/profile harness input with migration guidance while retaining protobuf wire compatibility.
• Move advisor provenance enforcement to gateway-stamped endpoints and keep advisor binaries separated from explicit rules, including rule-name collisions.
• Migrate legacy stored policy-history provenance without dropping unknown nested wire fields.
• Update policy architecture, provider profile docs, and 0.1.0 migration notes.

Testing

• mise run pre-commit
• cargo test --workspace --exclude openshell-server -q
• cargo test -p openshell-server --lib -q
• cargo test -p openshell-supervisor-network --lib -q
• cargo test -p openshell-sandbox --lib -q
• mise run go:ci
• mise run sdk:ts:ci
• mise run test:python
• Focused policy/profile/protobuf migration and provenance regressions.
• mise run e2e:docker — blocked before execution because the local Docker daemon was unreachable.

Checklist

• Implementation is scoped to refactor(policy)!: remove deprecated NetworkBinary.harness #3054.
• Tests and pre-commit checks pass.
• Documentation and migration guidance are updated.
• Commit is signed off for DCO.
• Docker-backed E2E completed (local Docker daemon unavailable).

[copy-pr-bot]

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

[gmenher]

@varshaprasad96 I did a manual validation and it works as explained, scalar provider profile lint/import/export/delete works against a local Docker gateway; legacy harness profiles are rejected with the intended migration diagnostic; sandbox creation works.
I also ran the Docker E2E suiteand the other tests. The ambiguous-policy behavior was correct: the update was rejected and the active policy was preserved. One existing assertion outside this PR's diff was sensitive to diagnostic line wrapping, so I treated that as an unrelated test-harness issue during testing.
Only one non blocking question, would it make sense to add a regression test to confirm that migrating legacy payloads preserves unrelated unknown nested protobuf fields? I think it could help protect the forward-compatibility path.

[github-actions]

Label test:e2e applied, but pull-request/3222 does not exist yet. A maintainer needs to comment /ok to test 28495d8803b4380252435ea0d72fed565bbb5611 to mirror this PR. Once the mirror exists, re-apply the label or re-run Branch E2E Checks from the Actions tab.

[johntmyers]

/ok to test 28495d8

[johntmyers]

gator-agent

PR Review Status

This implementation matches the linked removal and migration scope, preserves advisor-origin provenance at the endpoint boundary, and has no blocking findings from the initial code review. Required Branch Checks and E2E workflows are running for the current head, so Gator will monitor them before requesting maintainer approval.

Blocking findings:

• No blocking findings remain

Carried findings:

• None

Non-blocking suggestions:

• None

Gator metadata

• Validation: Project-valid implementation of linked issue #3054
• Docs: Architecture, provider profile, and 0.1.0 migration documentation updated
• Checks: Current-head Branch Checks and E2E are in progress; Helm Lint is green
• E2E: test:e2e applied, mirror authorization posted, and Branch E2E Checks running
• Head SHA: 28495d8803b4380252435ea0d72fed565bbb5611
• Base SHA: e1084e19db622fd1134454d3b589aab2022aaddf
• Merge base SHA: 2ad86c1b2e3e81a26e0fd38ec860398acb31f2fc
• Patch ID: 7e8e9eb0678cc383090be852ce40f199d97608ca
• Gator payload: 8
• Review mode: initial
• Previous reviewed SHA: none
• Review budget exhausted: no
• Maintainer decision required: no
• Next state: gator:watch-pipeline

[varshaprasad96]

Thanks @gmenher, have added in 758f64309.

The stored-policy compatibility regression adds an unrelated future field (tag 99) alongside the former harness tag, round-trips the payload through policy_record_from_parts → policy_payload_from_record → policy_record_from_parts, and verifies that both fields survive before the legacy provenance migration runs. It then confirms the migrated policy still splits explicit and advisor binaries correctly.

Validation steps:

• cargo test -p openshell-server --lib stored_policy_decode_migrates_legacy_provenance_and_preserves_unknown_fields
• mise run pre-commit

[johntmyers]

/ok to test 758f643

[johntmyers]

gator-agent

Re-check After Author Update

Thanks @varshaprasad96. I checked the regression test added in 758f64309 after your September 9 response: it exercises an unrelated future nested protobuf field through the stored-policy record/payload round trip and then confirms the legacy provenance migration still separates explicit and advisor binaries correctly. The focused follow-up review found no blocking issues. Required current-head Branch E2E Checks are now queued and running.

Disposition: resolved.

Remaining items:

• No blocking items remain

Gator metadata

• Validation: Project-valid implementation of linked issue #3054
• Docs: Architecture, provider profile, and 0.1.0 migration documentation were updated in the reviewed patch; this follow-up is test-only
• Checks: Current-head required Branch Checks, Helm Lint, and E2E status contexts are pending
• E2E: test:e2e is applied; current-head mirror authorization was posted and Branch E2E Checks run 34410134126 is queued/running
• Head SHA: 758f64309b497cbbc06779dc695d73b6b0cae28f
• Base SHA: e1084e19db622fd1134454d3b589aab2022aaddf
• Merge base SHA: 2ad86c1b2e3e81a26e0fd38ec860398acb31f2fc
• Patch ID: b5d4531b4160b71369e101b89a00d980584b4346
• Gator payload: 8
• Review mode: follow_up
• Previous reviewed SHA: 28495d8803b4380252435ea0d72fed565bbb5611
• Review budget exhausted: no
• Maintainer decision required: no
• Next state: gator:watch-pipeline

[johntmyers]

@varshaprasad96 looks like some tests need to be fixed, if you could take a look then we should be able to prep to merge

[johntmyers]

/ok to test d50f18a

[johntmyers]

gator-agent

Re-check After Maintainer Update

Thanks @johntmyers. I checked the test and provenance concern you raised on September 9 against the author-only 758f64309..d50f18a7 delta. The update now treats clearing advisor provenance as an authorization change, rejects an undeclared existing binary inheriting exact-host private-address trust, keeps distinct-rule fallback separated, and adds focused regressions for both paths. The independent follow-up review found no blocking issues, and required current-head workflows are queued or running.

Disposition: resolved.

Remaining items:

• No blocking items remain

Gator metadata

• Validation: Project-valid implementation of linked issue #3054
• Docs: Architecture, provider profile, and 0.1.0 migration documentation were updated in the reviewed patch; this follow-up changes policy merge behavior and focused tests without adding a new user-facing surface
• Checks: Current-head Branch Checks and Helm Lint are queued/running
• E2E: test:e2e is applied; current-head mirror authorization was posted and Branch E2E Checks run 34430279538 is queued
• Head SHA: d50f18a79d26cbcf4c8b9b51b1bb0c424455d2b2
• Base SHA: e1084e19db622fd1134454d3b589aab2022aaddf
• Merge base SHA: 2ad86c1b2e3e81a26e0fd38ec860398acb31f2fc
• Patch ID: a08ffb08aae0d60185fc0e618e1866178dc8959e
• Gator payload: 8
• Review mode: follow_up
• Previous reviewed SHA: 758f64309b497cbbc06779dc695d73b6b0cae28f
• Review budget exhausted: no
• Maintainer decision required: no
• Next state: gator:watch-pipeline

[johntmyers]

/ok to test fd31886

[github-actions]

Label test:e2e applied for fd31886. Open the existing run and click Re-run all jobs to execute with the label set. The run will execute the standard E2E suite after building the required gateway and supervisor images once. The matching required CI gate status on this PR will flip green automatically once the run finishes.

[johntmyers]

gator-agent

Re-check After Maintainer Update

Thanks @johntmyers. I verified your current-head mirror authorization and the requested Branch E2E rerun; attempt 2 completed successfully. I also reviewed the author-only 80f9dac2..fd31886d patch delta after the rebase: the compatibility test moved cleanly with its module context, schema fingerprints were refreshed for the new base, and a redundant NetworkBinary default initializer was removed. The independent follow-up review found no blocking issues. Branch Checks are still running, so Gator will continue monitoring this head.

Disposition: resolved.

Remaining items:

• No blocking review items remain

Gator metadata

• Validation: Project-valid implementation of linked issue #3054
• Docs: Architecture and migration documentation remain updated; this follow-up adds no new direct user-facing UX surface
• Checks: Current-head Helm Lint, Trivy Changes, and E2E are green; Branch Checks remain in progress
• E2E: test:e2e is applied; Branch E2E Checks run 34621923853 attempt 2 succeeded
• Head SHA: fd31886d4b130d7c823513438528e16f19131eba
• Base SHA: d99f12a33be61a89de9d3a29a10a9c732f6cd47e
• Merge base SHA: d99f12a33be61a89de9d3a29a10a9c732f6cd47e
• Patch ID: fe14b5440c439743ff93e828689d71c27760f43c
• Gator payload: 8
• Review mode: follow_up
• Previous reviewed SHA: 80f9dac2f709da9dd38b837d501b2756be461dd4
• Review budget exhausted: no
• Maintainer decision required: no
• Next state: gator:watch-pipeline

[johntmyers]

gator-agent

Monitoring Complete

Monitoring is complete because this PR has merged.

Final status: the current head passed Branch Checks, Helm Lint, and E2E; the latest independent review found no blocking issues, and maintainer approval was present before merge.

I removed the active gator:* label because there is nothing left for gator to monitor on this PR.

Gator metadata

• Head SHA: fd31886d4b130d7c823513438528e16f19131eba
• Gator payload: 8
• Final state: merged