Repository navigation
test: no port or environment races in the test harness - #377
Merged
Merged
Conversation
- free_tcp_port hands out ports from 20000-32767, below Linux's ephemeral range (where every outgoing connection also takes a port), through one pid-seeded counter per process with a bind check; the spawn retry loop stays as the backstop. Under a loaded machine (load ~175) the old probe-release-rebind on 127.0.0.1:0 failed a random spawned-binary test with AddrInUse after its 5 retries. - embedded_denylist is its own test binary: its set_var of TRITON_DENIED_PRINCIPALS / TRITON_PAIRING_TOOLS / TRITON_AUDIT_REJECT_WINDOW_SECS leaked into every triton the shared 'it' binary spawned meanwhile (random mcp_era / twilio failures). Its tests share a tokio mutex so their env changes do not overlap. Stress: the full 'it' binary 3x green; two copies at once show no AddrInUse and no denylist leak (only the CPU-bound rasterizer 20-renders-in-10s check slips under doubled load).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
Under load, the pre-push gate (
scripts/check.sh) failed 3 of 4 times, each time on a different test unrelated to the change being pushed (embedded_denylist,mcp_era,twilio_whatsapp_template). That tempts--no-verify, which CLAUDE.md forbids. There were two causes.What
free_tcp_portused to bind127.0.0.1:0, release the port, and let the spawned triton bind it later. Under load, another socket took the port in between, often an outgoing connection, since the kernel gives those ports from the same range. Ports now come from 20000–32767, below Linux's ephemeral range (32768–60999), through one pid-seeded counter per process with a bind check. The spawn retry loop stays as the backstop. A new unit test checks that 300 ports all come from the range with no repeats; it failed before this change.embedded_denylistmust prove that an embedded host readsTRITON_DENIED_PRINCIPALSwith no wiring, so it callsset_var. Every triton that the shareditbinary spawned meanwhile inherited a revoked principal or a pairing restriction. The test is now its own[[test]]binary, and its tests share a tokio mutex so their env changes don't overlap.doc/realizations.md§7: the port-race entry is updated, and there is a new entry on env-mutating tests.Evidence
embedded_denylist: 7/7, three runs in a row. Clippy is clean.it: 757/757, three runs in a row.itat once: noAddrInUseand no denylist leak. The only failure israsterizer::concurrent_requests_dont_deadlock, a CPU-bound check that 20 renders finish within 10 s. It slips under doubled load and is out of scope here.scripts/check.sh: 9/9 steps at default test parallelism, where it used to needRUST_TEST_THREADS=4.Open in [code]smith · Need help on this PR? Tag
@codesmith-botwith what you need.