feat: run a submission in a listed prebuilt simulator image

[jcschaff]

Why

Every submission runs in one shared container whose package list is fixed in this repo (the simulator registry). A process-bigraph workspace with conda-forge dependencies or a compiled extension can't be expressed that way.

The motivating case is vivarium-collective/viva-pde-particle: dolfinx, netgen, and a Smoldyn module built from source. Its owners can publish an OCI image instead, which the deployment lists by name.

What

• Setting: prebuilt_simulators: dict[str, str], from PREBUILT_SIMULATORS as JSON, mapping a simulator name to an image reference.
  • Default empty: behaviour is unchanged until the deployment lists something.
  • Pin images by digest or an immutable tag, because the cached SIF is keyed on the reference.
• POST /simulation/run?simulator=<name>:
  • The name is checked before anything else. An unlisted name is a 400 naming the listed ones, so a request can never pull an arbitrary image.
  • Omitted: the shared container, as today.
• Definition: simulation/prebuilt.py produces an Apptainer definition that is just

  Bootstrap: docker
  From: <image>
  

  Its hash is the simulator version, as for any definition. No schema change.
• Download: _download_or_build_container pulls docker://<image> itself for a prebuilt definition, instead of the shared repository's <hash> tag. Building that definition (singularity build --fakeroot) stays the fallback.
• Unchanged: one SIF per definition, the sbatch template, results, status. The image's entrypoint must answer the job command:

  singularity run --compat --bind <exp>:/experiment <sif> run /experiment/<id>.<suffix> -o /experiment/output -n <interval>
  

Enabling it

Once the image is published (vivarium-collective/viva-pde-particle#46 builds and pushes it), add to kustomize/config/compose-api-rke/api.env:

PREBUILT_SIMULATORS={"viva-pde-particle": "ghcr.io/vivarium-collective/viva-pde-particle-compose:sha-<short>"}

That deployment config change isn't in this PR.

Interplay with #184 / #185 (registry enforcement)

Those refuse process addresses missing from this repo's registry manifest. A prebuilt image brings its own processes, so for simulator=<name> submissions the check should be skipped, or should consult the image's own manifest. Whichever lands second adapts. I left #184 alone here, because this branch is based on main.

Not in this PR

• The 30 min / 2 CPU / 8 GB job limits stay as they are. Per-simulator resources would be a natural follow-up; ensemble documents can be sized to fit.

Tests

tests/simulation/test_prebuilt.py covers:

• the definition round trip;
• only listed names resolving;
• the recorded simulator version;
• the pull source for prebuilt (docker://<image>) versus shared (<repository>:<hash>) simulators;
• the 400 for an unlisted name.

make check is clean (ruff, mypy --strict, deptry). Test runs:

• non-SLURM: 35 passed;
• SLURM simulation tests on the throwaway cluster: 6 passed, 3 skipped (cluster only).

🤖 Generated with Claude Code