Repository navigation
Allow Evaluation Using Multiple Python Versions #158
Description
Description
Install multiple python versions within the container, and add an option in the API to select between them.
Rationale
Currently, the bot supports evaluation of both 3.10 and 3.11 code, something I think should be kept and expanded in the future (to include major/important python versions). This is achieved by using the latest 3.10 image published here. This is fine for now, but it makes it harder to provide feature parity and avoid issues when users switch between modes, and it makes it difficult to patch and fix security issues.
Implementation
The way I see us doing this is by building all the versions of python we want to support directly into the image, then aiming the jail at the right executable. This would simplify implementation by having one server, with one codebase, while allowing us to evaluate user code on any version.
Specifics
Some more specific implementation details that can be up for debate:
- The docker image will simply install whatever versions of python you pass to it in the build arg, to well defined locations
- We'll build two images in CI, one with all versions we want supported, and one lightweight for dev
- We can expose an endpoint to get all supported versions
- We should investigate the best way to install all the versions to the docker image. I imagine the best solution is to just start from an image with all the required build-tools, install all the python versions, and use that as the base for the image. I found this docker image which has all the versions from 2.3-3.9 installed, and it comes out to a whooping... 600 MBs! I don't think this specific project would work for us since it hasn't been updated since 3.9, but it does make the idea seem feasiable.
- Are the exposed packages going to be an issue, or can we simply install all the same versions to all python interpreters.
Activity
onerandomusername commentedon Nov 20, 2022
I think it would be a nice idea to be able to have different packages installed. Any c-extension could compile differently based on the targeted python version, so it would be nice to have different versions.
Temporarily remove suport for evaling under 3.10
Remove the snekbox 3.10 deployment
Remove the snekbox 3.10 deployment (#152)
Remove mentions fo there being multiple snekbox containers
Remove mentions of there being multiple snekbox containers
Remove mentions of there being multiple snekbox containers (#988)
Temporarily remove suport for evaling under 3.10 (#2618)
ChrisLovering commentedon Oct 3, 2023
The snekbox Dockerfile now installs both 3.12.0 and 3.13-dev to
/lang/python/<version>.There is also
/lang/python/defaultwhich is a symlink to/lang/python/3.12/which is what snekbox currently uses for eval.Now we need to update snekbox's API to support multiple versions.
The following is my opinion on how to do that.
A new endpoint
GET /versionsis added, which returns all versions snekbox supports. The response body would include the name, a human friendly name, and whether it is a default. this would allow clients to easily give their users a list of versions that they can use.POST /evalwill be updated to have an optionalversionkey, which is the version to eval the code under. This value of this key would be thenamefield fromGET /versions. If it is not given, then the version marked as default inGET /versionsis used instead.MarkKoz commentedon Oct 4, 2023
@ChrisLovering This was essentially the approach taken in #175, which has some discussion.
I think it is better to accept an arbitrary path to a binary to execute. I'm hesitant to make design decisions that pigeon-hole snekbox into just a Python evaluator or just a code evaluator in general. Technically there is not much difference between that and sandboxing arbitrary binaries.
On the other hand, that is the common use case, so we can compromise here. I don't see a strong need for exposing versions via an API, but if we were to do it, I think it should return the paths to the binaries, so they can be passed to the eval endpoint.
onerandomusername commentedon Oct 17, 2025
Noticing there's not further progress on this, but it seems that snekbox supports this, just undocumented. What do we need to do to move forward on this?
jb3 commentedon Oct 17, 2025
There are ongoing internal discussions about this that probably need condensing into a well-formed public issue at some point.
Step 1 is complete, we do build additional interpreters in the snekbox image. We do this in a slightly different order to the initially proposed order (we build interpreters first and then build snekbox on top of that). I believe the reason we do this is probably for caching (building new interpreters is expensive, and if we can avoid changing those layers it's better).
This is partially documented, though not in the best places. We do document how to add new interpreters here: https://github.com/python-discord/snekbox/blob/main/.github/CONTRIBUTING.md#adding-and-updating-python-interpreters
Tasks still needed are:
/snekbin/and theexecutable_pathbody parameter to the/evalendpoint/snekbin/files whilst supporting symlinks from there to prevent accidental restriction of functionality.The idea with that last point is that the bot no longer needs updating when we change Python versions in snekbox but that's potentially something we can close this issue without solving.
The initial comment about exposed packages has proved to be mostly a non-issue as we just use version specifiers and update which packages are available in which interpreter environment as and when they become compatible.
The TL;DR is that we do support this issue 90% but I'm not sure the way we currently do this is the best, it certainly makes snekbox a lot less attractive to external users now that we bundle a bunch of other interpreter versions in our distributed images.
onerandomusername commentedon Oct 17, 2025
As cool as snekbox is, it is literally named snekbox: snake: python. This name sort of locks into being a python code runner rather than anything else. Plus, the endpoint is at
/eval, at the root.Changing scope would require updating and migrating the API in a backwards compatiable manner.
As for building interpreters, they can probably be installed with python build standalone, at least for the "mainstream" ones (jit would need to be built somewhere)
jb3 commentedon Oct 17, 2025
This is a little bit ridiculous and I disagree completely, I think that locking project scopes based on names is silly (unless they are descriptive names like
python-eval-apior something). Snekbox can be whatever we want and we actively have had ideas internally and in #dev-contrib before about opening up other options for !eval (things like allowing an!evalversion for things like bash).We named it because of who made it and it was an amusing name.
We aren't changing scope. We had to amend snekbox to support multiple python versions and would have to amend it again to support wider more flexible eval options (like
bashor whatever).Our current build solution works fine, we probably just need to rearrange things into the following:
./python_build.shscript (see how we currentlyCOPYbuilt Python versions)COPY --fromoptions to copy built Python versions out of pre-built and cacheable images.All of this is based on what we already have, based on our existing build Dockerfile and is more a rearrangement and separation of steps rather than a major refactor.
Additionally, since we never pushed snekbox far and wide we don't need to be hard backwards-compatible. We don't have versions of snekbox (but these changes would make it much nicer and easier to move towards having proper versioning for snekbox base).
onerandomusername commentedon Oct 17, 2025
(edit: wrote this before the above reply)
Here are my thoughts, disregard if its not helpful.
Details
My thoughts are as follows
Deprecate /eval, replace with the following:
/python
/python/exec/[version]
add /exec which can use a specific binary, basically the current /eval functionality minus the python specific endpoints.
For not bundling python: Add support for inside (os path) /snekbin where a user can add a volume containing python binaries. That folder contains an additional versions.json file which includes an object mapping of python version to which binary in there is for that version. This versions.json file should be complete with the patch version, eg 3.13.6t for 3.13.6 free threading.
If that folder is not found, /python returns an empty list, and /python/exec/[version] returns the same as if an invalid version is found.
This theoretical new api would look like the following:
POST /exec -> run with specified binary, no defaults
POST /eval -> deprecated, now undocumented
GET /python -> supported python versions
POST /python/exec -> run python code, akin to the existing /eval endpoint but without
{"input":...}supportGET /exec -> list the binaries in /snekbin which are the supported versions
jb3 commentedon Oct 17, 2025
This seems overcomplicated. My mental construction of how this looks is a lot simpler.
python-3.14j,python-3.14in/snekbin/(as we currently do), these can be added by a new Docker stage that builds after the base (instead of before like we currently have, see the end of my above comment)/snekbin/contents via a new API endpoint, confirm they have execute bit set, the symlink destinations exist, etc.python-*, that is what I will show the user" (and as an example is what Python bot would grab on start-up/when refreshed by an admin)executable_pathIf users want to do higher level orchestrating of mapping versions to binaries that can be handled externally by their own bespoke solution/JSON file/etc, but this doesn't need to be more complex than returning a list of files.
For this specific issue, the above is all that is really needed to close this issue and the wider things can come as new issues down the line if we decide to proceed that way.
onerandomusername commentedon Oct 17, 2025
That makes sense. I do like the idea of python being specially handled, but its not a requirement.
jb3 commentedon Oct 17, 2025
I think if we are trying to generalise then keeping specific functionality is something we need to carefully consider, and adding an endpoint of
/python/exec/3.14that just maps into/snekbin/python-3.14doesn't seem like it's worth keeping a specific Python thing.The client already has to know the version to call that endpoint, and having it as a path-param vs a generic backwards compatible body parameter doesn't give a huge benefit.