Repository navigation
Install Elyra image packages into system site - #3454
Merged
Merged
Conversation
The elyra/elyra image ran pip as jovyan. In the base image, the system site-packages under /usr/local belongs to root, so pip fell back to a user install and put Elyra, its lab extensions and its server config in /home/jovyan/.local. A volume mounted on the home directory, which is the Zero to JupyterHub default, hid all of it: users got the base image's JupyterLab 4.1.5 and no Elyra. Run the installs as root in a single layer instead, then switch back to jovyan. HOME is still /home/jovyan under root, so pip's cache is disabled on every install to keep root-owned files out of the user's home. This also drops a 39 MB cache the image used to ship. Add `make validate-elyra-image`, which runs validate-image.sh inside the image as built and with an empty tmpfs on the home directory, and run it from create-release.py before images are published. Document how to clean up a home volume that still holds a copy from the 4.1.x or 4.2.0 images, since that copy would take precedence. Fixes elyra-ai#3453 Signed-off-by: Luciano Resende <lresende@apple.com>
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #3454 +/- ##
=======================================
Coverage 84.81% 84.81%
=======================================
Files 159 159
Lines 19297 19297
Branches 502 502
=======================================
Hits 16366 16366
Misses 2891 2891
Partials 40 40
Flags with carried forward coverage won't be shown. Click here to find out more. 🚀 New features to boost your workflow:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changes were proposed in this pull request?
The
elyra/elyraimage installed Elyra and its dependencies in the user site (/home/jovyan/.local) instead of the system site-packages. pip ran asjovyan, and injupyterhub/k8s-singleuser-sample:3.3.7the system site-packages under/usr/localbelongs to root, so pip fell back to a user install. A volume mounted on/home/jovyan, which is the Zero to JupyterHub default, hid all of it. Users got JupyterLab without Elyra, and it was the base image's JupyterLab 4.1.5: the 17 base packages Elyra upgrades had also been upgraded only in the user site.Image (
etc/docker/elyra/Dockerfile):rootin one layer, then the image switches back tojovyan. Packages, lab extensions and server config now go to/usr/local/lib/python3.11/site-packages,/usr/local/share/jupyter/labextensionsand/usr/local/etc/jupyter, which are on Jupyter's default system paths.--no-cache-dir.HOMEis still/home/jovyanunder root, so a cache would leave root-owned files in the user's home. This also drops the 39 MB cache therequirements.txtinstall used to leave in the image.--root-user-action=ignoreis passed per command rather than throughENV PIP_ROOT_USER_ACTION, so the setting doesn't carry into the runtime environment.PYTHONNOUSERSITE=1, suggested in the issue, is not set. It turns off the user site entirely:pip installasjovyanwould fail with a permission error instead of falling back to~/.local, and packages users already installed there would stop importing.Validation (
etc/docker/elyra/validate-image.sh,make validate-elyra-image):/home/jovyan, like a fresh JupyterHub user volume. A named volume doesn't work for this, because Docker seeds a new named volume with the image's home directory, which hides the failure.elyraimports from the system prefix and that the@elyralab extensions and theelyraserver extension are enabled and OK. As built, it also checks that the user site is empty and that there are no root-owned files in the home directory.create-release.pyruns it aftermake container-images, so a broken image fails the release before anything is published.make validate-elyra-image image=<name>validates any other image.Docs:
root. A new section explains how to clean up a home volume that still holds a copy from the 4.1.x or 4.2.0 images, since that copy takes precedence over the image's.etc/docker/elyra/README.mddocuments the validation target and drops the reference to aDockerfile.devthat doesn't exist.How was this pull request tested?
make validate-elyra-image image=quay.io/elyra/elyra:4.2.0fails 3 of 5 checks as built, and all 3 with an empty home (ModuleNotFoundError: No module named 'elyra').Defaulting to user installationor root-user warnings, and the home directory in the image is 16 KB, down from 752 MB.pip installasjovyanstill goes to~/.localand imports, andPIP_ROOT_USER_ACTIONisn't set.~/.localon the fixed image. After the two documented commands it loads from/usr/local, and the validator passes.blackandbash -npass.jupyterhub/k8s-singleuser-sample:3.3.7, because Docker Hub wasn't reachable from the build environment. The stand-in is the 4.2.0 image with~/.localand~/.cacheremoved. Every layer 4.2.0 added on top of the base wrote only there or to/usr/local/bin/start-elyra.sh, so its/usr/localmatches the base. To check on the real base:make release elyra-image validate-elyra-image. A Zero to JupyterHub deployment wasn't tested either.Fixes #3453
Developer's Certificate of Origin 1.1