Skip to content

Install Elyra image packages into system site - #3454

Merged
lresende merged 1 commit into
elyra-ai:mainfrom
lresende:fix-image-user-site
Nov 13, 2026
Merged

lresende merged 1 commit into
elyra-ai:mainfrom
lresende:fix-image-user-site

Conversation

@lresende

Copy link
Copy Markdown
Member

What changes were proposed in this pull request?

The elyra/elyra image installed Elyra and its dependencies in the user site (/home/jovyan/.local) instead of the system site-packages. pip ran as jovyan, and in jupyterhub/k8s-singleuser-sample:3.3.7 the system site-packages under /usr/local belongs to root, so pip fell back to a user install. A volume mounted on /home/jovyan, which is the Zero to JupyterHub default, hid all of it. Users got JupyterLab without Elyra, and it was the base image's JupyterLab 4.1.5: the 17 base packages Elyra upgrades had also been upgraded only in the user site.

Image (etc/docker/elyra/Dockerfile):

  • The three pip installs run as root in one layer, then the image switches back to jovyan. Packages, lab extensions and server config now go to /usr/local/lib/python3.11/site-packages, /usr/local/share/jupyter/labextensions and /usr/local/etc/jupyter, which are on Jupyter's default system paths.
  • Every install uses --no-cache-dir. HOME is still /home/jovyan under root, so a cache would leave root-owned files in the user's home. This also drops the 39 MB cache the requirements.txt install used to leave in the image.
  • --root-user-action=ignore is passed per command rather than through ENV PIP_ROOT_USER_ACTION, so the setting doesn't carry into the runtime environment.
  • PYTHONNOUSERSITE=1, suggested in the issue, is not set. It turns off the user site entirely: pip install as jovyan would fail with a permission error instead of falling back to ~/.local, and packages users already installed there would stop importing.

Validation (etc/docker/elyra/validate-image.sh, make validate-elyra-image):

  • Runs inside the image twice: as built, and with an empty tmpfs on /home/jovyan, like a fresh JupyterHub user volume. A named volume doesn't work for this, because Docker seeds a new named volume with the image's home directory, which hides the failure.
  • Checks that elyra imports from the system prefix and that the @elyra lab extensions and the elyra server extension are enabled and OK. As built, it also checks that the user site is empty and that there are no root-owned files in the home directory.
  • create-release.py runs it after make container-images, so a broken image fails the release before anything is published. make validate-elyra-image image=<name> validates any other image.

Docs:

  • The custom image example in the JupyterHub recipe now installs packages as root. A new section explains how to clean up a home volume that still holds a copy from the 4.1.x or 4.2.0 images, since that copy takes precedence over the image's.
  • etc/docker/elyra/README.md documents the validation target and drops the reference to a Dockerfile.dev that doesn't exist.

How was this pull request tested?

  • Fails on 4.2.0: make validate-elyra-image image=quay.io/elyra/elyra:4.2.0 fails 3 of 5 checks as built, and all 3 with an empty home (ModuleNotFoundError: No module named 'elyra').
  • Passes on the fix: all 8 checks pass on an image built from this Dockerfile. The build log has no Defaulting to user installation or root-user warnings, and the home directory in the image is 16 KB, down from 752 MB.
  • Checks can fail: planting a user-site directory and a root-owned file in the fixed image makes both structural checks fail.
  • Runtime: with an empty home, JupyterLab is 4.6.6 rather than the base's 4.1.5. A plain pip install as jovyan still goes to ~/.local and imports, and PIP_ROOT_USER_ACTION isn't set.
  • Upgrade note: on a named volume seeded from the 4.2.0 image, Elyra loads from ~/.local on the fixed image. After the two documented commands it loads from /usr/local, and the validator passes.
  • black and bash -n pass.
  • Not covered: the build above used a stand-in for jupyterhub/k8s-singleuser-sample:3.3.7, because Docker Hub wasn't reachable from the build environment. The stand-in is the 4.2.0 image with ~/.local and ~/.cache removed. Every layer 4.2.0 added on top of the base wrote only there or to /usr/local/bin/start-elyra.sh, so its /usr/local matches the base. To check on the real base: make release elyra-image validate-elyra-image. A Zero to JupyterHub deployment wasn't tested either.

Fixes #3453

Developer's Certificate of Origin 1.1

   By making a contribution to this project, I certify that:

   (a) The contribution was created in whole or in part by me and I
       have the right to submit it under the Apache License 2.0; or

   (b) The contribution is based upon previous work that, to the best
       of my knowledge, is covered under an appropriate open source
       license and I have the right under that license to submit that
       work with modifications, whether created in whole or in part
       by me, under the same open source license (unless I am
       permitted to submit under a different license), as indicated
       in the file; or

   (c) The contribution was provided directly to me by some other
       person who certified (a), (b) or (c) and I have not modified
       it.

   (d) I understand and agree that this project and the contribution
       are public and that a record of the contribution (including all
       personal information I submit with it, including my sign-off) is
       maintained indefinitely and may be redistributed consistent with
       this project or the open source license(s) involved.

The elyra/elyra image ran pip as jovyan. In the base image, the system
site-packages under /usr/local belongs to root, so pip fell back to a
user install and put Elyra, its lab extensions and its server config
in /home/jovyan/.local. A volume mounted on the home directory, which
is the Zero to JupyterHub default, hid all of it: users got the base
image's JupyterLab 4.1.5 and no Elyra.

Run the installs as root in a single layer instead, then switch back
to jovyan. HOME is still /home/jovyan under root, so pip's cache is
disabled on every install to keep root-owned files out of the user's
home. This also drops a 39 MB cache the image used to ship.

Add `make validate-elyra-image`, which runs validate-image.sh inside
the image as built and with an empty tmpfs on the home directory, and
run it from create-release.py before images are published. Document
how to clean up a home volume that still holds a copy from the 4.1.x
or 4.2.0 images, since that copy would take precedence.

Fixes elyra-ai#3453

Signed-off-by: Luciano Resende <lresende@apple.com>
@lresende lresende added this to the 4.3.0 milestone Nov 13, 2026
@lresende lresende added the kind:bug Something isn't working label Nov 13, 2026
@codecov

codecov Bot commented Nov 13, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 84.81%. Comparing base (a6187dd) to head (472078c).

Additional details and impacted files

Impacted file tree graph

@@           Coverage Diff           @@
##             main    #3454   +/-   ##
=======================================
  Coverage   84.81%   84.81%           
=======================================
  Files         159      159           
  Lines       19297    19297           
  Branches      502      502           
=======================================
  Hits        16366    16366           
  Misses       2891     2891           
  Partials       40       40           
Flag Coverage Δ
integration-editors 48.14% <ø> (ø)
integration-misc 25.64% <ø> (ø)
integration-pipeline 47.58% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.
see 2 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@lresende
lresende merged commit 26674d3 into elyra-ai:main Nov 13, 2026
19 checks passed
@lresende
lresende deleted the fix-image-user-site branch November 13, 2026 15:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

kind:bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Elyra is installed in ~/.local, so it disappears under JupyterHub with persistent home storage

1 participant