Repository navigation
Fix kf-notebook config hidden by workspace volume - #3456
Merged
Merged
Conversation
The kf-notebook image wrote its Elyra customizations to $HOME during the build: a jupyter_elyra_config.py that enables only the Kubeflow Pipelines runtime, and the examples component catalog. Kubeflow mounts the workspace volume on the home directory and seeds a new volume only from /tmp_home, so neither reached the notebook server. Users got the Airflow runtime and no examples catalog. Install both in the conda environment instead. The config goes to jupyter_core's sys.prefix config dir, which Jupyter loads before ~/.jupyter, so users can still override it. The catalog is created with ELYRA_METADATA_PATH pointing at the sys.prefix data dir, where Elyra lists it as a factory instance next to its runtime images, and is made world-readable like them. Write the config directly instead of generating the template and editing it with sed, which silently did nothing if the template's commented-out default changed. The pip upgrade no longer leaves a cache in $HOME either. Add `make validate-kf-notebook-image`, which starts the image as built and with an empty tmpfs on the home directory, and checks the runtime types and catalogs through Elyra's REST API. Run it from create-release.py before images are published. Signed-off-by: Luciano Resende <lresende@apple.com>
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #3456 +/- ##
==========================================
- Coverage 84.83% 84.82% -0.01%
==========================================
Files 159 159
Lines 19297 19297
Branches 502 500 -2
==========================================
- Hits 16370 16369 -1
- Misses 2887 2889 +2
+ Partials 40 39 -1
Flags with carried forward coverage won't be shown. Click here to find out more. 🚀 New features to boost your workflow:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changes were proposed in this pull request?
The
elyra/kf-notebookimage loses its Elyra customizations on Kubeflow. With an empty volume on the home directory, which is what a new Kubeflow workspace volume looks like, the server reports the Airflow runtime and no component catalogs:KUBEFLOW_PIPELINES,LOCALkubeflow_pipelines_examplesAPACHE_AIRFLOW,KUBEFLOW_PIPELINES,LOCALThe build wrote both customizations to
$HOME:~/.jupyter/jupyter_elyra_config.py, which enables only the KFP runtime, and the examples catalog in~/.local/share/jupyter/metadata. Kubeflow mounts the workspace volume on/home/jovyan, and its01-copy-tmp-homeinit script seeds a new volume only from/tmp_home, so neither file reached the notebook server. Elyra's packages aren't affected:/opt/condabelongs tojovyan, so pip installs into the conda environment. This is related to #3453, which was about packages in theelyra/elyraimage.Image (
etc/docker/kubeflow/Dockerfile):jupyter_core'ssys.prefixconfig dir (/opt/conda/etc/jupyter). Jupyter loads it before~/.jupyter, so users can still override it there.c.PipelineProcessorRegistry.runtimes = ['kfp']) instead of generating the template and editing it withsed, which silently did nothing if the template's commented-out default changed.ELYRA_METADATA_PATHpointing at thesys.prefixdata dir (/opt/conda/share/jupyter). Elyra lists metadata from there as factory instances, next to its runtime images, and a user's own instance with the same name still takes precedence. Elyra writes new metadata for the owner only, so the catalog is made world-readable to matchruntime-images.--no-cache-dir, so it no longer leaves a cache in$HOME.Validation (
etc/docker/kubeflow/validate-image.py,make validate-kf-notebook-image):/init), as built and with an empty tmpfs on/home/jovyan, and pipes the script into the running container.create-release.pyruns it aftervalidate-elyra-image, so a broken image fails the release before anything is published.make validate-kf-notebook-image image=<name>validates any other image.The kubeflow README documents the target and explains why the image doesn't write to
$HOME.How was this pull request tested?
The image was built on the real
ghcr.io/kubeflow/kubeflow/notebook-servers/jupyter:v1.8.0base. Docker Hub wasn't reachable from the build environment, so the build used a copy of the Dockerfile without its# syntax=docker/dockerfile:experimentalline (no other changes; nothing in this Dockerfile needs that frontend) and the legacy builder (DOCKER_BUILDKIT=0), because BuildKit couldn't get a token forghcr.io.quay.io/elyra/kf-notebook:4.2.0. As built, all three pass./opt/conda/etc/jupyter, the catalog is in/opt/conda/share/jupyter/metadata/component-catalogswith the same permissions asruntime-images(directory755, file644), and nothing Elyra-related is in$HOME.c.PipelineProcessorRegistry.runtimes = ["kfp", "airflow"]in~/.jupyter/jupyter_elyra_config.pyon a persistent volume, the server enables Airflow after a restart.docker run/docker execcommands directly, because the build environment's sandbox doesn't allowmaketo reach the Docker socket.blackpasses, andflake8passes with the repo's 120-character limit.make release kf-notebook-image validate-kf-notebook-image.Developer's Certificate of Origin 1.1