eggd_purple_plotter

A DNAnexus app that takes per-sample outputs from the PURPLE/AMBER/CNVkit pipeline and generates a single-file IGV.js HTML copy-number viewer.

igv.js 3.8.3 and all track data are embedded in the HTML, so no server and no third-party script are needed. The hg38 genome (sequence, cytobands, aliases) and the RefSeq annotation are fetched from igv.org and UCSC when the file is opened, so an internet connection to those two hosts is needed to render it. See Security.

---

What it does

Given per-sample input files, the app:

1. Validates that at least one of purple_tar or cnvkit_cnr is present
2. Extracts the PURPLE tar archive (AMBER BAF, target-region CN, PURPLE segments)
3. Reads purity, ploidy and the fit status from the purity/ploidy report (PURPLE purity.tsv or compatible). If none is supplied, purity, ploidy and fit status show ? and a warning states that ploidy 2.0 is assumed for the CN/ploidy segment tracks
4. Reads MSI score (defaults to ? if absent)
5. Converts all available data to BEDGraph/SEG strings
6. Renders a single HTML file with up to 7 igv.js tracks (igv.js and track data embedded)
7. Uploads the HTML and registers it as the igv_html output

Tracks whose input was not supplied are omitted. If an input was supplied but has no rows to plot (for example a genemetrics file with no genes), its track is kept, empty, and its name ends in "(no data)", so the two cases can be told apart.

---

Inputs

Name	Class	Required	Description
sample_id	string	✅	Sample identifier; also the output file name, so it must not contain /, \ or NUL, or be ./.. (the job fails otherwise)
purple_tar	file	one of these two	{sample}.purple.tar.gz from the PURPLE pipeline stage
cnvkit_cnr	file	one of these two	{sample}.cnr from CNVkit
qc_report	file	—	Purity/ploidy TSV: PURPLE's final-pass {sample}.purple.purity.tsv, or a compatible report such as {sample}.qc_report.tsv. See Purity, ploidy and fit status
cnvkit_call_cns	file	—	{sample}.call.cns from CNVkit
cnvkit_genemetrics	file	—	{sample}.genemetrics.tsv (CNVkit batch v2.1.0) or the older {sample}.genemetrics.csv; both tab-separated
msi_report	file	—	{sample}.msi.tsv
locus	string	—	Initial IGV locus (default: all)
genome_image	boolean	—	Also render the whole-genome view to PNG and SVG (default: false). See Export an image of the whole-genome view

At least one of purple_tar or cnvkit_cnr must be supplied. The app exits with an error if both are absent.

Purity, ploidy and fit status

• qc_report takes any tab-separated file with a header row naming purity and ploidy columns, whose first data row gives their numeric values; columns other than these and the optional status are ignored. This includes PURPLE's final-pass {sample}.purple.purity.tsv (output purity_tsv of eggd_cgp-purple v1.0.1). Fractional ploidy is used as is: PURPLE segments are drawn as log₂(CN / ploidy), and so are CNVkit called segments that carry a cn value.
• status, if present, is shown as Fit status (blank or missing → UNKNOWN). It is the fit status from that file, for example NORMAL or NO_TUMOR. It is not a QC pass/fail assessment: the plotter does not evaluate PURPLE's QC file or eggd_cgp-qc-flags flags.
• With no qc_report, purity, ploidy and fit status show ?, and the CN/ploidy segment tracks use ploidy 2.0. That is a rendering assumption, not a measured ploidy, and the viewer header and exported images say so.
• A supplied file fails the job, with a message naming the problem, if it is missing, empty or unreadable; is not tab-separated; has no numeric purity/ploidy; has a first data row whose field count differs from the header; or has purity outside 0–1 or ploidy below 0.1 (the segment conversion cannot draw a smaller ploidy as given). It is never treated as "no report".
• Intended Atlas CNV workflow v2.0.0 link (not yet built or signed off): stage-purple.purity_tsv → stage-purple_plotter.qc_report.

Output

Name	Class	Description
igv_html	file	{sample_id}.igv.bars.html — HTML viewer (igv.js and track data embedded; genome fetched when opened)
genome_png	file (optional)	{sample_id}.genome.png — whole-genome view with the sample/QC banner; only when genome_image=true
genome_svg	file (optional)	{sample_id}.genome.svg — the same view as SVG; only when genome_image=true

---

Tracks in the output HTML

#	Track	Source
1	Coverage + CN log₂ bars (merged)	PURPLE target-region CN
2	PURPLE CNV segments (SEG)	PURPLE segments
3	BAF scatter + fitted BAF bars (merged)	AMBER BAF + PURPLE segments
4	PURPLE CNV segments strip (SEG)	PURPLE segments
5	CNVkit log₂ scatter (merged)	CNVkit CNR
6	CNVkit called segments (SEG)	CNVkit CNS
7	CNVkit genemetrics bar chart	CNVkit genemetrics

igv.js 3.8.3 is embedded in the HTML (from resources/home/dnanexus/purple_plotter/assets/igv.min.js).

---

Quick start

Run locally

# 1. Clone and create venv
git clone https://github.com/eastgenomics/eggd_purple_plotter
cd eggd_purple_plotter
python3 -m venv .venv
.venv/bin/pip install -r requirements.txt

# 2. Run tests
.venv/bin/pytest tests/ -v

# 3. Generate HTML from local files (PURPLE only)
.venv/bin/python3 resources/home/dnanexus/purple_plotter/purple_plotter.py \
    --sample 26134S0005 \
    --purple_tar /path/to/26134S0005.purple.tar.gz \
    --qc_report  /path/to/26134S0005.qc_report.tsv \
    --output_dir /tmp/

# 4. Open the result
open /tmp/26134S0005.igv.bars.html

Export an image of the whole-genome view

Use igv.js's own Save Image menu in the viewer toolbar (Save as SVG / Save as PNG). In these viewers, it exports the current view (whatever locus is on screen) with a banner carrying the sample ID, purity, ploidy, status, MSI, genome build and the locus shown, so an image is never separated from its sample or region. The file is named {sample}.{locus}.svg / .png. The per-track right-click "Save Image" items are igv.js's plain single-track export, without the banner. The viewer area stays invisible and inert (it cannot be clicked; a "Loading viewer…" status is shown, then kept for screen readers only) until this is in place, so the toolbar can never save an unlabelled image; if igv.js fails to start, that line shows the error instead.

For a file without opening a browser, render the whole-genome view headlessly:

.venv/bin/pip install playwright==1.64.1          # not needed by the app itself
.venv/bin/python3 resources/home/dnanexus/purple_plotter/render_image.py \
    --html /tmp/26134S0005.igv.bars.html \
    --out_prefix /tmp/26134S0005.genome \
    --chromium_path /usr/bin/chromium            # or set $CHROMIUM_PATH
# -> /tmp/26134S0005.genome.svg, /tmp/26134S0005.genome.png

No network is used: igv.js is embedded in the HTML, and the hg38 chromosome-size, cytoband and alias files are served from resources/home/dnanexus/purple_plotter/assets/ (see its README). The image has no gene-annotation track.

On DNAnexus, set genome_image=true to also get {sample_id}.genome.png and .svg outputs. Playwright and Chromium come from the pp_headless_chromium asset (source in asset/), pinned by record ID under runSpec.assetDepends in dxapp.json, so a build always uses that exact bundle. The asset is always downloaded to the worker, even when genome_image is false, because assetDepends cannot be conditional.

To rebuild the asset (needs CONTRIBUTE on the target project and a --bill-to-able region; the 001_Reference project is UPLOAD-only for most users, so build in a dev project and dx cp the record):

dx mkdir -p <project>:/app_assets/pp_headless_chromium/pp_headless_chromium_v1.0.0
dx build_asset asset -d <project>:/app_assets/pp_headless_chromium/pp_headless_chromium_v1.0.0/

Current record: 001_Reference:/app_assets/pp_headless_chromium/pp_headless_chromium_v1.0.0/pp_headless_chromium (record-JFV8Y9Q42qG5FFbkGjQq23Kj). Build takes ~5 min; ~440 MB installed. After rebuilding, put the new record ID in dxapp.json ("id", with name/version kept for readability).

Run on DNAnexus

# Build (dev/test — creates a fast-iterating applet in a single project)
dx build eggd_purple_plotter/ \
    --destination "project-xxxx:/applets/" --overwrite

# Run
dx run applet-xxxx \
    -isample_id="26134S0005" \
    -ipurple_tar="project-xxxx:file-aaa" \
    -iqc_report="project-xxxx:file-bbb" \
    --destination "project-xxxx:/igv_plots/" \
    --priority high --watch

Release

Releases follow the team's sign-off workflow in the Bioinformatics Development Manual v3. A release is the app object (an applet is for development only), and the app that is published is the one that was tested and signed off. Nothing is rebuilt at release.

During development (DEVELOPMENT)

1. Before the first build, make sure the app's authorised users and developers on DNAnexus are only org-emee_1, as in dxapp.json (dx api app-eggd_purple_plotter listDevelopers and listAuthorizedUsers; change them with dx add|remove developers|users). dx build applies the manifest's lists to the whole app, including published versions.
2. Build from a clean checkout of the reviewed commit on the release branch: dx build --app --no-version-autonumbering --yes (add --bill-to <org> if needed). While the version is unpublished, later builds update it in place; rebuild whenever the code changes, and re-run the tests. --no-version-autonumbering stops dx build creating an X.Y.Z+build.<timestamp> version.
3. Check the built app's version is exactly the dxapp.json version (dx describe <app-id>), and record in the Confluence testing document the app ID, the commit it was built from, and the app's modified time.
4. Run the validation jobs with that app ID.
5. Run CI on the release branch (gh workflow run CI --ref <release-branch>). Check the new run's commit is the recorded one and that it passed. Link it in the testing document: its release-evidence-<commit> artefact holds the SBOM (sbom.json) and the hashed lists of what ships, and its security job shows the pip-audit and Trivy results. (CI on a pull request runs on the PR's merge commit, so it is not the evidence for the build.)

If the code changes after the testing document records an app ID, rebuild, re-test and update the document before SIGNOFF. Do not build again after sign-off.

After the Approver moves the story to DEPLOY TO PROD

The repository ruleset that blocks updates to and deletion of v* tags must be active, so a tag keeps pointing at the commit that was signed off.

1. Check the release branch's head is still the tested commit (if it has moved, stop: the change needs building, testing and sign-off again). Then merge the branch's pull request into main with a merge commit (not squash or rebase), so the tested commit stays in main's history and release.yml is on the default branch.
2. Check the app is still the one in the testing document: same ID and version, the same modified time, and not yet published.
3. Tag the tested commit vX.Y.Z and push the tag. If the tag already exists, check it points at the tested commit before going on.
4. Publish the GitHub release for that tag with notes on what changed. Wait for release.yml to pass and check the SBOM and hashed lists are attached to the release. If it fails, find out why before going on.
5. Publish the tested app: dx publish eggd_purple_plotter/X.Y.Z.

Then run it as before: dx run app-eggd_purple_plotter with the same -i arguments as the applet example above.

---

Security

Every generated viewer has the same structure:

1. A <meta http-equiv="Content-Security-Policy"> as the first element after <meta charset>.
2. One non-executable <script type="application/json" id="pp-data"> block holding every per-sample value: all track data, the initial locus and the download file name. It is JSON-encoded with <, >, & and all non-ASCII characters escaped, so a sample ID or file content can never end the element or start a comment.
3. igv.js 3.8.3, inlined from assets/igv.min.js and checked against its SHA-256 when loaded (IGV_JS_SHA256 in purple_plotter.py). Its MIT licence is reproduced in an HTML comment above it; the DOMPurify notice is inside the script.
4. The viewer code, inlined from resources/home/dnanexus/purple_plotter/viewer.js.

Items 3 and 4 are byte-identical for every sample. igv.js builds click popups as HTML from raw values, so for segment tracks, whose popups show the sample ID from the SEG data, viewer.js supplies its own popup content with every value HTML-escaped (the CSP stops scripts, not injected markup). The embedded coverage, CN, BAF and CNVkit wig/merged tracks keep igv.js's own popup, which holds only numbers, our fixed track names and igv.js's own markup. The RefSeq gene track is external annotation from UCSC, rendered by igv.js's own popup with its text and links, as in any igv.js viewer; the per-sample HTML-context values (<title>, header) are HTML-escaped. Tabs, CR and LF in the sample ID are replaced with _ in the SEG track data, where the ID is a tab-delimited column. The CSP lets only those two scripts run, by hash:

default-src 'none';
script-src 'sha256-<igv.js>' 'sha256-<viewer.js>';
connect-src blob: https://igv.org https://hgdownload.soe.ucsc.edu;
img-src data: blob:;
style-src 'unsafe-inline';
base-uri 'none';
form-action 'none';
object-src 'none'

• connect-src blob: — igv.js reads the embedded track data through blob: URLs.
• https://igv.org — genomes3.json, cytobands, aliases, the MANE gene-search index.
• https://hgdownload.soe.ucsc.edu — the hg38 2bit sequence and the RefSeq track (igv.js 3.8.3 rewrites the igv.org hg38 FASTA to UCSC).
• img-src data: blob: — igv.js icons, and the PNG export (an SVG blob: drawn into an image).
• style-src 'unsafe-inline' — igv.js injects <style> elements and the legend uses style="". Styles cannot run code.
• No 'unsafe-inline' or 'unsafe-eval' for scripts, and no 'wasm-unsafe-eval': igv.js 3.8.3 contains a WebAssembly module, but this viewer never instantiates it (checked in a browser).

These directives were established in a real browser by recording securitypolicyviolation events while loading the whole genome, a chromosome, a 2 kb and a 100 bp window (sequence from UCSC), a gene search (EGFR), the RefSeq track, and both Save Image menu items, from file:// and http://localhost: no violations. tests/test_csp.py keeps two of these checks, both opening the viewer as a file:// URL:

• test_csp_offline (runs in CI): genome files served from assets/; whole genome, a chromosome and a 2 kb window, and both Save Image menu items. No gene search, sequence or RefSeq track (those need the network).
• test_csp_live (only with PP_LIVE_NETWORK=1): real igv.org/UCSC; adds the EGFR gene search, a 100 bp window with UCSC sequence and the RefSeq track, and checks the 2bit and RefSeq requests were made.

If igv.org's genome list is unreachable, igv.js would fall back to raw.githubusercontent.com, and for the sequence to UCSC's AWS mirror. The CSP blocks both, so the viewer then fails to load rather than reaching other hosts.

A <meta> CSP cannot carry frame-ancestors, reporting or a report-only mode; a site that serves the viewers should add the same policy as a response header, with frame-ancestors 'none'. Browsers enforce both policies, so the header must allow the script hashes of every plotter release whose viewers the site serves (each release's viewer.js has its own hash; list them all while that release's viewers are still served). Viewers from v1.0.0 cannot meet such a policy at all: they load igv.js from the igv.org CDN and their inline script contains the sample's data, so it differs per file. Regenerate them with the v1.1.1 app or later before the header is applied. The alternative, a separate legacy policy that allows the CDN and 'unsafe-inline' (or a hash per viewer), brings back the third-party code and injection risks this release removes.

Script hashes

.venv/bin/python3 resources/home/dnanexus/purple_plotter/purple_plotter.py --print-script-hashes

prints the two sha256-… values. The table gives them for the code tagged v1.1.0 and for v1.1.1, which changes neither script. Viewers made by other builds may differ, so check their hashes before serving them:

Script	Hash
igv.js 3.8.3	sha256-rPTumc3JG+iEIDkw25w5W8hiZ1UpM7SD2gtgnBbqBao=
viewer.js	sha256-wnmjHM4hwBfbWKi8HygdXHo53W2qtoo98Q6QaN62tSM=

Any change to viewer.js or igv.min.js changes these; record the new values with each release.

---

Architecture

src/code.sh                          ← DNAnexus entry point; all platform I/O here
resources/home/dnanexus/
  purple_plotter/purple_plotter.py   ← Pure Python CLI; no dxpy dependency
  purple_plotter/viewer.js           ← Viewer code, inlined unchanged into every HTML
  purple_plotter/assets/             ← igv.min.js (inlined), its licence, offline genome files
  packages/                          ← Bundled wheels for offline pip install
resources/usr/bin/
  mark-section                       ← DNAnexus structured logging helper
  mark-success                       ← DNAnexus job success marker
tests/                               ← pytest unit + CLI integration tests

code.sh downloads inputs, creates a venv, installs bundled wheels, calls purple_plotter.py, uploads the output. It contains no data-conversion logic.

purple_plotter.py has no DNAnexus dependency — it reads local file paths, converts data to BEDGraph/SEG strings, and renders the HTML. It is fully testable without a DNAnexus connection.

Bundling pandas for offline install

The DNAnexus worker has no internet access during a job. Bundle the pandas wheel before building the applet:

pip download pandas \
    --platform manylinux2014_x86_64 \
    --python-version 312 \
    --only-binary=:all: \
    -d resources/home/dnanexus/packages/

---

Development

# Run all tests
.venv/bin/pytest tests/ -v

# Image-rendering tests run only if playwright and a Chromium are available
# (CHROMIUM_PATH=/usr/bin/chromium); otherwise they are skipped.

# Check no dxpy in the Python module
grep "import dxpy\|from dxpy" resources/home/dnanexus/purple_plotter/purple_plotter.py
# expected: no output

# Validate dxapp.json
python3 -c "import json; json.load(open('dxapp.json')); print('valid')"

CI

.github/workflows/pytest.yml runs three jobs on every pull request, on pushes to main, and when dispatched by hand (for the release evidence above):

• test: pytest tests/ -v against the bundled wheels (checked with scripts/release_manifest.py check-installed), with the runner's preinstalled Chrome for the image-export tests.
• lint: Ruff (ruff check resources tests scripts; rules in pyproject.toml), zizmor (zizmor --offline --strict-collection .github/workflows/) and ShellCheck 0.11.0 (pinned via shellcheck-py) on src/code.sh, infra/*.sh and resources/usr/bin/*.
• security: bandit (fails on HIGH only; Ruff's S rules, a subset of bandit's checks, run at every severity in the lint job), pip-audit -r requirements.txt, and pip-audit on what the app ships: the bundled wheels and the asset's Python packages (lists from scripts/release_manifest.py, below), and the Trivy scan of the SBOM. It uploads the SBOM and both lists as the release-evidence-<commit> artefact.

scheduled-scan.yml also runs the same pip-audit scans, a Trivy scan of the SBOM described below, and Trivy (dependencies, misconfiguration and secrets, failing on HIGH or CRITICAL) every Monday. GitHub disables a scheduled workflow after 60 days with no repository activity; re-enable it under Actions if that happens. It installs Trivy itself (pinned release, SHA-256 checked) because the action's installer currently fails.

release.yml runs when a GitHub release is published. It checks that the tag is v plus the dxapp.json version, runs the tests against the bundled wheels, and audits and records what the app actually ships, which is not requirements.txt: the wheels bundled in resources/home/dnanexus/packages/, igv.js embedded in every viewer, and the pp_headless_chromium asset. scripts/release_manifest.py writes the artefacts, which are attached to the release (supply chain, CAF A4.a):

Artefact	Contents
requirements-hashed.txt	each bundled wheel as name==version --hash=sha256: of the shipped file; audited with pip-audit
asset-requirements.txt	the Python packages the asset installs (PINS in asset/Makefile); audited with pip-audit. Taken from the build source at the tagged commit, not read from the asset record
sbom.json	CycloneDX SBOM: the app, the bundled wheels, igv.js (with its file hash and the libraries it names in licence banners: DOMPurify, vanilla-picker, pako, Canvas 2 Svg) and the asset (record ID, its Python packages, and the Chromium and Node.js versions their Playwright pins). It is marked incomplete: it has no dependency graph, and does not list igv.js's CRAM bundle, native libraries vendored inside wheels (such as NumPy's OpenBLAS), or the asset's Ubuntu libraries and venv pip (#13)

Accepted exceptions live in .trivyignore.yaml. Each entry names its governance record (GOVR hazard) and has an expired_at date; once it passes, the release and weekly scans fail again. Current entries: two DOMPurify advisories in the igv.js bundle, accepted until 2027-02-04 (GOVR-14; upgrade in #14).

What is checked for known vulnerabilities: the bundled wheels and the asset's Python packages (pip-audit, and Trivy through the SBOM), and the libraries igv.js bundles (Trivy, as npm packages in the SBOM). Chromium and Node.js are listed with versions but are not matched against a vulnerability database; they change only when the asset's Playwright pin changes.

The release workflow runs after the GitHub release is published, which is after sign-off, as in the template repo. The evidence the Approver signs off on comes earlier: the CI run on the release branch at the tested commit, which runs the same audits and produces the same SBOM.

To run the lint job locally:

uvx ruff@0.17.3 check resources tests scripts
uvx zizmor@1.31.1 --offline --strict-collection .github/workflows/
uvx --from shellcheck-py==0.11.0.1 shellcheck src/code.sh infra/*.sh resources/usr/bin/mark-section resources/usr/bin/mark-success

---

Code walkthrough

A 4-part annotated HTML walkthrough of purple_plotter.py is in docs/. It describes v1.0.0: it predates the inlined igv.js, the JSON data block and the CSP (see Security).

Part	File	Contents
1	260623_eggd_purple_plotter_1.html	High-level outline — phases, data flow, function inventory
2	260623_eggd_purple_plotter_2.html	Key concepts — dataclasses, pandas, tarfile, log₂, BEDGraph/SEG, blob URLs
3	260623_eggd_purple_plotter_3.html	Function walkthrough — all 16 functions
4	260623_eggd_purple_plotter_4.html	Line-by-line reference — 5 hardest functions annotated

---

Platform

Item	Value
Org	org-emee_1
Region	aws:eu-central-1
Ubuntu	24.04
Instance	mem1_ssd1_v2_x2
igv.js	3.8.3 (embedded)
Genome build	hg38