eukan: Eukaryotic Genome Annotation Pipeline

A comprehensive annotation pipeline tailored for eukaryotic genomes, particularly those from less well-studied organisms.

Installation

Currently, Eukan installation is only supported via Docker and Conda.

CPU requirement: the prebuilt bioconda tool binaries target x86-64-v3 (AVX2/BMI2/FMA — Intel Haswell / AMD Excavator or newer). On an older CPU they abort with SIGILL (illegal instruction), which eukan check detects and explains. Running on a pre-AVX2 host requires pinning the affected tools back to their x86-64-v2 builds (conda_pin in eukan/data/tools.toml) and rebuilding Trinity via scripts/fix-trinity-avx2.sh.

Docker

The Docker image installs all external tools via conda (from the same environment.yml used for local installs), then builds fitild and the spaln --spsp utilities from source, installs the pinned combinr release binary, and optionally includes GeneMark. The build needs network access and only the repository checkout (plus the optional GeneMark files below); there are no build arguments to set. The image needs an x86-64-v3 CPU (see the note above).

git clone https://github.com/BFL-lab/eukan.git
cd eukan
docker build -t eukan -f docker/Dockerfile .

To include GeneMark (license required), place gmes_linux_64*.tar.gz and gm_key_64.gz in the project root before building; the build fails if the archive is there without its key. If both are omitted, the build will succeed but eukan check will report GeneMark as missing.

The GeneMark key expires. When ~/.gm_key exists on the host, eukan-docker mounts it read-only over the key baked into the image, and GeneMark uses the host key instead. A renewed key then needs no rebuild (gunzip -c gm_key_64.gz > ~/.gm_key), but the host key must be kept current: an expired ~/.gm_key breaks GeneMark even in a freshly built image with a valid key. Without ~/.gm_key the baked key is used.

The image creates a user with UID/GID 1000 whose home is /home/eukan; eukan-docker runs as your own UID/GID. If yours differ, build with --build-arg USER_ID=$(id -u) --build-arg GROUP_ID=$(id -g) so the container's home is writable by you.

A separate development image (built on top of the eukan image) adds test tooling (NCBI datasets CLI, SRA Toolkit, procps):

docker build -t eukan-dev -f docker/Dockerfile.dev .

Conda

Installs all external tools via bioconda and eukan itself via pip, in one step:

git clone https://github.com/BFL-lab/eukan.git
cd eukan
conda config --set channel_priority strict   # bioconda requirement (see note below)
conda env create -f environment.yml
conda activate eukan
eukan check

Strict channel priority is required. Bioconda must be solved with conda-forge first and strict priority (the channels: order in environment.yml is already correct). Without strict priority the solver mixes builds across channels and produces inconsistent runtime dependencies — e.g. an R stringi linked against an icu version not present in the env, which fails a bioconductor post-link script. If you don't want to change your global config, prefix the create instead: CONDA_CHANNEL_PRIORITY=strict conda env create -f environment.yml.

The eukan CLI configures all required environment variables (e.g. $ZOE, $ALN_TAB) automatically at startup. If you need to run the underlying tools directly outside of eukan, install the optional activation script:

mkdir -p $CONDA_PREFIX/etc/conda/activate.d
cp conda-activate.sh $CONDA_PREFIX/etc/conda/activate.d/eukan.sh
conda deactivate && conda activate eukan

The environment.yml is auto-generated from eukan/data/tools.toml. To regenerate after modifying tool versions: python scripts/generate-env.py.

A few tools aren't on conda and are installed by a helper script after creating the environment: it builds fitild from source, downloads the pinned combinr release binary, and installs GeneMark if its (license-gated) archive is present.

# Download GeneMark (license required) from:
#   https://topaz.gatech.edu/GeneMark/license_download.cgi
# Place gmes_linux_64_4.tar.gz and gm_key_64.gz in the repository root, then:
./scripts/install-extras.sh

# Or install only some of fitild, combinr and genemark:
./scripts/install-extras.sh combinr

eukan check will tell you exactly what's missing and how to install it.

Local development

Requires Python >= 3.11 and Poetry. External tools must be installed separately (via conda or Docker):

poetry install --with dev   # omit --with dev for a runtime-only install
poetry run eukan --help

The dev group adds pytest, ruff, and mypy.

Dependencies

Python (managed by Poetry): click, click-option-group, gffutils, biopython, requests, pydantic-settings, pyhmmer, scipy, pysam.

External tools (via Docker image or conda): AUGUSTUS, SNAP, CodingQuarry, spaln, GenomeThreader, Trinity, minimap2, samtools, diamond, jellyfish, TRF, RepeatModeler, RepeatMasker, rmblast, table2asn.

Manual install:

GeneMark-ES/ET/EP+: license required
combinr: pre-built release binary from github.com/BFL-lab/combinr/releases (installed automatically by Docker and scripts/install-extras.sh)
fitild: github.com/ogotoh/fitild (only needed for spaln-based protein alignment in default fitild mode)
spaln analysis utilities (utn, npssm, exinpot, etc.) and their Perl scripts: built from spaln source (only needed for --spsp species-specific parameter mode). Build only the utilities, as the Docker image does: make all also builds fitild & co., which link gsl from /usr/local/lib and fail. The explicit CFLAGS replace configure's default -march=native and add the env's gsl headers:
```
git clone --depth 1 https://github.com/ogotoh/spaln && cd spaln/src
CFLAGS="-O3 -I$CONDA_PREFIX/include" ./configure
make utn dvn exinpot npssm kmers rdn eij2ild
cp utn dvn exinpot npssm kmers rdn eij2ild "$CONDA_PREFIX/bin/"
cp ../perl/*.pl ../perl/*.pm "$CONDA_PREFIX/share/spaln/alndbs/"
chmod +x "$CONDA_PREFIX"/share/spaln/alndbs/*.pl
```

Usage

Docker

Use eukan-docker as a wrapper to run any subcommand inside the Docker container. It bind-mounts the current directory (at the same path) and runs as your user, so every input file must live under the current directory; paths outside it are not visible in the container.

The typical workflow runs each subcommand from the same working directory. Each subcommand writes into its own subdirectory under that working directory (repeats/, assemble/, annotate/, func-annot/, submission/), and downstream subcommands auto-discover outputs from earlier ones via the cross-pipeline artifact registry — no need to pass paths between steps.

# 1. Download reference databases
./eukan-docker db-fetch

# 2. Soft-mask repeats (optional but recommended for AUGUSTUS)
./eukan-docker mask-repeats -g genome.fasta

# 3. Assemble transcriptome from RNA-seq reads (optional but recommended)
./eukan-docker assemble \
    -g genome.fasta \
    -l left_reads.fastq -r right_reads.fastq \
    -S RF

# 4. Annotate:  auto-discovers assembly outputs, repeat hints, and databases
./eukan-docker annotate -g genome.fasta -p proteins.fasta --kingdom protist

# 5. Functional annotation:  extract the predicted proteins, then annotate them
#    (the databases from step 1 are found in databases/)
./eukan-docker gff3toseq -g genome.fasta -i annotate/final.gff3 --output-format protein -o proteins.faa
./eukan-docker func-annot -p proteins.faa --gff3 annotate/final.gff3

# 6. NCBI submission prep (table2asn validator + .sqn)
./eukan-docker prep-submission -t template.sbt --organism "Genus species"

# Extract sequences from GFF3
./eukan-docker gff3toseq -g genome.fasta -i genes.gff3 --output-format protein -o proteins.faa

Each auto-discovered input can be overridden explicitly (see subcommand docs below).

Set EUKAN_IMAGE to use a custom image name (default: eukan):

EUKAN_IMAGE=myregistry/eukan:latest ./eukan-docker annotate ...

Local (development)

poetry run eukan annotate -g genome.fasta -p proteins.fasta --kingdom fungus
poetry run eukan assemble -g genome.fasta -l left.fq -r right.fq -S RF

Subcommands

Settings can also come from environment variables: EUKAN_<SETTING> for annotate (e.g. EUKAN_NUM_CPU=8, EUKAN_GENETIC_CODE=6), EUKAN_ASSEMBLE_<SETTING> for assemble (e.g. EUKAN_ASSEMBLE_MAX_INTRON_LEN=2000), and EUKAN_REPEATS_, EUKAN_FUNC_ and EUKAN_SUBMIT_ for mask-repeats, func-annot and prep-submission. <SETTING> is the field name in eukan/settings.py, which can differ from the flag (--max-intron sets max_intron_len, --code sets genetic_code). A flag given on the command line wins over its variable, and the variable wins over the default --help shows.

`eukan annotate`

Run the genome annotation pipeline. When run in the same directory as eukan assemble, transcript evidence (FASTA, GFF3, RNA-seq hints) is discovered automatically. UTRs and alternative isoforms are folded in from the transcript evidence by the combinr consensus engine.

Usage: eukan annotate [OPTIONS]

Required input:
  -g, --genome PATH               Genome FASTA with no lower-case nucleotides (the pipeline
                                  soft-masks repeats by converting to lower-case). [required]
  -p, --proteins PATH             One or more protein FASTA files. [required]

Pipeline parameters:
  -k, --kingdom [fungus|protist|animal|plant]
                                  Target organism kingdom (tunes predictor parameters).
  -n, --numcpu INTEGER            Number of CPU threads. [default: all]
  -w, --weights INTEGER           Weights for evidence sources: protein, gene predictions,
                                  transcripts. [default: 2, 1, 3]
  --combinr-stringent-overlap FLOAT
                                  combinr --stringent-overlap for the consensus --alt-splice isoform
                                  grouping: min span overlap (% of the shorter isoform) for two
                                  isoforms to attach to one gene (0 = any overlap). [default: 0]
  --orf-patch [supported|all]     Patch transcript ORFs that overlap no consensus gene into the
                                  final models, labelled eukan-orf-patch / support=transcript_orf:
                                  'supported' only those whose CDS overlaps a protein-alignment CDS
                                  on the same strand, 'all' every one (default: off).
  -c, --code INTEGER              NCBI genetic code table number. [default: 11]

Override options:
  -tf, --transcripts-fasta PATH   Override auto-discovered transcript FASTA.
  -tg, --transcripts-gff PATH     Override auto-discovered transcript GFF3.
  -r, --rnaseq-hints PATH         Override auto-discovered RNA-seq hints GFF.
  --combinr-path FILE             Path to the combinr binary (default: 'combinr' on PATH).
  --splice-permissive             Allow rarer non-canonical splice sites in AUGUSTUS: drops the
                                  default >= 1% junction-share gate (keeps >= 10 junctions, >= 50
                                  unique reads); allows AT-AC without assembly evidence.

Experimental:
  --spsp                          Build species-specific spaln parameters from transcripts
                                  (alternative to fitild). See "Protein alignment modes" below.

Re-run steps:
  --run-genemark                  Force re-run GeneMark gene prediction.
  --run-prot-align                Force re-run protein alignment (spaln/gth).
  --run-augustus                  Force re-run AUGUSTUS training and prediction.
  --run-snap                      Force re-run SNAP (and CodingQuarry) prediction.
  --run-consensus                 Force re-run consensus model building.

Notes on `annotate` input

It's tempting to provide more protein sequence data to the pipeline with the idea of covering more of the genome, but typically this leads to a degradation in performance at worst, or strongly diminishing returns at best. This is mostly due to the presence of the same orthologs, paralogs and derived sequences across multiple different species within the database. The further those sequences are to the target genome, lower the useful signal in aligning them. If can often lead to artifacts or misleading evidence that can interfere with the consensus building process, unless down-weighted. It's more ideal to find ten proteomes of the closest neighboring species available, otherwise the UniProt-SwissProt verified collection is a good alternative.

`eukan assemble`

Assemble transcriptome from RNA-seq reads for use with eukan annotate. Provide either paired-end reads (--left and --right) or single-end reads (--single).

Usage: eukan assemble [OPTIONS]

Required input:
  -g, --genome PATH               Genome FASTA. [required]
  -l, --left PATH                 Left paired-end reads.
  -r, --right PATH                Right paired-end reads.
  -s, --single PATH               Single-end reads.

Pipeline parameters:
  -n, --numcpu INTEGER            Number of CPU threads. [default: all]
  -S, --strand-specific [RF|FR|R|F]
                                  Strand-specific library type (RF/FR for paired, R/F for single).
  --non-canonical [auto|force|off]
                                  Non-canonical splice re-mapping with minimap2 (-J 0 -C 3
                                  --splice-flank=no): 'auto' when the soft-clip diagnostic finds
                                  extensive non-canonical splicing, 'force' always, 'off' never.
                                  [default: auto]
  --diagnose-softclips / --no-diagnose-softclips
                                  Run the soft-clip + intron BAM diagnostic after read mapping
                                  (trans-splicing, non-canonical splicing; gates --non-canonical
                                  auto). [default: on]
  -c, --code INTEGER              NCBI genetic code table number. [default: 1]
  -m, --min-intron INTEGER        Minimum intron length. [default: 20]
  -M, --max-intron INTEGER        Maximum intron length, hard-imposed: minimap2 maps with it (-G),
                                  Trinity genome-guided uses it, models are split at longer introns
                                  and combinr enforces it (0 disables the split). Changing it on a
                                  resumed run re-runs them automatically. [default: 5000]
  -j, --jaccard-clip / --no-jaccard-clip
                                  In-house jaccard clipping of fused Trinity transcripts (needs
                                  paired reads). [default: on]
  --jaccard-greediness FLOAT      Jaccard low-coverage sensitivity (0 = Trinity's fixed floor).
                                  [default: 1.5]
  --jaccard-max-trough FLOAT      Jaccard trough-depth gate; lower = more stringent. [default: 0.05]
  --jaccard-min-delta FLOAT       Jaccard flanking-hill rise; higher = more stringent.
                                  [default: 0.35]
  --jaccard-max-adaptive-trough FLOAT
                                  Ceiling on the coverage-adaptive trough gate. [default: 0.30]
  --combinr-stringent-overlap FLOAT
                                  combinr --stringent-overlap: min span overlap (% of the shorter
                                  transcript) to cluster (0 = any overlap). [default: 0]
  --sl-sequence TEXT              Override the spliced-leader sequence for trans-splice acceptor
                                  detection.
  --sl-cluster-window INTEGER     Genomic window (bp) for consolidating SL acceptor sites.
                                  [default: 5]
  --adapter-sequence TEXT         Extra adapter sequence to exclude from SL detection (repeatable).
  --sl-adapter-filter / --no-sl-adapter-filter
                                  Screen SL candidates against known sequencing adapters.
                                  [default: on]
  --combinr-path PATH             Path to the combinr binary (else PATH or
                                  $EUKAN_ASSEMBLE_COMBINR_PATH).
  --uniprot PATH                  SwissProt FASTA or diamond .dmnd: enables homology splice-strand
                                  correction (unstranded libraries) and --defuse.
  --defuse                        Split chimeric transcripts with >=2 distinct SwissProt hits
                                  (requires --uniprot).
  --defuse-overlap-tolerance FLOAT
                                  Max fractional overlap for two hits to count as distinct.
                                  [default: 0.10]
  --memory-gb INTEGER             Assembly memory cap in GiB. [default: 60% of available, min 4]

Re-run steps:
  -A, --run-minimap2              Force re-run minimap2 read mapping.
  -T, --run-trinity               Force re-run Trinity de novo + genome-guided assembly.
  --run-jaccard                   Force re-run jaccard clipping of fused transcripts.
  --run-map-transcripts           Force re-run minimap2 transcript→genome mapping.
  --run-strand-correct            Force re-run homology-based splice-strand correction.
  --run-defuse                    Force re-run homology-based transcript de-fusion.
  --run-max-intron-split          Force re-run the max-intron split of transcript models.
  --run-sl-detect                 Force re-run SL trans-splice acceptor detection.
  --run-sl-cut                    Force re-run the genomic SL cut of transcript models.
  --run-combinr                   Force re-run combinr transcript consolidation.
  -f, --force                     Force re-run all steps (ignore cached outputs).

The pipeline runs minimap2 read mapping, genome-guided + de novo Trinity assembly, jaccard clipping of fused transcripts, minimap2 transcript→genome mapping, homology splice-strand correction (with --uniprot) and de-fusion (with --defuse), the max-intron split, spliced-leader acceptor detection and cut, and combinr transcript consolidation. The read-mapping step also profiles splice site types from junction evidence (splice_site_summary.json), which the annotation pipeline uses to allow non-canonical splice sites in AUGUSTUS. With no step flags, every enabled step runs and steps already completed in the work dir are skipped. A step flag (-A/--run-minimap2, -T/--run-trinity, --run-jaccard, --run-map-transcripts, --run-strand-correct, --run-defuse, --run-max-intron-split, --run-sl-detect, --run-sl-cut, --run-combinr) re-runs only that step plus every downstream step that consumes its output; -f alone re-runs every step.

Soft-clip + intron BAM diagnostic

Immediately after minimap2 read mapping, eukan assemble walks the sorted BAM once to produce softclip_diagnostic_summary.json. It surfaces two signals downstream gene prediction needs to know about:

Trans-splicing — soft-clip ends are clustered by a K-bp substring anchored at the alignment boundary (K=12 by default), with Hamming-tolerant per-locus consolidation and cross-locus canonicalization. A few clusters supported by many genomic loci is the trans-splicing fingerprint (the splice-leader sequence sits on the 5' end of every trans-spliced transcript). The top non-low-complexity cluster's K-bp anchor key is reported alongside a per-column majority-vote consensus motif that extends past the anchor — for kinetoplastid datasets this typically recovers ~30–40 bp of the conserved SL. The verdict is categorical (STRONG / MODERATE / ABSENT) and a WARNING is logged when present so the user knows reads may need splice-leader trimming before annotation.
Non-canonical splice usage — the per-N-cigar-op donor/acceptor dinucleotide histogram is rolled up into a canonical_pct plus a top non-canonical pair. Verdict (EXTENSIVE / MODERATE / ABSENT) tracks the prevalence; EXTENSIVE is the signal that the genome warrants --splice-permissive on eukan annotate.

The diagnostic runs once per read mapping: the non-canonical re-map within a run reuses its summary, and a re-run of the read-mapping step rebuilds it. It adds ~30s per 100M reads on top of minimap2's runtime. Tune the consensus stringency with consensus_min_majority_fraction on diagnose_bam() when invoking the library directly; defaults are tuned for whole-genome RNA-seq.

`eukan mask-repeats`

Soft-mask repeats with RepeatModeler + RepeatMasker. The de novo families library is inferred by RepeatModeler, then RepeatMasker uses it to lower-case repetitive bases in the genome.

Usage: eukan mask-repeats [OPTIONS]

Required input:
  -g, --genome PATH               Genome FASTA. [required]

Pipeline parameters:
  -n, --numcpu INTEGER            Number of CPU threads. [default: all]
  --engine [rmblast|ncbi]         Search engine for BuildDatabase / RepeatMasker. [default: rmblast]
  --lib PATH                      Pre-built repeat-family library FASTA. When set, RepeatModeler is
                                  skipped.

Re-run steps:
  --run-modeler                   Force re-run BuildDatabase + RepeatModeler (also re-runs
                                  RepeatMasker).
  --run-masker                    Force re-run RepeatMasker.
  -f, --force                     Force re-run all steps (ignore cached outputs).

--run-modeler also re-runs RepeatMasker, which masks with the rebuilt library. Without step flags, RepeatMasker re-runs when its library (a rebuilt one, or another --lib) or the genome has changed since it last ran.

Outputs (in the working directory):

<stem>.masked.fasta — soft-masked genome (lower-case in repeats).
<stem>.repeats.gff — raw RepeatMasker GFF.
hints_repeatmask.gff — AUGUSTUS-format nonexonpart/src=RM hints.

Pass the masked genome to the next stage (eukan annotate -g <stem>.masked.fasta); AUGUSTUS auto-discovers hints_repeatmask.gff from the working directory and weights it via the RM extrinsic source already declared in the shipped augustus.config.

`eukan func-annot`

Add functional annotations to predicted proteins. The proteins are not discovered automatically: extract them from annotate/final.gff3 with eukan gff3toseq and pass them with --proteins, and pass --gff3 annotate/final.gff3 to get the annotated func-annot/final.mod.gff3 that eukan prep-submission picks up. The reference databases default to where eukan db-fetch puts them (databases/ in the current directory).

Two homology sources are supported (selected with --homology-db); Pfam is searched in both modes:

--homology-db uniprot (default) — phmmer against UniProt-SwissProt. Broad coverage of curated proteins; emits product=<description> and inference=similar to AA sequence:UniProtKB:<accession>.
--homology-db kofam — KEGG Orthology assignment. hmmscan against the eukaryote subset of the KOfam HMM database, with per-KO bit-score thresholds (full or domain score depending on the KO) loaded from ko_list. Only KOs whose hit score meets the curated cutoff drive product/EC assignment; below-threshold hits are recorded but not promoted. Emits product=<KO definition>, ec_number=<EC> (parsed out of [EC:…] tags in the definition), Dbxref=KEGG:K<number>, and inference=protein motif:KOFAM:K<number>.

KOfam mode is adapted from KofamKOALA: the database, per-KO thresholds, eukaryote.hal filter, and full-vs-domain adaptive scoring all come from the KofamKOALA paper. If you use --homology-db kofam, please cite: Aramaki T, Blanc-Mathieu R, Endo H, Ohkubo K, Kanehisa M, Goto S, Ogata H. KofamKOALA: KEGG ortholog assignment based on profile HMM and adaptive score threshold. Bioinformatics. 2020 Apr 1;36(7):2251–2252. doi:10.1093/bioinformatics/btz859.

Usage: eukan func-annot [OPTIONS]

Pipeline parameters:
  -n, --numcpu INTEGER            Number of CPU threads. [default: all]
  --homology-db [uniprot|kofam]   Homology source: 'uniprot' runs phmmer vs SwissProt; 'kofam' runs
                                  hmmscan vs KOfam with per-KO bit-score thresholds. Pfam hmmscan
                                  runs in both modes. [default: uniprot]
  -e, --evalue TEXT               E-value cutoff. [default: 1e-1]

Override options:
  -p, --proteins PATH             Amino acid sequences in FASTA format.
  --uniprot PATH                  UniProt-SwissProt database FASTA.
  --kofam PATH                    KOfam pressed HMM database.
  --ko-list PATH                  KOfam ko_list TSV (per-KO thresholds + definitions).
  --pfam PATH                     Pfam HMM database.
  --gff3 PATH                     GFF3 file to annotate with functional info.
  -f, --force                     Force re-run all steps (ignore cached outputs).

Runs the chosen homology search plus hmmscan against Pfam (both via pyhmmer). Writes into func-annot/, never next to the inputs:

<proteins stem>.mod<extension>: annotated FASTA with functional descriptions in headers (--proteins proteins.faa gives func-annot/proteins.mod.faa).
<gff3 stem>.mod<extension>: (if --gff3 provided) GFF3 with product/inference (and, for KOfam, Dbxref/ec_number) attributes.
<proteins stem>.phmmer.search.json (.kofam.search.json in KOfam mode): the cached search hits, so a re-run on the same proteome skips the search.

For UniProt mode, hits with e-values between 1e-3 and the cutoff are reported as marginal. For KOfam mode, hits below the per-KO bit-score threshold are tagged [marginal KO hit] in the FASTA description and do not contribute to the GFF3 product/Dbxref/inference lines — the KofamKOALA paper's evaluation showed that the adaptive thresholds yield substantially better precision/recall than a single global E-value cutoff.

`eukan prep-submission`

Validate and package an annotated genome for NCBI submission. Wraps NCBI's table2asn with the standard recipe (-split-logs -W -J -Z -euk -T -V b plus -c/-M/-a), producing a .sqn upload-ready file alongside .val, .dr, and .stats validator reports for iterative GFF3 refinement. When run after eukan annotate and eukan func-annot, the genome eukan annotate ran on (from eukan-run.json) and annotated GFF3 (final.mod.gff3, falling back to final.gff3) are auto-discovered.

Usage: eukan prep-submission [OPTIONS]

Required input:
  -t, --template PATH             NCBI submission template (.sbt). [required]

Source qualifiers:
  --organism TEXT                 Organism scientific name (e.g. 'Homo sapiens'). Required unless
                                  --source-info is given.
  --isolate TEXT                  Isolate / strain identifier (optional).
  --source-info TEXT              Raw -j string for table2asn (e.g. '[organism=Foo] [isolate=Bar]
                                  [country=Canada]'). Overrides --organism / --isolate when set.
  --locus-tag-prefix TEXT         NCBI-registered locus tag prefix (required for new-genome
                                  submissions).

Override options:
  -g, --genome PATH               Genome FASTA. Auto-discovered from eukan-run.json when omitted.
  -i, --gff3 PATH                 Annotated GFF3. Defaults to final.mod.gff3 (or final.gff3) in the
                                  working directory.

Pipeline parameters:
  --cleanup TEXT                  table2asn -c cleanup flags. [default: befw]
  --mode TEXT                     table2asn -M flatfile mode. [default: n]
  -a, --assembly-type TEXT        table2asn -a assembly type / gap configuration. [default: r10k]
  --extra-args TEXT               Extra table2asn arguments, shell-quoted (e.g. --extra-args
                                  '-split-dr -huge').
  --cleanup-gff3 / --no-cleanup-gff3
                                  Pre-process the GFF3 (strip UniProt cruft, drop CDS-less mRNAs,
                                  cap inferences) before handing it to table2asn. [default: on]

Output options:
  -o, --output-file PATH          Output .sqn path. [default: <output-dir>/<genome-stem>.sqn]
  -d, --output-dir PATH           Output directory for .sqn and validator reports.
                                  [default: ./submission]
  --print-command                 Print the resolved table2asn command and exit (no outputs
                                  written).
  --dry-run                       Print the command and create the output directory, but don't run
                                  table2asn.

The .sbt submission template must be created via NCBI's web form — it is the only mandatory file input. Outputs land in ./submission/:

<genome-stem>.sqn — GenBank-format submission file ready for upload.
<genome-stem>.val — validator report (FATAL/ERROR/WARNING/INFO counts are logged on completion).
<genome-stem>.dr — discrepancy report.
<genome-stem>.stats — feature statistics.
<gff3-stem>.cleaned.gff3 — the post-cleanup GFF3 actually fed to table2asn (when --cleanup-gff3 is enabled).

By default the GFF3 is pre-processed before table2asn sees it: UniProt-style metadata (OS=...OX=...GN=...PE=...SV=...) is stripped from product= values (without this, table2asn rewrites every product to "hypothetical protein", losing all functional annotation), (Fragment) suffixes are removed, mRNAs with no CDS children are dropped (along with newly-orphaned genes), and inference= lists are capped at three accessions per feature. Pass --no-cleanup-gff3 to bypass.

The intended workflow is iterative:

# First pass — typical errors surface as ERROR/WARNING in the .val report.
eukan prep-submission -t template.sbt --organism "Genus species"

# Inspect submission/<genome-stem>.val and .dr, refine the GFF3, re-run.
# Use --print-command to inspect the exact table2asn invocation without running it:
eukan prep-submission -t template.sbt --organism "Genus species" --print-command

The command exits non-zero if table2asn reports any FATAL validation errors; the .val report path is included in the error message.

`eukan gff3toseq`

Extract protein or cDNA sequences from a GFF3 + genome.

Usage: eukan gff3toseq [OPTIONS]

Required input:
  -g, --genome PATH               Genome FASTA. [required]
  -i, --gff3 PATH                 GFF3 with gene models. [required]

Pipeline parameters:
  --output-format [protein|cdna]  Output sequence type. [default: protein]
  -c, --code INTEGER              NCBI genetic code table number. [default: 1]

Output options:
  -o, --output-file FILENAME      Write FASTA to this file ('-' for stdout). [default: stdout]

`eukan db-fetch`

Download reference databases. Without -d, Pfam is always fetched; the homology DB to pair it with is chosen by --homology-db.

Usage: eukan db-fetch [OPTIONS]

Pipeline parameters:
  -o, --output-dir PATH           Directory to download databases into. [default: databases]
  --homology-db [uniprot|kofam]   Homology DB to fetch alongside Pfam: 'uniprot' downloads
                                  SwissProt; 'kofam' downloads the KOfam HMM profiles + ko_list and
                                  presses a eukaryote-only HMM database. [default: uniprot]
  -d, --database [uniprot|pfam|kofam|ko_list]
                                  Specific database(s) to fetch. Overrides --homology-db when given.
  -f, --force                     Re-download even if databases are up to date.

Downloads and prepares:

Pfam-A.hmm — Pfam HMM profiles (decompressed and pressed for hmmscan). Always fetched without -d.
uniprot_sprot.faa — UniProt-SwissProt protein sequences (--homology-db uniprot, default).
kofam_eukaryote.hmm + ko_list.tsv — KOfam profiles and per-KO bit-score thresholds (--homology-db kofam). The fetcher downloads KofamKOALA's profiles.tar.gz and ko_list.gz from https://www.genome.jp/ftp/db/kofam/, filters profiles via the shipped eukaryote.hal list (~16k of ~27k KOs), concatenates them into a single HMM file, and presses it. ko_list is the per-KO metadata table used by func-annot --homology-db kofam; see the eukan func-annot section for the KofamKOALA citation.

Each database has an entry in databases/.manifest.json (md5 + source URL + download date); re-running eukan db-fetch is a no-op when files match the manifest. Use -d <name> for a targeted refresh of a single file (e.g. -d pfam to refresh only Pfam without touching the homology DB).

`eukan compare`

Compare predicted gene models against a reference or previous annotation to assess annotation quality. Reports gene-level classification (exact, inexact, missing, merged, fragmented, novel), subfeature-level metrics (mRNA, CDS, intron), and overlap-based sensitivity/specificity/F1 scores. Repeat --predicted to compare several predictions against the same reference and tabulate, per gene-level classification, which subsets of predictions agreed.

Usage: eukan compare [OPTIONS]

Required input:
  -r, --reference PATH            Reference GFF3 file. [required]
  -p, --predicted PATH            Predicted GFF3 file. Repeat to compare multiple predictions
                                  against the same reference. [required]

Pipeline parameters:
  -L, --label TEXT                Short label per --predicted (must appear once per --predicted, or
                                  be omitted to use file stems).

Output options:
  -o, --output-file PATH          Write per-feature details to a TSV file. In multi-prediction mode
                                  a leading 'prediction' column is prepended.

The classification system and metrics are further described in the paper referenced in Citation. Gene-level classifications:

exact: prediction coordinates match reference exactly.
inexact: prediction overlaps a single reference with boundary differences.
missing: reference gene with no overlapping prediction (false negative).
merged: prediction spans 2+ reference genes.
fragmented: 2+ predictions each cover a single reference gene.
novel: prediction with no reference overlap (possibly false positive).

For matched features, reports overlap-based sensitivity (overlap / reference length), specificity (overlap / prediction length), and F1 score. Boundary differences (5' and 3') are reported for inexact matches.

# Single prediction
eukan compare -r reference.gff3 -p predicted.gff3

# Single prediction with per-feature TSV
eukan compare -r reference.gff3 -p predicted.gff3 -o details.tsv

# Multiple predictions against the same reference
eukan compare -r reference.gff3 \
    -p eukan.gff3 -p braker.gff3 -p maker.gff3 \
    -L Eukan -L Braker -L Maker \
    -o details.tsv

Multi-prediction mode

When two or more --predicted inputs are supplied, the report adds a comparative summary on top of the per-prediction breakdowns:

F1 by level / prediction — count-based F1 at gene, mRNA, CDS, and intron level for each prediction, side by side.
Per-class powerset — for each gene-level classification (match = exact|inexact, missing, merged, fragmented), and for each reference gene, records the sorted tuple of prediction labels whose classification of that gene was that class, then tallies these tuples. Counts within a class sum to the number of reference genes; the (none) row counts genes no prediction assigned to that class. Full enumeration up to N=6 predictions; condensed (shared-by-all / shared-by-none / uniquely-per-pred) above.

--label defaults to each prediction's filename stem; if two predictions share a stem they are auto-numbered with a warning. Use --label to set explicit names; counts must match --predicted.

The N=1 path is unchanged from earlier releases — the comparative section is omitted entirely and the per-feature TSV keeps the legacy schema.

`eukan check`

Verify Python dependencies, external tools, and databases.

Usage: eukan check [OPTIONS]

Pipeline parameters:
  --for [annotate|assemble|func-annot|db-fetch|mask-repeats|prep-submission]
                                  Only check tools needed by these subcommands. If omitted, check
                                  all.
  --db-dir PATH                   Database directory to check. [default: databases]
  --homology-db [uniprot|kofam]   When set, only check the chosen homology DB (plus Pfam). Without
                                  it, every registered database is checked.

Checks Python dependencies, probes each external tool with a version/help command (and runs GeneMark's own install check, which catches a missing or expired license key), and verifies database integrity. Exits 0 if all checks pass, 1 if any fail.

# Check everything
eukan check

# Check only what's needed for annotation
eukan check --for annotate

# Check multiple subcommands
eukan check --for annotate --for assemble

Example output:

Checked 14 external tools:

  12 tools OK:
    ✓ samtools                       samtools 1.20
    ✓ AUGUSTUS                        AUGUSTUS (3.5.0)
    ...

  2 tools MISSING or BROKEN:
    ✗ codingquarry                   `CodingQuarry` not found on PATH; env not set: $QUARRY_PATH
      used by: annotate
    ✗ fitild                         `fitild` not found on PATH
      used by: annotate
      hint: Build from source: git clone https://github.com/ogotoh/fitild ...

Pipeline Overview

The annotation pipeline (eukan annotate) runs the following steps:

ORF finding: Identify ORFs in transcript assemblies (if provided). Uses the configured genetic code (-c/--code) so that alternative stop codons (e.g., code 6 where TAA/TAG encode glutamine) are handled correctly.
GeneMark: Ab initio gene prediction (ES mode, or ET mode if RNA-seq intron hints are available with >= 150 introns). Passes --gcode for non-standard genetic codes (codes 6 and 26).
Protein alignment: Spliced alignment via spaln (intron-rich genomes, > 25% introns/gene) or GenomeThreader (intron-poor). See Protein alignment modes.
AUGUSTUS: Train species-specific parameters from concordant GeneMark/protein models, then predict genes using protein + RNA-seq hints. Non-canonical splice sites (e.g., AT-AC) are allowed automatically when supported by sufficient junction evidence from the minimap2 read alignments (>= 10 junctions, >= 50 unique reads, and >= 1% of all junctions); --splice-permissive drops the 1% share requirement so rarer types pass.
SNAP: Train and predict (all kingdoms). CodingQuarry also runs for fungus/protist genomes.
combinr consensus: Build weighted consensus gene models from all evidence sources, folding in UTRs and alternative isoforms from the transcript evidence in a single pass. Next to the models, consensus_models/combinr.evidence_report.txt lists for each gene the evidence supporting every exon and intron (EVM .evm.out style); it uses combinr's gene IDs from consensus_models/consensus_models.gff3, which final.gff3 replaces with locus tags.
Final output: Assign sequential locus tags and correct CDS phases. Column 2 of final.gff3 is eukan. With --orf-patch, transcript ORFs that overlap no consensus gene on either strand are also patched into the final model set: under supported only those whose CDS overlaps a CDS of the protein alignment on the same strand, under all every one. Their rows say eukan-orf-patch in column 2 and their mRNAs carry support=transcript_orf (the consensus mRNAs carry combinr's support=consensus). The patch is off by default: the patched genes have no ab initio support, and under all no homology support either.

Output: final.gff3 in the working directory.

Protein alignment modes

spaln protein alignment supports two modes for modeling intron structure:

Default (fitild): Builds an intron length distribution from GeneMark predictions and feeds it to spaln via the -yI parameter. This is the established approach and requires the fitild tool.

Species-specific parameters (--spsp): Uses transcript data to build full species-specific spaln parameters (splice site models, intron potential, codon usage) via spaln's make_eij.pl and make_ssp.pl scripts. This produces richer alignment parameters than fitild alone, at the cost of additional computation. Requires transcript data (from eukan assemble or provided via --transcripts-fasta).

# Default mode (fitild)
eukan annotate -g genome.fasta -p proteins.fasta

# Species-specific parameter mode (experimental)
eukan annotate -g genome.fasta -p proteins.fasta --spsp

When --spsp is used, protein alignment results are written to a separate directory (prot_align_ssp/) so both modes can coexist for comparison.

Run tracking and resume

All pipelines write to a shared eukan-run.json manifest in the working directory, tracking per-step status, timing, and output checksums, and for each pipeline its latest run's status, timing, CPUs, tool versions and inputs. This enables:

Resume: Re-running a subcommand skips steps that already completed.
Selective re-run: Use --run-* flags to force specific steps to re-execute.
Integrity checking: Before reusing a completed step, the pipeline checks that its output still exists, is non-empty, matches its recorded checksum (text outputs and any file up to 256 MiB) and still parses as GFF where applicable, and that the inputs and settings the step tracks are unchanged. A step that fails any check is rebuilt automatically, with a warning in the log, and a step left interrupted is cleaned up and re-run.
Progress monitoring: eukan status reads the manifest and shows a block per pipeline (its latest run's status, inputs and settings), then every step's status.

# View run status
eukan status

# Re-run only protein alignment
eukan annotate -g genome.fasta -p proteins.fasta --run-prot-align

# Re-run only combinr consolidation in assembly
eukan assemble -g genome.fasta -l left.fq -r right.fq --run-combinr

Testing

Unit tests

poetry run pytest tests/ -v

Unit tests cover GFF3 parsing/serialization, genomic interval operations, ORF finding, configuration validation, run manifest tracking, database integrity checks, and CLI wiring. They run without external tools or network access.

Pipeline integration test

A development CLI at tests/run_pipeline.py drives a full end-to-end pipeline run using S. pombe chromosome III as the test organism.

Prerequisites

All external tools installed (Docker or conda environment; verify with eukan check)
NCBI datasets CLI and SRA Toolkit on PATH (for downloading test data)

When using Docker, build and use the dev image (eukan-dev) which includes the NCBI datasets CLI:

docker build -t eukan-dev -f docker/Dockerfile.dev .

1. Download test data

python tests/run_pipeline.py setup-test-data [-o tests/data]

Downloads from NCBI:

Genome: S. pombe chromosome III (NC_003424.3, ~2.5 Mbp)
Proteins: 10 close neighbor proteomes via datasets
RNA-seq: 5 SRA paired-end runs via prefetch + fasterq-dump

Accession lists live in tests/data/*.txt and are never deleted by cleanup. Downloads are idempotent.

2. Run the pipeline

# Full run: assembly + annotation (default kingdom: fungus)
python tests/run_pipeline.py test-pipeline --kingdom fungus -n 8

# Protein-only: skip transcriptome assembly
python tests/run_pipeline.py test-pipeline --protein-only -n 8

# Custom directories
python tests/run_pipeline.py test-pipeline -d tests/data -w tests/pipeline-run

The test pipeline runs:

Transcriptome assembly: minimap2 read mapping, Trinity (genome-guided + de novo), combinr consolidation
Genome annotation: GeneMark, protein alignment (spaln/gth), AUGUSTUS, SNAP, combinr consensus

If assembly fails, it falls back to protein-only annotation automatically.

Output lands in tests/pipeline-run/ with one subdirectory per pipeline step (assemble/, annotate/, func-annot/, submission/). View run details with eukan status -d tests/pipeline-run. The end-of-run summary reports the gene/mRNA counts from final.gff3 plus the percentage of mRNAs that received a UniProt or Pfam inference.

3. Clean up

# Remove pipeline outputs only
python tests/run_pipeline.py clean-test-data

# Remove outputs + downloaded data (genome, proteins, reads)
python tests/run_pipeline.py clean-test-data --all

All three subcommands accept -h for detailed help.

Citation

If you use eukan, please cite:

Sarrasin M, Burger G, Lang BF. Eukan: a fully automated nuclear genome annotation pipeline for less studied and divergent eukaryotes. NAR Genomics and Bioinformatics. 2026 Mar;8(1):lqag003. doi:10.1093/nargab/lqag003

License

See LICENSE.md.

Name		Name	Last commit message	Last commit date
.github/workflows		.github/workflows
docker		docker
eukan		eukan
scripts		scripts
tests		tests
.dockerignore		.dockerignore
.gitignore		.gitignore
CITATION.cff		CITATION.cff
CLAUDE.md		CLAUDE.md
LICENSE.md		LICENSE.md
README.md		README.md
conda-activate.sh		conda-activate.sh
environment.yml		environment.yml
eukan-docker		eukan-docker
poetry.lock		poetry.lock
pyproject.toml		pyproject.toml

Latest commit

History

Folders and files

Repository files navigation

eukan: Eukaryotic Genome Annotation Pipeline

Installation

Docker

Conda

Local development

Dependencies

Usage

Docker

Local (development)

Subcommands

eukan annotate

Notes on annotate input

eukan assemble

Soft-clip + intron BAM diagnostic

eukan mask-repeats

eukan func-annot

eukan prep-submission

eukan gff3toseq

eukan db-fetch

eukan compare

Multi-prediction mode

eukan check

Pipeline Overview

Protein alignment modes

Run tracking and resume

Testing

Unit tests

Pipeline integration test

Prerequisites

1. Download test data

2. Run the pipeline

3. Clean up

Citation

License

About

Resources

Stars

Watchers

Forks

Releases

Packages

Used by

Contributors

Languages

`eukan annotate`

Notes on `annotate` input

`eukan assemble`

`eukan mask-repeats`

`eukan func-annot`

`eukan prep-submission`

`eukan gff3toseq`

`eukan db-fetch`

`eukan compare`

`eukan check`