Rewrite CANDy as an installable Python package (CLI + API) - #2
Merged
Merged
Conversation
Reimplements all 13 stages of the Colab notebook pipeline as a proper pip-installable package (src/candy) instead of Colab-only cells, with pluggable clustering (CD-HIT/MMseqs2), alignment (MAFFT), phylogenetics (FastTree), and domain-curation (manual/Gemini) backends so future tool swaps don't require another rewrite. Adds a Typer CLI (`candy run`), a SQLAlchemy-backed result database, and 79 tests covering the pure-logic modules plus mocked end-to-end pipeline runs for both input modes. While porting, several real bugs in the original notebook were found and fixed (each documented in-code and covered by a regression test): duplicated taxonomy codes corrupting characterized-enzyme organism names, a BLAST-fallback crash from state leaking across loop iterations, swapped taxonomy/organism_name columns in the SQLite schema, and a value-based list.index() bug mis-pairing domain names to positions. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…nstall Replaces the conda-only external-tool requirement with a "pip install and run" experience: FAMSA (via pyfamsa) and VeryFastTree (via veryfasttree) are now real pip dependencies with bundled binaries, and MMseqs2 is auto-downloaded and cached from the official GitHub release on first clustering run. CD-HIT/MAFFT/FastTree remain available as opt-in alternatives via the (now much smaller) conda environment for anyone who wants the original notebook's exact toolchain. Testing this for real against live binaries (not mocks) surfaced two upstream bugs, both fixed with regression tests: veryfasttree's default ext="AUTO" crashes with an access violation on Windows/AVX2 machines (now always passes an explicit ext matching the bundled binary), and MMseqs2's Windows build needs its mmseqs.bat wrapper (not mmseqs.exe directly) since its clustering workflows are internally POSIX shell scripts -- documented clearly since first use may prompt for administrator permission once, mirroring MMseqs2's own upstream guidance. Also simplifies the CLI to a single positional target (`candy GH173` / `candy sequences.fasta`, auto-detected, no `run` subcommand needed), adds CANDY_EMAIL env var support, restores jobname-collision avoidance that had been dropped from the original notebook port, and adds a --db-preference flag to reprioritize InterPro member databases without needing to spell out the full default ranking. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Manually running CANDy end-to-end against a live CAZy family (GH173) surfaced two distinct MMseqs2 failures on Windows, both fixed with regression tests and verified against a real 155-sequence clustering run: - Clustering silently produced no output when the project path contained a space (e.g. under "...\OneDrive - Org\..."): MMseqs2's Windows build routes its clustering workflow through a bundled Cygwin/busybox POSIX shell layer, which mishandles such paths. Clustering now always runs from a space-free temp directory with relative filenames. - easy-cluster's internal FASTA-conversion step (result2flat) segfaulted through that same Cygwin process-spawning layer on real data, even though every preceding clustering step succeeded. Replaced the easy-cluster convenience workflow with its constituent steps run individually (createdb -> cluster -> result2repseq -> convert2fasta), using the shell-capable mmseqs.bat only for the one step that actually needs a POSIX shell (cluster's cascaded-clustering algorithm) and the plain compiled binary directly for the rest. - mmseqs.bat always exits 0 regardless of internal failure, so success is now verified by checking the expected output file actually exists rather than trusting the return code. Also broadens NCBI/CAZy fetch retry logic beyond urllib.error.HTTPError to cover the wider class of transient connection failures (a real run hit http.client.IncompleteRead mid-fetch on a large batched Entrez request, which the narrower except-clause didn't catch). Documents the interactive domain-name curation flow and how to switch to Gemini-based curation with your own API key, and prints a citation reminder (Windels et al., PLoS One 2024) at the start of every pipeline run. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
The hardcoded gemini-2.0-flash default lost its free-tier quota, and the next hardcoded guess (gemini-2.5-flash) turned out to be restricted for new users too -- discovered via a real run, not just docs. Switch to Google's gemini-flash-latest alias, which tracks whatever their current free-tier flash model is, and add a --curation-model flag so a future rename/deprecation doesn't need a code change. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
- Add top-level LICENSE (previously only embedded in SECURITY.md, which GitHub/PyPI license detection doesn't look at) - Add project.urls and bump version off the 3.0.0.dev0 pre-release marker - Add CI: run pytest on every push/PR, and publish to PyPI on GitHub Release via trusted publishing (OIDC, no stored API token) - Archive the original notebook under archive/, update README to reflect the package as the primary distribution rather than a WIP branch Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
fetch_family_page previously retried a 404 (nonexistent family) up to 5 times against the same permanent failure before surfacing a raw requests.HTTPError traceback. Now it raises a clear ValueError immediately, and the CLI catches ValueError from run_pipeline to print a one-line "Error: ..." message and exit non-zero instead of a full traceback. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
test_cli_bare_invocation_is_candy_target_no_subcommand_needed failed on every CI matrix job (but passed locally) because Typer's Rich-rendered error panel is colorized there, and the ANSI escape codes happened to land mid-phrase at a wrapped line boundary, breaking a substring check that only stripped box-drawing characters, not color codes. Add an ANSI-stripping helper and use it for all Rich-panel-derived assertions; verified locally by forcing the same colorized rendering with FORCE_COLOR=1. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
candy-cazyme) with both a CLI (candy GH173) and a Python API, faithfully porting the original 13-stage pipeline while fixing several real bugs found via live testing against CAZy/NCBI/InterPro.pip install candy-cazymeworks with minimal external setup on Windows, macOS, and Linux; the original CD-HIT/MAFFT/FastTree conda path is still available as an opt-in alternative.archive/.Test plan
pytest— 106 passedpython -m build, installed the wheel into a clean, isolated venv, confirmedcandy --helpand the entry point work with no leftover dev-environment dependencies--treeand Gemini-based curation