WIP: Move niche dependencies to optional extras and replace removed Biopython wrappers

[Adamtaranto]

Summary

This PR slims the default install, declares missing dependencies, and fixes modules broken by recent Biopython releases. There are three commits, which can be reviewed separately.

1. Optional extras (build:)

• Core dependencies are now only what widely imported modules need: biopython, CrossMap, deap, ete4, matplotlib, more-itertools, natsort, networkx, numpy, palettable, pandas, rich, scipy and seaborn.
• Removed graphviz, genomepy and runtime cython, which nothing imports.
• Extras: assembly, aws, excel, fetch, graphviz, grabseeds, lp, ngs, ontology, projects, variation and web, combined in all. gurobi (licensed) and segment (PyTorch) stay out of all.
• Newly declared packages the code already imported: bcbio-gff, beautifulsoup4, bitarray, botocore, bx-python, cyvcf2, hmmlearn, holoviews, hvplot, matplotlib-venn, pandas, pyfaidx, pygraphviz, pyliftover, PyVCF3, scikit-learn, xlrd and xlwt.
• Missing-extra message: apps.base gains require() and missing_extra(). An action that needs a missing extra now logs pip install 'jcvi[<extra>]' and exits, instead of raising a bare ImportError.
• Tests: tests depends on jcvi[all], so CI installs everything as before. Tests that need an extra use pytest.importorskip, so a core-only install skips them.
• Contributor tools such as black, isort, hatch and pre-commit move from tests to a new dev extra.
• README documents the extras.

2. Biopython application wrappers (fix:)

Biopython 1.88 no longer ships Bio.Application or the Bio.Emboss, Bio.Phylo and Bio.Align .Applications modules. That broke imports of jcvi.apps.phylo and the annotation.reformat needle action.

• New module: jcvi.apps.commandline provides replacements for every wrapper jcvi used. They take the same keyword arguments and build the same command strings. tests/apps/test_commandline.py checks them against output from Biopython 1.85.
• Behaviour change: a tool that exits with an error is logged as a warning instead of raising. Callers already check that output files exist.
• compara.ks drops its inline fallback and uses the new module.
• apps.phylo no longer prompts for Gblocks, ffitch or TreeFix paths when imported, because the path lookups are now lazy.

3. Conda environment (fix(env):)

• conda-forge libheif 1.23.4 links against a Homebrew-only libsharpyuv path, which stops ImageMagick and Wand from loading on macOS. environment.yml now excludes that version.
• Adds tesseract, which pytesseract needs for GRABSEEDS label OCR.
• Drops libmagic, which nothing uses.

Known gaps

• jcvi.projects.str and jcvi.variation.str import pyfasta, and jcvi.utils.taxonomy imports ClientForm. Neither package has a Python 3 release, so these modules still fail to import. Fixing them needs a port, for example to pyfaidx.
• The YAML-driven tests import PyYAML, which now reaches tests only indirectly through bokeh. Consider adding PyYAML back to tests.
• The bioconda recipe may need its run requirements updated to match.

Testing

• The full suite passes locally on macOS arm64 with Python 3.12 and .[tests] installed. 277 tests passed with none skipped, including GRABSEEDS.
• A core-only install into a fresh venv imports the core modules without any optional package. It prints the install hint for actions that need an extra, and the guarded tests skip.
• Each new package resolves as a wheel on Python 3.11 and 3.13 for Linux x86_64 and macOS arm64, except the pure-Python source packages. PyVCF3, pyliftover, xlwt and bcbio-gff build and import on Python 3.12 and 3.13.
• The Biopython commit passes its tests on its own.

🤖 Generated with Claude Code

[Adamtaranto]

In progress. Doing an audit of unused dependencies and undeclared imports.

Aim is to remove unused deps and move rarely used deps off to 'optional' deps lists. Will then trim down bioconda recipe so that test builds run faster and install does not drag in unused deps.