v4.8.4

We are proud to release sourmash v4.8.4, now with even fewer bugs!

In addition to the patches and updates below, we would like to announce the availability of branchwater, a sourmash plugin that supports fast, low-memory, and multithreaded search, compare, and gather operations. branchwater v0.8.0 is now available and is between 10 and 100 times faster and lower memory than sourmash. (However, it does not support all the sourmash features; that will come with time!) Please give it a try!

Last but not least, we would like to ask people who use sourmash regularly to fill out this form so that we can include your information in some of our upcoming requests for funding support. Your use will not be made public but your support will help us continue to develop sourmash!

On to the release notes for sourmash v4.8.4!

Bug fixes:

• fix exponential time explosion in sig check (#2762)
• fix duplicate md5 in picklist problem (#2747)
• avoid compressing sig files in directory output twice (#2752)

Developer updates:

• Remove BIGSI and SBT code from Rust layer (#2732)
• bump version to 4.8.4-dev (#2720)
• build: updates spun off mastiff PR (#2723)
• Build: Remove deprecated system_packages from readthedocs (#2740)
• build: remove specific tooling that breaks s390x (#2725)
• Add cargo deny and bump deps now that MSRV is 1.64 (#2724)

Dependabot updates:

• Bump serde_json from 1.0.105 to 1.0.106 (#2757)
• Bump actions/checkout from 2 to 4 (#2746)
• Bump supercharge/redis-github-action from 1.6.0 to 1.7.0 (#2745)
• Bump cachix/install-nix-action from 22 to 23 (#2744)
• Bump ouroboros from 0.17.2 to 0.18.0 (#2743)
• Bump chrono from 0.4.26 to 0.4.30 (#2755)
• Bump thiserror from 1.0.47 to 1.0.48 (#2742)
• Bump thiserror from 1.0.44 to 1.0.47 (#2730)
• Bump serde_json from 1.0.104 to 1.0.105 (#2731)
• Bump actions/upload-artifact from 1 to 3 (#2729)

v4.8.3

Minor new features:

• add support for -E/--extension to sig split (#2712)
• add a sourmash.cli.parse_args([...]) function (#2691)
• update plugin support for argparse usage/description/epilog (#2696)

Cleanup and documentation updates:

• add gtdb-rs214 info (#2618, #2621, #2690)
• update JOSS paper for v4 (#1361)
• fix tutorial notebook links (#2633)
• Update URL for sourmash/fig.py (#2586)
• change matching_hashes.sig to matching-hashes.sig in documentation (#2713)
• patch lineage_summary documentation (#2693)
• switch dib-lab links to sourmash-bio links in .md files (#2588)

Developer updates:

• add upload token for codecov (#2632)
• build: fix clippy lints and MSRV check (#2680)
• maturin: move deprecated definition from Cargo.toml to pyproject.toml (#2597)
• update the core documentation building packages: Sphinx, myst-parser, and docutils (#2657)
• update the dependabot config to allow automatic rebases (#2634)
• update version to 4.8.3-dev (#2584)
• Build: bump maturin to 1.x (#2650)
• Clippy beta (1.70) fix (#2596)
• Fix clippy lint introduced in 1.71 beta (#2630)
• fix release procedure based on v4.8.0 release (#2551)
• Update docutils requirement from <0.20,>=0.17.1 to >=0.17.1,<0.21 (#2611)
• Update pytest requirement from <7.4.0,>=6.2.4 to >=6.2.4,<7.5.0 (#2659)
• Fix: update a manifest test to use temp directory (#2689)

Dependabot updates:

• Bump cachix/install-nix-action from 20 to 21 (#2625)
• Bump cachix/install-nix-action from 21 to 22 (#2651)
• Bump chrono from 0.4.24 to 0.4.25 (#2628)
• Bump chrono from 0.4.25 to 0.4.26 (#2636)
• Bump criterion from 0.3.6 to 0.5.1 (#2629)
• Bump getrandom from 0.2.9 to 0.2.10 (#2645)
• Bump log from 0.4.17 to 0.4.18 (#2623)
• Bump log from 0.4.18 to 0.4.19 (#2641)
• Bump log from 0.4.19 to 0.4.20 (#2715)
• Bump memmap2 from 0.5.10 to 0.6.1 (#2613)
• Bump memmap2 from 0.6.1 to 0.6.2 (#2626)
• Bump memmap2 from 0.6.2 to 0.7.0 (#2643)
• Bump memmap2 from 0.7.0 to 0.7.1 (#2658)
• Bump once_cell from 1.17.1 to 1.17.2 (#2622)
• Bump once_cell from 1.17.2 to 1.18.0 (#2635)
• Bump ouroboros from 0.15.6 to 0.17.0 (#2654)
• Bump ouroboros from 0.17.0 to 0.17.2 (#2687)
• Bump proptest from 1.1.0 to 1.2.0 (#2627)
• Bump pypa/cibuildwheel from 2.12.1 to 2.12.3 (#2590)
• Bump pypa/cibuildwheel from 2.12.3 to 2.13.0 (#2624)
• Bump pypa/cibuildwheel from 2.13.0 to 2.13.1 (#2642)
• Bump pypa/cibuildwheel from 2.13.1 to 2.14.0 (#2673)
• Bump pypa/cibuildwheel from 2.14.0 to 2.14.1 (#2681)
• Bump pypa/cibuildwheel from 2.14.1 to 2.15.0 (#2716)
• Bump serde from 1.0.160 to 1.0.162 (#2609)
• Bump serde from 1.0.162 to 1.0.163 (#2612)
• Bump serde from 1.0.163 to 1.0.164 (#2644)
• Bump serde from 1.0.164 to 1.0.166 (#2666)
• Bump serde from 1.0.166 to 1.0.168 (#2679)
• Bump serde_json from 1.0.100 to 1.0.103 (#2682)
• Bump serde_json from 1.0.103 to 1.0.104 (#2701)
• Bump serde_json from 1.0.96 to 1.0.97 (#2653)
• Bump serde_json from 1.0.97 to 1.0.99 (#2660)
• Bump serde_json from 1.0.99 to 1.0.100 (#2675)
• Bump supercharge/redis-github-action from 1.5.0 to 1.6.0 (#2700)
• Bump tempfile from 3.5.0 to 3.6.0 (#2639)
• Bump tempfile from 3.6.0 to 3.7.0 (#2686)
• Bump tempfile from 3.7.0 to 3.7.1 (#2705)
• Bump thiserror from 1.0.40 to 1.0.43 (#2674)
• Bump thiserror from 1.0.43 to 1.0.44 (#2685)
• Bump wasm-bindgen from 0.2.84 to 0.2.86 (#2615)
• Bump wasm-bindgen from 0.2.86 to 0.2.87 (#2640)
• Bump wasm-bindgen-test from 0.3.34 to 0.3.36 (#2620)
• Bump wasm-bindgen-test from 0.3.36 to 0.3.37 (#2656)
• Bump web-sys from 0.3.61 to 0.3.63 (#2619)
• Bump web-sys from 0.3.63 to 0.3.64 (#2655)

v4.8.2

This is a bugfix release for a sourmash tax bug - see #2579.

Minor new features:

• add optional reporting for duplicated names in sketch fromfile (#2580)

Bug fixes:

• fix lineage_summary tax bug (#2579)

Dependabot updates:

• Bump serde from 1.0.159 to 1.0.160 (#2575)
• Bump serde_json from 1.0.95 to 1.0.96 (#2574)

v4.8.1

Minor new features:

• add --create-empty-results option to gather (#2557)

Bug fixes:

• fix exception error when no arguments are provided to sig intersect (#2555)

Cleanup and documentation updates:

• update sourmash sketch docs for building merged sketches (#2563)
• fix broken crates.io badge (#2556)
• add --csv to command-line docs for compare (#2565)

Developer updates:

• Remove codecov from tox, use GH action instead (#2568)

Dependabot updates:

• Update pytest requirement from <7.3.0,>=6.2.4 to >=6.2.4,<7.4.0 (#2558)
• Bump tempfile from 3.4.0 to 3.5.0 (#2561)
• Bump serde from 1.0.158 to 1.0.159 (#2560)
• Bump getrandom from 0.2.8 to 0.2.9 (#2559)

v4.8.0

Major new features:

• Add taxonomic utilities for LINs (#2469, #2468, #2519, #2518)

Minor new features:

• update prefetch/gather output to be clearer (#2543)

Cleanup and documentation updates:

• update version of screed to >= 1.1,<2 (#2415)
• minor fixes to release docs after v4.7.0 (#2498)

Developer updates:

• Remove pkg_resources usage (#2505)
• Add LICENSE and test data to sdist (#2490)
• Build pyodide wheels in CI (#2433)
• Update nix deps (#2506)
• bump to pyodide 0.23 (#2545)

Dependabot updates:

• Bump serde_json from 1.0.94 to 1.0.95 (#2540)
• Bump needletail from 0.5.0 to 0.5.1 (#2541)
• Bump serde from 1.0.156 to 1.0.158 (#2534)
• Bump thiserror from 1.0.39 to 1.0.40 (#2533)
• Bump typed-builder from 0.13.0 to 0.14.0 (#2527)
• Bump serde from 1.0.152 to 1.0.156 (#2530)
• Bump minimum rust version to 1.60 (#2528)
• Bump myst-parser from 0.19.1 to 1.0.0 (#2525)
• Bump chrono from 0.4.23 to 0.4.24 (#2524)
• Bump pypa/cibuildwheel from 2.12.0 to 2.12.1 (#2523)
• Bump myst-parser from 0.18.1 to 0.19.1 (#2507)
• Bump rayon from 1.6.1 to 1.7.0 (#2515)
• Bump tempfile from 3.3.0 to 3.4.0 (#2514)
• Bump needletail from 0.4.1 to 0.5.0 (#2512)
• Bump memmap2 from 0.5.9 to 0.5.10 (#2516)
• Bump thiserror from 1.0.38 to 1.0.39 (#2509)
• Bump mymindstorm/setup-emsdk from 11 to 12 (#2508)
• Bump serde_json from 1.0.93 to 1.0.94 (#2510)
• Bump typed-builder from 0.12.0 to 0.13.0 (#2511)

v4.7.0

sourmash release 4.7.0

Major new features:

• provide an initial plugin architecture for sourmash that supports new signature saving & loading mechanisms (#2428)
• add plugin support for new command-line subcommands (#2438)
• debias all containment values (#2243)

Minor new features:

• Use RankLineageInfo to simplify reading lineages (#2467)
• store taxids in lineageDB (#2466)
• Use new tax classes for taxonomic summarization (#2443)
• add tax summarization dataclasses for safety and flexibility (#2439)
• add --scaled to sourmash compare (#2414)
• replace lca_utils.LineagePair with tax_utils.LineagePair (#2441)
• Add new classes for lineage manipulation (#2437)

Cleanup and documentation updates:

• ReadTheDocs updates (#2445)
• update sourmash compare command-line docs (#2400)

Developer updates:

• fix python tests by bumping tox and pip cache versions (#2424)
• Update sphinx requirement from <6,>=4.4.0 to >=4.4.0,<7 (#2430)
• Build: replace milksnake with maturin (#2393)
• importlib_metadata is a dependency on old Python versions (#2484)
• Release docs: use two separate sed commands (#2483)
• minor fixes to release behavior (#2479)
• Use screed and maturin from nixpkgs in flake.nix (#2481)
• update release procedure after v4.6.0 and v4.6.1 (#2386)
• Update makefile and docs (#2432)

Dependabot updates:

• Bump once_cell from 1.17.0 to 1.17.1 (#2488)
• Bump ouroboros from 0.15.5 to 0.15.6 (#2487)
• Bump memmap2 from 0.5.8 to 0.5.9 (#2486)
• Bump supercharge/redis-github-action from 1.4.0 to 1.5.0 (#2485)
• Bump proptest from 1.0.0 to 1.1.0 (#2460)
• Bump web-sys from 0.3.60 to 0.3.61 (#2461)
• Bump serde_json from 1.0.91 to 1.0.93 (#2471)
• Bump wasm-bindgen-test from 0.3.33 to 0.3.34 (#2463)
• Bump cachix/install-nix-action from 18 to 19 (#2459)
• Bump wasm-bindgen from 0.2.83 to 0.2.84 (#2464)
• Bump typed-builder from 0.11.0 to 0.12.0 (#2451)
• Bump bumpalo from 3.9.1 to 3.12.0 (#2450)
• Bump pypa/cibuildwheel from 2.11.4 to 2.12.0 (#2447)
• Bump bzip2 from 0.4.3 to 0.4.4 (#2444)
• Bump once_cell from 1.14.0 to 1.17.0 (#2429)
• Bump serde from 1.0.151 to 1.0.152 (#2423)
• Bump pypa/cibuildwheel from 2.11.3 to 2.11.4 (#2422)
• Bump serde_json from 1.0.89 to 1.0.91 (#2418)
• Bump serde from 1.0.150 to 1.0.151 (#2419)
• Bump thiserror from 1.0.37 to 1.0.38 (#2417)
• Bump finch from 0.4.3 to 0.5.0 (#2416)
• Bump rayon from 1.6.0 to 1.6.1 (#2404)
• Bump serde from 1.0.149 to 1.0.150 (#2403)
• Bump pypa/cibuildwheel from 2.11.2 to 2.11.3 (#2402)
• Bump serde from 1.0.148 to 1.0.149 (#2397)
• Bump capnp from 0.14.5 to 0.14.11 (#2396)

v4.6.1

This is a quick patch-fix for sourmash v4.6.0, which introduced bug #2390. This bug broke sourmash sketch ... -o <file> with multiple ksizes, so that .zip and .sqldb output files contained only one ksize.

Bug fixes:

• fix sourmash sketch ... -o <file>.zip bug introduced in #2329 and released in v4.6.0 (#2391)
• disable cibuildwheel on PRs again (#2389)

v4.6.0

The major new feature in this release is the addition of tax summarize, which produces a human-readable summary of taxonomy databases.

The various tax functions also now support ingest of the output of tax annotate as a lineage spreadsheet - see the tax prepare documentation. This allows you to (for example) run tax summarize on the output of tax annotate.

---

Major new features:

• add tax summarize and support gather-tax input to taxonomy functions (#2333)
• report both weighted and unweighted % recovered in gather (#2301)
• replace chernoff bounds with exact probabilities (#2268)

Minor new features:

• switch remaining sig submodule commands over to sourmash_args sig output (#2377)
• use modern signature saving API throughout main CLI commands. (#2338)
• add column 3 to kreport (#2306)
• allow gzipped gather csv inputs to tax (#2339)
• display a better error message on attempting to write a read-only sqlite database (#2376)
• fix manifest load function to properly catch gzip.BadGzipFile (#2375)
• update kreport proportion for better resolution; match other tool outputs (#2331)

Bug fixes:

• Fix multigather so that the output CSV contains all matches. (#2322)
• remove default ksize of 31 from help message when it's not actually true. (#2295)

Cleanup and documentation updates:

• Updated python version (#2286)
• update docs re using multiple dbs (#2296)
• fix some tax doc issues (#2365)
• fix kreport documentation (#2302)

Developer updates:

• Fix cibuildwheel actions (#2384, #2385, #2388)
• return Err for angular_similarity when abundance tracking is off (#2327)
• cargo check fixes for Rust beta (1.65) (#2298)
• fix unnecessary typecasts in Rust (#2366)
• fix Signature.minhash API during sourmash sketch (#2329)
• fix return type of LCA_SqliteDatabase.select (#2382)

Dependabot updates:

• Bump conda-incubator/setup-miniconda from 2.1.1 to 2.2.0 (#2363)
• Bump counter from 0.5.6 to 0.5.7 (#2336)
• Bump finch from 0.4.1 to 0.4.3 (#2283)
• Bump getrandom from 0.2.7 to 0.2.8 (#2347)
• Bump memmap2 from 0.5.7 to 0.5.8 (#2364)
• Bump myst-parser from 0.18.0 to 0.18.1 (#2345)
• Bump pypa/cibuildwheel from 2.11.1 to 2.11.2 (#2353)
• Bump pypa/cibuildwheel from 2.9.0 to 2.10.2 (#2307)
• Bump rayon from 1.5.3 to 1.6.0 (#2373)
• Bump serde from 1.0.145 to 1.0.147 (#2348)
• Bump serde from 1.0.147 to 1.0.148 (#2378)
• Bump serde_json from 1.0.86 to 1.0.87 (#2349)
• Bump serde_json from 1.0.87 to 1.0.88 (#2374)
• Bump serde_json from 1.0.88 to 1.0.89 (#2379)
• Bump typed-builder from 0.10.0 to 0.11.0 (#2356)
• Update bitstring requirement from <4,>=3.1.9 to >=3.1.9,<5 (#2372)
• Update docutils requirement from <0.18,>=0.17.1 to >=0.17.1,<0.20 (#2344)
• Update pytest requirement from <7.2.0,>=6.2.4 to >=6.2.4,<7.3.0 (#2354)
• Update pytest-cov requirement from <4.0,>=2.12 to >=2.12,<5.0 (#2346)
• Version bumps (#2282)
• Combine latest dependabot PRs: cibuildwheel, finch, serde_json (#2343)
• Rust deps updates without MSRV bump (#2315)

v4.5.0

sourmash v4.5.0 provides several minor bug fixes, as well as a number of new features.

This release also includes two minor Python API breaking changes - by default, SourmashSignature objects loaded from files are "frozen", and we force explicit keyword arguments on MinHash object construction.

Finally, this release updates the sourmash documentation with several new tutorials, including one on using sourmash tax to classify metagenomes with MAGs + GTDB.

Bug fixes

• Fix sourmash tax argument parsing for multiple -g and -t arguments (#2218)
• Prevent loading multiple independent gather results files in sourmash tax (#2244)
• Fix query_abundance column when --ignore-abundance is set in gather (#2251)
• fix pickle protocol to properly adjust ksize in __getstate__ (#2265)
• clean up zip error handling for bad zip files (#2270)

Minor new features

• Use the bias factor for containment when estimating ANI (#2057)
• add human output format to sourmash tax; provide tutorials (#2158)
• add kreport output format to tax metagenome (#2239, #2249)
• add --distance-matrix option to sourmash compare (#2225)
• update database load UX for gather etc. (#2204)
• add generic support for gzipped and zipfile CSVs (#2195)
• implement tax grep to produce identifier picklists from taxonomies (#2178)

Cleanup and documentation fixes

• add sourmash tax tutorial (#2158)
• revise command-line docs for sourmash sig subcommands (#1714, #1717)
• Clarify containment direction for matrix output (#2215)
• Add ANGUS tutorial to docs (#1114)
• update links to static rmd (#1177)
• update search documentation, help, and output. (#2222)
• Fix signature filter command (#2159)
• fix notification message about query scaled (#2183)
• adjust gather output width on terminal (#2176)

Developer updates

• Add FrozenSourmashSignature (#1610)
• force explicit kwargs on MinHash constructor (#2174)
• fix ReadTheDocs by using a more recent conda version (#2231)
• refactor and add tests for containment direction for ANI calculation (#2215)
• fix test_storage_convert to allow success of sourmash convert (#2232)
• Updating tests/test_sourmash.py::test_storage_convert to use runtmp fixture instead of utils.TempDirectory() (#1739)
• Bump pypa/cibuildwheel from 2.8.1 to 2.9.0 (#2207)
• use stderr for test output printing (#2217)
• fix for sphinx 5.10 (#2147)

v4.4.3

Minor new features:

• use and report ANI from tax genome summarization (#2005)

Performance improvements:

• avoid instantiating a hashes class (#2132)

Cleanup and documentation fixes:

• update various descriptions to talk about k-mers, not just DNA (#2137)

Developer updates:

• fix docs building for pip 22.2 (#2143)
• change dependabot rebase-strategy to disabled for rust dependencies (#2142)
• Rust deps and nix flakes updates (#2141)
• add pytest-xdist and -n4 to pytest and tox configs (#2138)
• update release instructions after v4.4.2 (#2131)