Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

deprecate sourmash multigather for 5.0 #1614

Open
ctb opened this issue Jun 20, 2021 · 4 comments
Open

deprecate sourmash multigather for 5.0 #1614

ctb opened this issue Jun 20, 2021 · 4 comments
Labels
5.0 issues to address for a 5.0 release
Milestone

Comments

@ctb
Copy link
Contributor

ctb commented Jun 20, 2021

IMO.

code is messy, is not as well tested as gather, it was never that much faster than gather, and doesn't support picklists. our goal should be to continue to make gather even faster :)

@ctb
Copy link
Contributor Author

ctb commented Jun 26, 2021

an alternative brainstormed in #1624 would be to make multigather faster by doing a prefetch on all signatures combined - that is, optimize around multiple queries, rather than multiple databases (which was the original intent of multigather).

@ctb
Copy link
Contributor Author

ctb commented Aug 19, 2023

A few thoughts here -

@ctb
Copy link
Contributor Author

ctb commented Feb 29, 2024

  • instead of deprecating/removing multigather, we could make it a plugin

  • note that the pyo3_branchwater plugin has fastmultigather which is indeed much faster than multigather also; so we could upgrade fastmultigather to do what people want and need

after ~6 months or so, I'm doubling down on this opinion - see #2722 (comment),

@bluegenes is doing different experiments in re output files with fastmultigather over in sourmash-bio/sourmash_plugin_branchwater#197, and so I think we have routes to explore better output over there.

It might be that a long-term future for multigather is to deprecate it in sourmash and direct people to fastmultigather, which is actually much faster than multigather.

@ctb
Copy link
Contributor Author

ctb commented Jun 20, 2024

note: as of sourmash_plugin_branchwater v0.9.5 link, the results from fastgather and fastmultigather are now identical to those from sourmash gather. So I think we should just update to say, "use fastmultigather".

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
5.0 issues to address for a 5.0 release
Projects
None yet
Development

No branches or pull requests

2 participants