Cool with custom channel anaconda release?

[dkoslicki]

Hi @ctb , @bluegenes and others! We'd (@chunyuma) like to publish YACHT to conda and it uses a specific commit of branchwater as a dependency. However, conda doesn't like that the way we're currently using pip, and it was recommended to us to make a conda release of branchwater in a custom channel. Are y'all ok with this? We'd make sure to clearly indicate the brancheater in this custom channel is meant only for installing YACHT.

[ctb]

A few quick thoughts -

• looks like you're using version 0.8.1, not a specific commit - excellent!
• I am OK with a conda release of branchwater for a custom channel, but I don't know if or how that will mess things up when we (sooner? later?) post branchwater on conda-forge. @luizirber any thoughts on a good short-term solution here? I'm thinking maybe if you name it something other than 'branchwater' that might avoid future problems?

thank you for checking!

[dkoslicki]

Thanks! We’ll wait to hear from @luizirber before pursuing a bioconda release in a custom channel. Looks like the most straightforward way in the time-being is a post-install pip script. We’ll use that until a full conda approach becomes feasible and/or you all release branchwater to conda

[ctb]

seeing if maybe we can just build a conda-forge recipe for branchwater - trial run here

[luizirber]

seeing if maybe we can just build a conda-forge recipe for branchwater - trial run here

Yup, that was going to be my suggestion! conda-forge is upstream of bioconda (and required to use it properly), so it is already in the channel list. Another custom channel would still be hard to distribute ¹, and conda-forge allows more customization (like extra archs) that bioconda doesn't support, which makes more sense for non-pure Python packages.

Footnotes

1. it is already hard to get people to setup bioconda+conda-forge properly 😅 ↩

[ctb]

whoo! linux build worked!

[dkoslicki]

Awesome! Let us know if we can help at all with getting it on conda-forge (eg. @ctb I have folks who may be able to help with the recipe)! Our YACHT conda release is blocking on branchwater, so we're rather motivated to help! :)

[ctb]

heads up - per #17, we're renaming to sourmash_plugin_branchwater over in #156

[ctb]

hi @dkoslicki I put through the rename and got the Linux build working. Unfortunately, I'm out of time to work on it intensively today (and probably tomorrow). SO, if you & yours would like to help that would be lovely!

The two thoughts I had -

• we could just support Linux for now, if that would work for you - and if so, I could use guidance on how to disable win/osx builds.
• alternatively, OS X support would be fantastic, but that build is not working right now. So help getting that build working would be fantastic!

Probably the easiest would be to just submit PRs into my branch, the source branch here, and I can merge them quickly - I have time today, just not big chunks.

[ctb]

The linux-only build seems to have worked. Shall I put the PR up for merge or do we want to try to get OS X to work first? @luizirber @dkoslicki @bluegenes

[dkoslicki]

We've been having a heck of a time getting OS X to work, mainly due to bioconda moving to Azure, it looks like their build system is missing a necessary clang lib to compile Rust stuff. I'd vote just go with linux for the time being as we pursue the missing lib issue in the bioconda Gitter channel.

[ctb]

conda-forge/staged-recipes#24530 merged! I'll close this issue (and open a new one about win and osx targets) once I confirm that install from conda-forge is working.

[ctb]

progress! conda-forge/sourmash_plugin_branchwater-feedstock@3ff1132#commitcomment-134263657

[dkoslicki]

Whoot! I just noticed it's officially available on conda-forge now. Thanks @ctb !

[ctb]

Yep, and looks like it all works!

% mamba create -n branch-test-dec5 -y sourmash_plugin_branchwater sourmash
% mamba activate activate branch-test-dec-5
% sourmash scripts

== This is sourmash version 4.8.4. ==
== Please cite Brown and Irber (2016), doi:10.21105/joss.00027. ==

options:
  -h, --help  show this help message and exit

available plugin/extension commands:
  sourmash scripts manysearch       - search many metagenomes for contained genomes
  sourmash scripts multisearch      - massively parallel in-memory sketch search
  sourmash scripts fastgather       - massively parallel sketch gather
  sourmash scripts fastmultigather  - massively parallel sketch multigather
  sourmash scripts index            - Build Branchwater RevIndex
  sourmash scripts check            - Check Branchwater RevIndex
  sourmash scripts manysketch       - massively parallel sketching

Let us know if you run into any problems building on top of this - excited to see YACHT out there!

[ctb]

closing in favor of #160 for future work!

[ctb]

We've been having a heck of a time getting OS X to work, mainly due to bioconda moving to Azure, it looks like their build system is missing a necessary clang lib to compile Rust stuff. I'd vote just go with linux for the time being as we pursue the missing lib issue in the bioconda Gitter channel.

so @luizirber got sourmash-minimal working on conda-forge, and I copied his approach in conda-forge/sourmash_plugin_branchwater-feedstock#3 - so both sourmash-minimal and branchwater should be working on MacOS now!

[dkoslicki]

Awesome, thanks for the update! That looked rather nontrivial to do!