feat: add clade inheritance, requires new augur clades feature

[corneliusroemer]

Description of proposed changes

This is preparation for once augur PR nextstrain/augur#846 is merged and released.

New inheritance feature allows more succinct and reliable clade definitions.

Testing

Augur clades function read_in_clade_definitions should be run on old and new version of clades.tsv to check the output is consistent.

Importantly, if this is merged, we need to update the min augur version to the augur version of the new feature release.

Requires augur version 14.0.0 or higher

Release checklist

If this pull request introduces backward incompatible changes, complete the following steps for a new release of the workflow:

• Determine the version number for the new release by incrementing the most recent release (e.g., "v2" from "v1").
• Update docs/src/reference/change_log.md in this pull request to document these changes and the new version number.
• After merging, create a new GitHub release with the new version number as the tag and release title.

If this pull request introduces new features, complete the following steps:

• Update docs/src/reference/change_log.md in this pull request to document these changes by the date they were added.

[corneliusroemer]

@trvrb I've held back on merging this for a while because I wanted to let people have a chance to upgrade their ncov workflows to v14. I think now enough time has passed. Inherited clade definitions will make our lives a bit easier, see for example Omicron before and after:

Before:

21K (Omicron)	nuc	8782	C
21K (Omicron)	nuc	14408	T
21K (Omicron)	nuc	18163	G
21K (Omicron)	nuc	23403	G
21K (Omicron)	nuc	28881	A
21K (Omicron)	nuc	28882	A
21K (Omicron)	nuc	15240	T
21K (Omicron)	nuc	23202	A
21K (Omicron)	nuc	23525	T
21K (Omicron)	nuc	23599	G
21K (Omicron)	nuc	27259	C

21L (Omicron)	nuc	8782	C
21L (Omicron)	nuc	14408	T
21L (Omicron)	nuc	18163	G
21L (Omicron)	nuc	23403	G
21L (Omicron)	nuc	28881	A
21L (Omicron)	nuc	28882	A
21L (Omicron)	nuc	21618	T
21L (Omicron)	nuc	22775	A
21L (Omicron)	nuc	23599	G

21M (Omicron)	nuc	8782	C
21M (Omicron)	nuc	14408	T
21M (Omicron)	nuc	18163	G
21M (Omicron)	nuc	23403	G
21M (Omicron)	nuc	28881	A
21M (Omicron)	nuc	28882	A
21M (Omicron)	nuc	23599	G

After:

21K (Omicron)	clade	21M (Omicron)	
21K (Omicron)	nuc	15240	T
21K (Omicron)	nuc	23202	A
21K (Omicron)	nuc	23525	T
21K (Omicron)	nuc	27259	C

21L (Omicron)	clade	21M (Omicron)	
21L (Omicron)	nuc	21618	T
21L (Omicron)	nuc	22775	A

21M (Omicron)	clade	20B
21M (Omicron)	nuc	18163	G
21M (Omicron)	nuc	23599	G

I think it would make sense to test/merge this first before adding BA.2.12.1 - this should now reproduce just what the current clade definitions are.

[emmahodcroft]

Just to be clear, this is backward-compatible, though right? If you haven't updated your clades.tsv file to include inheritance, and still do it longhand, it still works? I think that's the case but wanted to check.

[corneliusroemer]

Correct, backwards compatible in that it doesn't require inheritance. But Augur v14+ is required for ncov from hereon since old Augur doesn't understand inherited clades.

…

On Wed, Apr 27, 2022, 11:46 Emma Hodcroft ***@***.***> wrote: Just to be clear, this is backward-compatible, though right? If you haven't updated your clades.tsv file to include inheritance, and still do it longhand, it still works? I think that's the case but wanted to check. — Reply to this email directly, view it on GitHub <#855 (comment)>, or unsubscribe </~https://github.com/notifications/unsubscribe-auth/AF77AQJB76CLUFCRIF3WHZDVHEEIVANCNFSM5NRMQJ7Q> . You are receiving this because you authored the thread.Message ID: ***@***.***>

[trvrb]

@corneliusroemer: Awesome! Thanks for this.

Can I ask that you generally spin out trial builds for testing purposes? You can do this by going to "Actions" then to "Rebuild GISAID phylogenetic datasets" then to "Run workflow" and select the branch of interest and type in this branch name as "short name" for trial build:

I just did this here.

Edit:

Trial builds are available at:

• https://nextstrain.org/staging/ncov/gisaid/trial/clades-inheritance/global/
• https://nextstrain.org/staging/ncov/gisaid/trial/clades-inheritance/africa/
• https://nextstrain.org/staging/ncov/gisaid/trial/clades-inheritance/north-america/
  ...

Everything looks as it should.

[trvrb]

Additionally, where do we specify minimum version requirements for the ncov workflow? This needs to be updated to be clear that Augur v14.0 is now required to run the workflow. However, I can't now find where this is documented / specified. cc @huddlej @victorlin

[victorlin]

@trvrb: where do we specify minimum version requirements for the ncov workflow?

I think the closest thing is workflow/envs/nextstrain.yaml which specifies augur=15.0.0, but this file isn't recommended in the current setup steps which just point to the generic Nextstrain tools setup guide.

Maybe we could add a note on this new requirement in change_log.md for the next version v12? And we could specify minimum requirement of augur=15.0.0 if we want to document support for relative dates in the workflow.

[trvrb]

Thanks @victorlin! And thanks again for the PR @corneliusroemer!

I believe we can just go ahead with merging this basically as is in this case. I'll just update the change log as needed.

[corneliusroemer]

Thanks @trvrb for showing how to do test builds, that's super useful! Glad it all works as expected.