Change Notes

The most important change in this release is support for Groovy 3.0 and JDK11+. This ensures that Bpipe keeps using well supported versions of Groovy and Java, and also paves the way to longer term use of advanced JDK versions (21+), where some significant new JVM features should allow resolution of some long standing issues with scalability of Bpipe pipelines.

Note: for this release, both Groovy 3.0 and Groovy 2.5.13 version will be provided as downloads. Future releases may
transition to only supporting Groovy 3.0 - all users are encouraged to use the Groovy 3.0 version unless
issues are experienced. Groovy 2.5.x can not support recent Java versions so this will be a necessary migration
at some point.

Other notable changes include a convenient "dev" mode that operates similarly to using bpipe test, but
allows you to see colorised commands showing which parts Bpipe recognises as input or output variables, as well
as showing other contextual information about the command that is to execute such as variables,
assigned resources and container settings. A key element of this mode is that it can dynamically update when
you edit the source code to allow fast iteration to get the command looking right.

A new syntax also allows convenient creation of "named" branches by multiplying a Groovy Map literal. As an example:

'%.csv' * [  
  quality_control: check_format + check values,
  analysis: clean_data + train_model
] + finalise_results

This will create two branches named quality_control and analysis. As always, Bpipe allows for
dynamic construction of these inside your run method to give lots of flexibility.

Apart from the listed features and fixes, there are a huge number of optimisations and minor bug fixes
included in this release. Bpipe should run faster and more reliably than before.

• FEATURE: support for Groovy 3.0 and JDK11+

• FEATURE: experimental 'dev' mode - use bpipe dev ...
  to enter test mode that highlights
  input and outputs in commands, hot updates with changes
  and shows branch/command context

• FEATURE: support for optional outputs via .optional pseudo
  extension

• FEATURE: named branches - use multiply syntax with
  groovy Map literal as alternative to list, to create named
  parallel branches

• FEATURE: dynamic splits. Use forwardSplit(...) to create
  pipeline branches that are dynamically determined by result
  of a previous stage. (see docs)

• FEATURE: Command configuration can now be done based on stage name rather
  than command name. This significantly relieves the need to add dedicated
  config names to commands in order to adjust their configuration.

• FEATURE: duplicate output files are now explicitly detected and warned about

• FEATURE: enhanced warnings and messages when transform / produce / filter
  do not create expected files

• FEATURE: add display of cores into stats table

• FIX: Slurm executor polling not operating correctly

• FIX: Don't create stray empty .bpipe folders

• FIX: Autoarchive leaving behind empty dirs on some file systems

• FIX: wrong input may be used in from(multiple) transform() to() construct

• DEPRECATED: the Multi command is no deprecated as there are no scenarios where
  this is preferred over paralleising using branching structures