Implement with items for Orquesta workflows

[m4dcoder]

Add support of with items and concurrency in the task spec of Orquesta workflows. This PR is dependent on changes in the Orquesta engine at StackStorm/orquesta#94. Unit and integration tests are written to cover single and multiple items list, with or without concurrency, and additional operations such as pause, resume, and cancel of workflow while the task is being executed. The following examples are different variations of how workflow authors can define with items for a task.

task1:
  with: <% ctx(x) %>
  action: core.echo message=<% item() %>

task1:
  with:
    items: <% ctx(x) %>
    concurrency: 2
  action: core.echo message=<% item() %>

task1:
  with: i in <% ctx(x) %>
  action: core.echo message=<% item(i) %>

task1:
  with:
    items: i in <% ctx(x) %>
    concurrency: 2
  action: core.echo message=<% item(i) %>

task1:
  with: i, j in <% zip(ctx(x), ctx(y)) %>
  action: mock.foobar a=<% item(i) %> b=<% item(j) %>

task1:
  with:
    items: i, j in <% zip(ctx(x), ctx(y)) %>
    concurrency: 2
  action: mock.foobar a=<% item(i) %> b=<% item(j) %>

[nzlosh]

Would it be possible to support an option asynchronous for with-items?

When a parent workflow uses with-items to loop over a large list and calls a sub-workflow the current (mistral) synchronous behaviour produces a batching effect where concurrent: X determines the number of sub-workflows are run to completion before executing the next X items.

I have a workflow where the sub-workflow upgrades machine packages, reboots the machine and checks the monitoring staus is OK. If in a single with-items iteration 9/10 complete in under 5 minutes but 1 takes 13 minutes, no subworkflows are executed until 13 minutes later. This is not ideal from a efficiency point of view.

If we had support for asynchronous with-items the behaviour would be to constrain the number of concurrent sub-workflows, but once a sub-workflow completes, then next sub-workflow is triggered immediately.

[m4dcoder]

@nzlosh This implementation is already operating as you described. The Orquesta conductor will schedule the next item(s) as soon as there are free slots as defined by concurrency. Also, you may want to recheck Mistral, I don't recall it working batch like.

[nzlosh]

❤️ great news! Silly question, is there an option for synchronous with-items? :)

[m4dcoder]

@nzlosh yep, concurrency=None 😜

[nzlosh]

Nice job, that's awesome!! 🥇 Thanks for the great work!

[bigmstone]

Approved, just an obligatory reminder to clean up orquestra branch path.

[m4dcoder]

@bigmstone Thanks. I just updated requirements.