Make the TCK easily executed by different language supports

[viktorklang]

TCK verification should be done as a part of the individual language supports' builds—it would be beneficial to package it as a Docker image where it is possible to pass in appropriate parameters as Docker run parameters, optionally making the TCK a Graal native-image executable to bring down the startup costs.

[viktorklang]

Ping @sleipnir @jroper

[sleipnir]

Thanks @viktorklang
The fact that it was packaged via Docker at first would be enough for implementers to follow independently from the main repository and automate their build steps with TCK being one of them.
I believe that the native build could be a plus added later.
At the moment Docker would already provide the necessary abstraction for functionality.

[marcellanz]

I'm in favour of this. It's what the go language support currently does and it also helps to cut dependencies from the main project, especially like compilers and runtimes one would not like to get track of (go, dart, php, …).

[marcellanz]

I'm in favour of this. It's what the go language support currently does and it also helps to cut dependencies from the main project, especially like compilers and runtimes one would not like to get track of (go, dart, php, …).

I've misread the issue a bit o_O. The Go language support clearly does not what the issue is proposing, it's actually the other way around… still like it :)

[viktorklang]

@marcellanz You are forgiven. :D

[marcellanz]

Since we have these proxy- and tck-images with #325 now, I iterated a bit on the idea of a nightly TCK run. I started with a travis matrix configuration that got quickly a bit messy and then went with two scripts.

It is mainly two bash scripts, a gist repository and an Observable notebook for the visual part:
/~https://github.com/marcellanz/cloudstate/tree/nightly/tck_run/nightly

It runs nightly with a travis job on a cloudstateio branch, but has no dependencies to the repo otherwise. I have a few more ideas (more precise runtime stats, runtime memory usage, history, notifications), but it does what I thought it could do. Also, perhaps there is a better way to run such a board also. I'm very much open to suggestions and ideas.

The visualization should be easily embeddable using Observable's embedded runtime.js:

<div id="observablehq-a5a2ab49"></div>
<script type="module">
import {Runtime, Inspector} from "https://cdn.jsdelivr.net/npm/@observablehq/runtime@4/dist/runtime.js";
import define from "https://api.observablehq.com/@marcellanz/cloudstate-go-tck-verification.js?v=3";
const inspect = Inspector.into("#observablehq-a5a2ab49");
(new Runtime).module(define, name => name === "viewof map" ? inspect() : undefined);
</script>

[pvlugter]

Good idea. Looks great.

[pvlugter]

Is there actually any need to create and maintain a native-image version of the TCK docker image?

[marcellanz]

Good question. I don't know how much of a production hot-path the TCK itself runs. As far as I understand it, the TCK uses the grpc-client stubs to issue service calls against the proxy and intercept them. The part of the TCK would not run in production, yes?

[ TCK (grpc-stub client) ] =a=> [ proxy ] =b=> [ user-function ]

This way, I think we can leave the tck image in its form. The TCK also covers a specific need to test the spec we defined, it might not be the tool to do end to end functional testing?

But I'm not sure. @viktorklang wdyt?

[pvlugter]

I don't think we need a native image of the TCK itself, but asking since it was mentioned in the original issue description. I don't think we need to save on startup times, and maintaining a native-image build is extra work. Or we could do this if it's useful as an ongoing exercise to improve the native-image build infrastructure.

Yes, we should probably have different end-to-end functional tests, and also any load/stress testing. I think we'll need to have tests for different persistence plugins. And run some tests in clustered mode — these would be useful to check native image builds as well.

[marcellanz]

Regarding the native TCK image mentioned in the issue:
The TCK image starts in 2 seconds on my box. The proxy in about 3.5s and native-proxy in ~<=1s measured visually. Thats good enough for me.

I like he idea of having a metric about native build times. It's perhaps not a thing for the normal development cycle but gives the project a sense for how optimisations affected the native build. Perhaps not a priority.

[pvlugter]

As we have the docker images published now, and don't need a native image version, I'll close this issue and we can continue with evolving the TCK