fix: remove refs from specification

[technosophos]

This removes the refs section from the specification, and intentionally backs out the definition of how invocation images are to perform substitutions of image names. This decision was arrived at on issue #117 after trying a few other ways of handling this behavior.

Supersedes #117
Closes #113

[jeremyrickard]

This seems reasonable to me. @carolynvs and I were actively debating this specifically with Porter and how we should generate that part of the bundle....so I'm good with this.

[jeremyrickard]

LGTM

[simonferquel]

This seems a bit far fetched. A more realistic scenario is:

For example, a CNAB publishing tool can copy a CNAB and all its component images in a new repository without having to rebuild the invocation image: it just have to modify the CNAB's image map. At invocation time, the run tool will use the injected image map to update the image references for the impacted components.

[ryanmoran]

@simonferquel maybe I am misunderstanding what you believe to be far-fetched, but I believe the scenario that is described is precisely what would occur for many bundle consumers running bundles inside of a network-partitioned environment. In this case, a CNAB publishing tool would have no understanding of the registry inside of the physical barrier. Instead, the images would need to be relocated from a DMZ into a registry inside the air-gapped network such that the platform running inside that network could take advantage of them.

[simonferquel]

Yes, but this should not be the role of the invocation image to do so (the run tool).
It should be an internal provisioning tool run within the DMZ (e.g., using cnab-to-oci to pull a bundle from a public registry, and push it back to an internal one).

This is a scenario we already cover with cnab-to-oci and docker-app:

• docker-app creates bundles with a /run/tool that understands the injected image-map
• cnab-to-oci can push a bundle with all its images in a single repository, rewriting the image-map so that component images are replaced with digested references in this repository
• cnab-to-oci can then be used to re-push the same bundle to different registries/repositories, each time rewriting the image-map.
• when docker-app (or duffle) is used to install such a rewritten bundle, it injects the image-map with the new images reference.

The run tool within the invocation image never copies anything from a registry to another, it just consumes the image-map which might point to different image references than those defined at build time.

[glyn]

Although I am personally in favour of making invocation images responsible for image relocation, some of those present at the CNAB Community meeting on 6 March 2019 expressed a preference for splitting that responsibility between the runtime and the invocation image.

In that approach, the runtime (or, presumably, other associated components) would be responsible for pushing images to any alternative registry (or registries) and providing the mapping between old and new images names to the invocation image so it could, for example, modify kubernetes configuration files to use new image names in place of old image names.

With that alternative approach, it might make more sense to retain (and complete the specification of) refs since this feature assists invocation images in applying the image mapping to bundle contents.

A decision still needs to be taken about which approach to adopt.

/cc @SteveLas @michelleN

[simonferquel]

@glyn as the invocation image has (obviously) a complete knowledge of its layout, it should only need a component-name -> image reference mapping (that is the approach we took for docker-app invocation images). The problem with refs, is that this kind of declarative replacements only works for a limited set of file types (e.g.: static yaml/json/xml/toml files). As soon as you get things less universally parsable (Helm-style heavily Go-Templated yaml, protobuf serialized payloads, or string constants in a compiled binary), this becomes completely useless.
As ultimately, the logic for doing the replacement will live in the invocation image, I don't think there is much value to put this kind of info in the injected metadata. And if an invocation image wants to use an general purpose yaml tool for doing replacement, it can have a (static) layout as:

/cnab/app
  run -> the main run tool
  refs.json -> a file in whatever format with a mapping "component -> file replacement expression"
  data/ -> dir with whatever yaml, xml, json, toml files referencing the components

At runtime, it will be injected the image-map.json, and it can use the rules defined in /cnab/app/refs.json with the values injected in /cnab/app/image-map.json. I think it makes it much more flexible (we don't need to rely on a rigidly specified refs structure, while still being able to do declarative replacements)

[glyn]

@simonferquel that's very helpful and I agree. On that basis, this PR should proceed regardless of the approach we ultimately take for image relocation.

[technosophos]

Yeah, I've come around to the view that the refs system is pretty fatally flawed. So my preference would be to get it out of the spec now, even if we eventually find something better.

[technosophos]

@simonferquel I addressed your objection above to make the run tool example less ambitious.

Any objections to my merging this PR?

[simonferquel]

lgtm