image-index: mark os.version, os.features and variant as "reserved" #632

AkihiroSuda · 2017-03-31T04:59:32Z

Updates #622.

Previous version of this PR as follows: AkihiroSuda@24b8332

 - os.version:  speculated Windows-specific convention from the example
   - I think this property can be marked as "reserved, SHOULD NOT be used" unless someone is already using /~https://github.com/opencontainers/image-spec/pull/632#discussion_r109088972
 - os.features: still unclear. should be clearly standardized in post-v1.0.0
   - ditto
 - variant:     speculated arm-specific convention from the example
   - ditto
 - features:    speculated x86-specific convention from the example, and put little modification
   - this should be standardized in v1.0.0 for x86, because runtime-spec has already some Xeon-specific spec: /~https://github.com/opencontainers/runtime-spec/commit/73a6002bf3a2fc80b4aad706b7271993b9989404

Signed-off-by: Akihiro Suda suda.akihiro@lab.ntt.co.jp

AkihiroSuda · 2017-03-31T05:05:26Z

The features part of my PR seems a dupe of #631.
I'll omit features part from this PR if #631 wins.

AkihiroSuda · 2017-03-31T05:14:31Z

image-index.md

@@ -55,19 +55,29 @@ For the media type(s) that this document is compatible with, see the [matrix][ma

    - **`os.version`** *string*

-        This OPTIONAL property specifies the operating system version, for example `10.0.10586`.
+        This OPTIONAL property specifies the operating system version.


I'm unsure this property is a minimal version requirement or an exact match.

cc @jhowardmsft

Is Windows already using this property?
If not, I think it is even ok to mark this property as "reserved, SHOULD NOT use".

If not, I think it is even ok to mark this property as "reserved, SHOULD NOT use".

That sounds good to me (or just removing the property from the spec entirely). Looking up docs for GetVersionEx() turned up this:

GetVersionEx may be altered or unavailable for releases after Windows 8.1. Instead, use the Version Helper functions

The version helper functions all look like comparisons, and poking around a bit turns up:

Rather than using GetVersionEx to determine the operating system platform or version number, test for the presence of the feature itself.

So it seems like the recommended Windows practice would be to drop os.version and instead use os.features.

@wking Please allow the windows folks to comment. This is not your area of expertise.

@jhowardmsft cc

wking · 2017-03-31T17:41:31Z

image-index.md

+        This OPTIONAL property specifies the variant of the CPU.
+        Image indexes SHOULD use, and implementations SHOULD understand the following convention:
+        - `arm` and `arm64`: `machine` field of the Linux `utsname` structure returned by `uname(2)`. e.g., `armv6l`.
+        If an architecture is not listed here, it SHOULD be submitted to this specification for standardization.


“If an architecture” → “If a variant”. Or maybe “If an OS”, if you consider the current Linux line to be sufficient for that OS.

@AkihiroSuda This needs line up with the GOARM in https://golang.org/doc/install/source#environment. However, I am concerned that we are making such strong statements when /~https://github.com/opencontainers/runtime-spec/blob/d094a5c9c1997ab086197b57e9378fabed394d92/config.md#platform is not populated. I don't think this should be defined in terms of uname, as that is linux specific.

A table might be good here to show common variants that are accepted.

How about marking this property "reserved, SHOULD NOT use" if there is a concern?
I think we don't need to rush to standardize this property toward v1.0.0

wking · 2017-03-31T17:46:49Z

image-index.md


    - **`os.features`** *array of strings*

        This OPTIONAL property specifies an array of strings, each specifying a mandatory OS feature (for example on Windows `win32k`).
+        A convention for other features SHOULD be submitted to this specification for standardization.


On Linux, this could include an array of CONFIG_ entries needed in the host kernel. For example, ["FUSE_FS"] or ["NFSD_V4"]. Or it could be implementation-defined on Linux until we want to get around to defining some values.

Is there an extant use case showing this to be useful?

Is there an extant use case showing this to be useful?

Using kernel config options? I don't have one. Is there an extent use case showing os.features to be useful? I was just trying to extend an assumed-useful property to Linux, since the spec definition currently lacks guidance for this property on Linux. But I'm also fine with leaving it implementation-defined on Linux if we want to punt.

wking · 2017-03-31T17:50:42Z

On Thu, Mar 30, 2017 at 10:05:28PM -0700, Akihiro Suda wrote: I'll omit `features` part from this PR if #631 wins.

I'm also happy to close #631 in favor of this PR if that would make life easier for the maintainers (although I think it will be easier if we have a separate PR for each field we're clarifying).

stevvooe · 2017-03-31T18:41:03Z

specs-go/v1/index.go

 	OSVersion string `json:"os.version,omitempty"`

 	// OSFeatures is an optional field specifying an array of strings,
 	// each listing a required OS feature (for example on Windows `win32k`).
 	OSFeatures []string `json:"os.features,omitempty"`

 	// Variant is an optional field specifying a variant of the CPU, for
-	// example `ppc64le` to specify a little-endian version of a PowerPC CPU.
+	// example `armv6l` to specify a particular CPU variant of the ARM CPU.


Could we please leave the power example in here?

cc @estesp

I was unsure why variant=ppc64le is useful, because architecture would be always ppc64le in such a variant.
So I just thought standardization for ppc64le would be better deferred to post-v1.0.0, but not sure.

I was unsure why variant=ppc64le is useful, because architecture would be always ppc64le in such a variant.

That is correct, based on https://golang.org/doc/install/source#environment.

These properties seems unclear and not suitable for standardization at the moment. Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>

AkihiroSuda · 2017-04-10T09:20:45Z

For moving this forward, I made following update to this PR:

Mark os.version, os.features, and variant as "reserved"
Remove change for CPU features. This can be discussed in @wking 's PR

The previous version of this PR is available as AkihiroSuda@24b8332

jonboulle · 2017-04-10T11:18:17Z

I do think this is probably preferable for 1.0, then we can explore #631 or similar a bit more thoughtfully. So, LGTM if enough other maintainers will agree.

coolljt0725 · 2017-04-10T11:47:19Z

image-index.md


    - **`variant`** *string*

-        This OPTIONAL property specifies the variant of the CPU, for example `armv6l` to specify a particular CPU variant of the ARM CPU.
+        This OPTIONAL property is *reserved* for future versions of the specification and hence SHOULD NOT be used.


I think we want this for supporting image on arm

stevvooe · 2017-04-10T21:04:20Z

@estesp PTAL

estesp · 2017-04-10T23:38:42Z

I believe that reserving variant for future use may break the usefulness of index capabilities for architectures that cannot be fully defined with a standardized Linux (or Go) arch definition.

As @coolljt0725 mentions, arm is one such architecture (and others in the traditional "embedded" space) which will be neutered if the only available specifier is os.arch. Because there is a direct link between compiled binary capabilities and the combo of an "arch" and "variant," I see it differently than version and features which can have debatable impact on image contents viability for a specific CPU/hardware platform. And, yes, there are exceptions, but to me variant + arch are a base requirement to the viability of using index across all currently known "container runtime" architectures.

One final caveat--the os.version may be important to the Windows platform as that was added by the Microsoft team to the Docker v2.2 image specification. Is there someone who can weigh in on that?

wking · 2017-04-10T23:47:39Z

On Mon, Apr 10, 2017 at 04:38:43PM -0700, Phil Estes wrote: As @coolljt0725 mentions, **arm** is one such architecture…

Are folks comfortable if ‘variant’ is only defined when ‘architecture’ is ‘arm’, and in that case we SHOULD understand the values for GOARM [1]? Or use @stevvooe's table suggestion [2]? Is GOARM sufficiently granular? To float some non-table wording, following the runtime-spec example would give something like: When `architecture` is `arm`, image indexes SHOULD use, and implementations SHOULD understand, `variant` entries listed in the Go Language document for GOARM. `variant` values for other `architecture`s are implementation-defined. If a variant is not included in the GOARM documentation, it SHOULD be submitted to this specification for standardization. [1]: https://golang.org/doc/install/source#environment [2]: #632 (comment)

AkihiroSuda · 2017-04-13T01:52:50Z

cc @jhowardmsft

Is Windows already using os.version and os.features?

Also, we need to make sure what is variant, as @wking described.
@estesp Is POWER already using variant?

lowenna · 2017-04-13T15:55:42Z

@jstarks (Microsoft image-spec maintainer) to confirm.

@AkihiroSuda I'm certain we need version on Windows. I don't believe there's a need for features at this point.

wking · 2017-04-13T16:26:10Z

On Thu, Apr 13, 2017 at 08:55:43AM -0700, John Howard wrote: @AkihiroSuda I'm certain we need version on Windows. I don't believe there's a need for features at this point.

Can you speak to the intended semantics of os.version? Are valid values ones that match the patterns listed in [1]? Is it aiming for an exact match (so for ‘"os.version": "10.0.10586"’ only that exact pattern matches)? Is it giving a minimum version (so for ‘"os.version": "10.0.10586"’, both 10.0.10586 and 11.0 would match)? Are implementatations expected to use the deprecated GetVersionEx [2,3]? Is there a non-deprecated alternative that they can use? [1]: https://msdn.microsoft.com/en-us/library/windows/desktop/ms724832%28v=vs.85%29.aspx [2]: https://msdn.microsoft.com/en-us/library/windows/desktop/ms724451%28v=vs.85%29.aspx [3]: #632 (comment)

estesp · 2017-04-13T16:51:18Z

@AkihiroSuda:

Also, we need to make sure what is variant, as @wking described.
@estesp Is POWER already using variant?

The current/modern use of POWER hardware with Linux is focused on the ppc64 architecture family, and specifically, latest distro support has focused on the "little endian" version of this architecture, which has a formal Linux and GOARCH name of ppc64le. Therefore, there is no need for variant with POWER as it stands today. Given I don't see any activity around containers for the embedded PPC family (in Go, or container runtimes), I'm not sure I even see a future use for variant for the overall RISC-family that would include the older ppc32 variants (like ppc4xx in Linux kernel terms).

With that said, I'm not sure of the value of having specific language that makes variant only viable when arch == arm, given it is hard to predict what architecture families may get popular in the near future? On the flip side, I understand a rev of the spec is possible were there to be a future need. Definitely variant is going to be most closely related to embedded architectures where CPU families are the first level of detail, and usually a second level of detail is required to know what binary code can run on that board, hence the GOARM solution for Go itself in this case.

wking · 2017-04-13T17:02:51Z

On Thu, Apr 13, 2017 at 09:51:19AM -0700, Phil Estes wrote: With that said, I'm not sure of the value of having specific language that makes `variant` only viable when arch == arm, given it is hard to predict what architecture families may get popular in the near future?

Some specs handle this with an out-of-spec registry, so you'd register your new value and not need to bump the spec itself. And the wording I'd floated in [1] leaves ‘variant’ implementation-defined outside of arm, so implementations can add new values as they see fit (documenting them locally) without having to wait for a spec/registry bump. [1]: #632 (comment)

lowenna · 2017-04-13T17:05:10Z

It would be a quadtuple such as below:

PS E:\go\src\github.com\docker\docker> docker images
REPOSITORY                    TAG                 IMAGE ID            CREATED             SIZE
microsoft/windowsservercore   10.0.14393.1066     5ad0f210c5f8        2 weeks ago         7.14GB
microsoft/windowsservercore   latest              5ad0f210c5f8        2 weeks ago         7.14GB
microsoft/nanoserver          10.0.14393.1066     a7e2fcd7133c        2 weeks ago         663MB
microsoft/nanoserver          latest              a7e2fcd7133c        2 weeks ago         663MB
PS E:\go\src\github.com\docker\docker> docker inspect microsoft/nanoserver -f "{{.OsVersion}}"
10.0.14393.1066
PS E:\go\src\github.com\docker\docker>

jstarks · 2017-04-17T17:53:21Z

We really do want to support os.version on Windows; as jhowardmsft mentions, this is our full OS version, major.minor.build.update.

This is important for us because the container image cross-version compatibility is not as good on Windows as it is on Linux since we do not have a long-term stable kernel/user ABI boundary. Windows will refuse to run container images with versions it does not support, so this version number is not used for this block. However, we think it's important to include this version in the image so that

Container orchestrators can ensure they are choosing image versions that are known to work with the running OS.
Users can easily see what version of the OS is in an image. This information could be extracted from the files inside the image, but that's much more expensive.

I'd love to avoid the need for this, but the reality of the Windows container architecture is that we do need os.version. I'd rather not defer this to post 1.0 if we can avoid it.

As for os.features, there is a similar problem here: a Microsoft Nano Server host cannot run a Windows Server Core container image because it is missing necessary kernel functionality (basically the kernel UI stack, as contained in the driver win32k.sys). However, a Windows Server Core host can run a Microsoft Nano Server container image. We tag the Windows Server Core images with an os.features setting of ["win32k"] to indicate that the image requires the kernel UI stack support.

I'd like for us to be able to continue tagging these images this way, but the truth is that we don't currently rely on this os.features setting in Docker. If we this it's best to defer this one until post 1.0, maybe that's OK.

wking · 2017-04-17T20:08:51Z

On Mon, Apr 17, 2017 at 10:53:22AM -0700, John Starks wrote: 1. Container orchestrators can ensure they are choosing image versions that are known to work with the running OS.

Given a value from os.version and a running OS, how will orchestrators decide whether the two are compatible? The deprecated `GetVersionEx` and some sort of SemVer-esque comparison [1]? Something else? [1]: #632 (comment)

We tag the Windows Server Core images with an `os.features` setting of `["win32k"]` to indicate that the image requires the kernel UI stack support.

Is there a registry of valid os.features values and their meanings somewhere? Otherwise interop between multiple Windows runtimes could be complicated. Or maybe there will only ever be one Windows runtime (although in that case, I don't see the point in specing that one runtime here)?

jstarks · 2017-04-18T23:50:01Z

Given a value from os.version and a running OS, how will orchestrators
decide whether the two are compatible? The deprecated GetVersionEx
and some sort of SemVer-esque comparison [1]? Something else?

Good catch. The simple reality Microsoft would like to preserve is that major.minor.build must match with the result of GetVersion. Yes, GetVersion(Ex) is deprecated for many/most use cases, but with a properly manifested app it can still be used to get the version number.

However, the actual reality is that Microsoft may need to break compatibility even with matching major.minor.build. To support this case, Windows has some additional registry keys on the host and inside the container image to specify the minimum update version that is supported on each side. Windows will look at this at container start time to determine whether it should attempt to launch the container. (Microsoft has not used this mechanism so far.)

So really, for an orchestrator to be able to accurately predict whether an image is compatible without having to first download the image and crack open the registry hives, this supported range information needs to be in the image format as well. So far we haven't added this to the Docker image format or proposed it for the OCI format, though.

Is there a registry of valid os.features values and their meanings
somewhere? Otherwise interop between multiple Windows runtimes could
be complicated. Or maybe there will only ever be one Windows runtime
(although in that case, I don't see the point in specing that one
runtime here)?

win32k is currently the only entry that is ever put in os.features in Windows images.

Your broader question is a good one -- it's not realistic for anyone else to build a Windows runtime that supports the style of Windows container images that Docker currently supports. However, I still think it's valuable to specify aspects of that runtime as part of this project since this is the best path to ensure that 1. multiple orchestrators work the same way on top of Windows and 2. Windows and Linux can use the same tools wherever possible.

wking · 2017-04-19T00:07:03Z

On Tue, Apr 18, 2017 at 04:50:03PM -0700, John Starks wrote: However, I still think it's valuable to specify aspects of that runtime as part of this project since this is the best path to ensure that 1. multiple orchestrators work the same way on top of Windows and 2. Windows and Linux can use the same tools wherever possible.

To your second point, I don't see why Linux users would care about valid values for os.version and os.features in Windows images. The fact that they share a consistent config format to allow the same fields/value where possible is nice, but these two properties seem well-separated. To your first point, I think the OCI could support the same amount of orchestrator consistency by saying: When ‘os’ is ‘windows’, the semantics of ‘os.version’ are implementation-defined. With something similar for os.features. And then somewhere (in OCI certification work?) linking to the Windows implementation and its docs. I'd also be ok putting that Windows-implementation doc link in the spec itself if we wanted to bake it in. It's not really a spec concern, but having a convenient link there for spec readers would be convenient if we expected the link to be stable enough to outlast that image spec's useful life.

jstarks · 2017-04-19T23:49:30Z

I think that's a reasonable approach for now. I don't have a Windows documentation link for these fields right now, unfortunately, but we can add that in a future change when one is available.

AkihiroSuda · 2017-04-24T08:27:56Z

Thank you all for the comment.

I split this PR to the following three PRs:
#650: variant
#651: os.version
#652: os.features

I'm closing this PR.

AkihiroSuda force-pushed the platform branch from 26d3b48 to e67faa6 Compare March 31, 2017 05:04

AkihiroSuda force-pushed the platform branch from e67faa6 to 24b8332 Compare March 31, 2017 05:09

AkihiroSuda commented Mar 31, 2017

View reviewed changes

wking reviewed Mar 31, 2017

View reviewed changes

stevvooe reviewed Mar 31, 2017

View reviewed changes

jonboulle mentioned this pull request Apr 3, 2017

image-index: Document Linux cpuinfo flags (on x86) for platform.features #631

Closed

AkihiroSuda mentioned this pull request Apr 4, 2017

specs-go/v1: unify the descriptor type #620

Merged

wking mentioned this pull request Apr 7, 2017

index.json: improve implementors note to better represent algorithms necessary #588

Closed

image-index: mark os.version, os.features and variant as "reserved"

1738745

These properties seems unclear and not suitable for standardization at the moment. Signed-off-by: Akihiro Suda <suda.akihiro@lab.ntt.co.jp>

AkihiroSuda force-pushed the platform branch from 24b8332 to 1738745 Compare April 10, 2017 09:18

AkihiroSuda changed the title ~~image-index: stipulate architecture-specific conventions~~ image-index: mark os.version, os.features and variant as "reserved" Apr 10, 2017

coolljt0725 reviewed Apr 10, 2017

View reviewed changes

wking mentioned this pull request Apr 13, 2017

improve definition of platform.features #622

Closed

This was referenced Apr 24, 2017

image-index: define platform.variant for arm #650

Merged

image-index: define platform.os.version on Windows #651

Merged

image-index: define platform.os.features on Windows #652

Merged

AkihiroSuda closed this Apr 24, 2017

kevpar mentioned this pull request Dec 10, 2021

[v1.6.0-beta.1] platforms.Normalize should not alter platforms.DefaultSpec containerd/containerd#6168

Closed

This was referenced Oct 30, 2023

Include OS version and features in OCI platforms moby/buildkit#4387

Merged

Suggestion: Deprecate and disrecommend Platform's os.features key. #1147

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

image-index: mark os.version, os.features and variant as "reserved" #632

image-index: mark os.version, os.features and variant as "reserved" #632

AkihiroSuda commented Mar 31, 2017 •

edited

Loading

AkihiroSuda commented Mar 31, 2017

AkihiroSuda Mar 31, 2017 •

edited

Loading

wking Mar 31, 2017

stevvooe Mar 31, 2017

wking Mar 31, 2017

stevvooe Mar 31, 2017

AkihiroSuda Mar 31, 2017 •

edited

Loading

wking Mar 31, 2017

stevvooe Mar 31, 2017 •

edited

Loading

wking Mar 31, 2017

wking commented Mar 31, 2017 via email

stevvooe Mar 31, 2017

AkihiroSuda Mar 31, 2017

stevvooe Mar 31, 2017

AkihiroSuda commented Apr 10, 2017

jonboulle commented Apr 10, 2017

coolljt0725 Apr 10, 2017

stevvooe commented Apr 10, 2017

estesp commented Apr 10, 2017

wking commented Apr 10, 2017 via email

AkihiroSuda commented Apr 13, 2017

lowenna commented Apr 13, 2017

wking commented Apr 13, 2017 via email

estesp commented Apr 13, 2017

wking commented Apr 13, 2017 via email

lowenna commented Apr 13, 2017

jstarks commented Apr 17, 2017

wking commented Apr 17, 2017 via email

jstarks commented Apr 18, 2017

wking commented Apr 19, 2017 via email

jstarks commented Apr 19, 2017

AkihiroSuda commented Apr 24, 2017

image-index: mark os.version, os.features and variant as "reserved" #632

image-index: mark os.version, os.features and variant as "reserved" #632

Conversation

AkihiroSuda commented Mar 31, 2017 • edited Loading

AkihiroSuda commented Mar 31, 2017

AkihiroSuda Mar 31, 2017 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

AkihiroSuda Mar 31, 2017 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

stevvooe Mar 31, 2017 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

wking commented Mar 31, 2017 via email

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

AkihiroSuda commented Apr 10, 2017

jonboulle commented Apr 10, 2017

Choose a reason for hiding this comment

stevvooe commented Apr 10, 2017

estesp commented Apr 10, 2017

wking commented Apr 10, 2017 via email

AkihiroSuda commented Apr 13, 2017

lowenna commented Apr 13, 2017

wking commented Apr 13, 2017 via email

estesp commented Apr 13, 2017

wking commented Apr 13, 2017 via email

lowenna commented Apr 13, 2017

jstarks commented Apr 17, 2017

wking commented Apr 17, 2017 via email

jstarks commented Apr 18, 2017

wking commented Apr 19, 2017 via email

jstarks commented Apr 19, 2017

AkihiroSuda commented Apr 24, 2017

AkihiroSuda commented Mar 31, 2017 •

edited

Loading

AkihiroSuda Mar 31, 2017 •

edited

Loading

AkihiroSuda Mar 31, 2017 •

edited

Loading

stevvooe Mar 31, 2017 •

edited

Loading