Add lifecycle for containers #231

mrunalp · 2015-10-22T23:54:36Z

The lifecycle described is generic and should apply to all platforms.
It provides leeway for the runtimes to be flexible in how they
implement it.

Signed-off-by: Mrunal Patel mrunalp@gmail.com

Thanks to @crosbymichael for the feedback!

vishh · 2015-10-23T00:38:23Z

runtime.md

+
+* OCI compliant runtime is invoked by passing the bundle path as argument.
+* The container's runtime environment is created.
+* The prestart hooks are invoked in the host environment by the runtime.


@mrunalp: If we split creation from start, then why can't users directly invoke the hooks?
I'm thinking something on the lines of

./oci-impl create foo <spec> ./my-pre-start-hook $(oci-impl state foo) ./oci-impl start foo ./my-post-start-hook $(oci-impl state foo) ./oci-impl stop foo # or my container exits / dies ./my-post-stop-hook $(oci-impl state foo)

@vishh This is just a timeline of container lifecycle. For some runtimes like runc, this would be an internal implementation detail of what happens during start. I think we can tackle whether we want a separate create when we discuss the actions/API/interface.

@mrunalp: But isn't that part of the lifecycle? When I think of a container in the OCI context, I assume this PR will describe its lifecycle.
Can you explain why you think this is an implementation detail?

mrunalp · 2015-10-23T17:23:26Z

Updated.

wking · 2015-10-23T17:38:46Z

runtime.md

+2. The container's runtime environment is created.
+3. The container's state is written to the filesystem.
+4. The prestart hooks are invoked by the runtime.
+   If any prestart hook fails, then the container is killed (step 8) and an error code is returned.


I think this is just “the container is killed and the lifecycle continues at step 8”. Actually killing the container would be part of step 7 (where you talk about signaling the container), and the rest of the activity after step 8 would happen in the “failed pre-start hook” case too.

mrunalp · 2015-10-27T16:50:04Z

@philips @vishh @vbatts PTAL

mrunalp · 2015-10-27T16:50:15Z

@LK4D4 PTAL

duglin · 2015-10-28T14:31:38Z

runtime.md

+   The container could also error out or crash.
+8. The container is destroyed by undoing the steps performed during create phase if required.
+9. The poststop hooks are invoked by the runtime and errors, if any, are logged.
+10. The state associated with the container is removed and the return code of the container is returned or logged.


s/of the container/of the container's user specific process/

s/state/state file (state.json)/

While we're tweaking the text, both of @duglin's suggestions look good to me too.

By state I assume you are only referring to the internal state. Any changes that the spawned processed introduced to the root filesystem will not be cleaned up right?

On Tue, Nov 17, 2015 at 05:41:08PM -0800, Vish Kannan wrote:

+10. The state associated with the container is removed and the
return code of the container is returned or logged.

By state I assume you are only referring to the internal
state. Any changes that the spawned processed introduced to the
root filesystem will not be cleaned up right?

+1 to not cleaning up bundle-side filesystem changes, and another
reason why @duglin's state.json clarification is useful.

Updated state to state.json

duglin · 2015-10-28T14:40:45Z

In order to avoid things getting out of sync, and duplicating information, I think it would make sense to combine the lifecycle information here with the definition of our ops, from runtime.md. Unless there's a reason to keep them separated? But they seem pretty tightly linked to me, at least until we get to the point where "ops" are exposed via some API/cmd-line spec, which we're not doing yet.

duglin · 2015-10-28T14:52:09Z

In particular, I think it would make sense to pull in the lower-level details mentioned in #225. So each step could have a one-liner summary followed but a more detailed list of the technical details. Seeing it all in one spot keeps it simple, aligned and easier to follow.

wking · 2015-10-28T18:11:15Z

On Wed, Oct 28, 2015 at 07:52:11AM -0700, Doug Davis wrote:

In particular, I think it would make sense to pull in the
lower-level details mentioned in #225. So each step could have a
one-liner summary followed but a more detailed list of the technical
details. Seeing it all in one spot keeps it simple, aligned and
easier to follow.

I like the approach with a one-liner overview linking out to more
details for each step (which is how I setup #207 1). That makes it
easy to get the big picture and still drill down into the individual
entries for more detail.

crosbymichael · 2015-11-09T22:20:53Z

LGTM

mrunalp · 2015-11-09T22:25:01Z

@vishh @LK4D4 PTAL. As discussed in the call last week, we can start with something simple here that we can refine.

vishh · 2015-11-18T00:20:55Z

runtime.md

-Process is killed from the outside.
-
-This event needs to be captured by runc to run onstop event handlers.
+The lifecycle describes the timeline of events that happen from when a container is started to when it is stopped or crashes unexpectedly.


nit: to when it ceases to exist maybe?

nit: How about container is created instead of started?

On Tue, Nov 17, 2015 at 04:21:21PM -0800, Vish Kannan wrote:

-This event needs to be captured by runc to run onstop event handlers.
+The lifecycle describes the timeline of events that happen from when a container is started to when it is stopped or crashes unexpectedly.

nit: How about container is created instead of started?

Both of Vish's suggestions sound like improvements to me.

vishh · 2015-11-18T01:15:01Z

runtime.md

+1. OCI compliant runtime is invoked by passing the bundle path as argument.
+2. The container's runtime environment is created.
+3. The container's state is written to the filesystem.
+4. The prestart hooks are invoked by the runtime.


This step is subject to change based on namespaces pinning POC right? Shall we explicitly mention that?

There is a note at the bottom to this effect.

mrunalp · 2015-11-20T23:21:09Z

Pushed updates to address comments.

wking · 2015-11-20T23:25:24Z

runtime.md

+1. OCI compliant runtime is invoked by passing the bundle path as argument.
+2. The container's runtime environment is created according to the configuration in config.json.
+   Any updates to config.json after container is running do not affect the container.
+3. The container's state.json file is written to the filesystem under /run/opencontainer/<runtime>/containers/<id>/.


We should not specify the path here (just stop after “to the filesystem”). See @vbatts here in response to a comment on an outdated diff.

wking · 2015-11-20T23:29:33Z

runtime.md

+5. The user specified process is executed in the container.
+6. The poststart hooks are invoked by the runtime.
+   If any poststart hook fails, then the container is killed and the lifecycle continues at step 8.
+7. Additional actions such as pausing the container, resuming the container or signaling the container may be performed using the runtime interface.


With ec85489 → bd549a2 you dropped “checkpointing the container”, but left in pause/resume/signalling. @vishh didn't call those out in his original comment, but they also seem like something we'd want to address after landing a simpler start/cleanup lifecycle. As it stands, this is the only reference to pausing in the spec, and it doesn't look detailed enough to be useful.

runtime interface --> runtime interfaces?

@hqhq interface would support multiple operations so I think singular form is fine here.

The lifecycle described is generic and should apply all platforms. It provides leeway for the runtimes to be flexible in how they implement it. Signed-off-by: Mrunal Patel <mrunalp@gmail.com>

wking · 2015-12-04T21:36:49Z

runtime.md

+5. The user specified process is executed in the container.
+6. The poststart hooks are invoked by the runtime.
+   If any poststart hook fails, then the container is stopped and the lifecycle continues at step 8.
+7. Additional actions such as pausing the container, resuming the container or signaling the container may be performed using the runtime interface.


I still think we want to drop pause/resume/signal until we have more clarity on how they will work. For more background here, especially around complications with shared state on Linux, see wking/oci-command-line-api#3. Alternatively, if we already have a clear idea of how these will work in the absence of /run/opencontainer, I'd appreciate some comments in the CLI-spec PR explaining what the CLI should look like.

dqminh · 2015-12-08T15:08:26Z

LGTM

I think this is a great first step, we can further discuss the details for each individual actions, but the overall flow and structure looks right.

vbatts · 2015-12-09T18:03:26Z

LGTM

Add lifecycle for containers

As discussed earlier [1,2]. I'm in favor of rolling it back into config.json [3], but we aren't there yet [4]. [1]: opencontainers#231 (comment) [2]: /~https://github.com/opencontainers/specs/pull/231/files#r46735828 [3]: https://groups.google.com/a/opencontainers.org/forum/#!topic/dev/0QbyJDM9fWY Subject: Single, unified config file (i.e. rolling back specs#88) Date: Wed, 4 Nov 2015 09:53:20 -0800 Message-ID: <20151104175320.GC24652@odin.tremily.us> [4]: /~https://github.com/opencontainers/specs/blob/4a63e81a807edec3d67ed2b6bd99a6c2b288676f/bundle.md#container-format Signed-off-by: W. Trevor King <wking@tremily.us>

Initial work landed in [1], but it doesn't get into the order of the setup process or explain how pause/resume/signalling work [2]. If we don't figure these things out ahead of time, I expect we'll have to figure them out in order to land compliance testing. [1]: opencontainers/runtime-spec#231 [2]: /~https://github.com/opencontainers/specs/pull/231/files#r46735313

I think this is resolved since 7713efc (Add lifecycle for containers, 2015-10-22, opencontainers#231), which was committed on the same day as the ROADMAP entry (4859f6d, Add initial roadmap, 2015-10-22, opencontainers#230). Signed-off-by: W. Trevor King <wking@tremily.us>

# digest/hashing target Most of this has spun off with [1], and I haven't heard of anyone talking about verifying the on-disk filesystem in a while. My personal take is on-disk verification doesn't add much over serialized verification unless you have a local attacker (or unreliable disk), and you'll need some careful threat modeling if you want to do anything productive about the local attacker case. For some more on-disk verification discussion, see the thread starting with [2]. # distributable-format target This spun off with [1]. # lifecycle target I think this is resolved since 7713efc (Add lifecycle for containers, 2015-10-22, opencontainers#231), which was committed on the same day as the ROADMAP entry (4859f6d, Add initial roadmap, 2015-10-22, opencontainers#230). # container-action target Addressed by 7117ede (Expand on the definition of our ops, 2015-10-13, opencontainers#225), although there has been additional discussion in a7a366b (Remove exec from required runtime functionalities, 2016-04-19, opencontainers#388) and 0430aaf (Split create and start, 2016-04-01, opencontainers#384). # validation and testing targets Validation is partly covered by cdcabde (schema: JSON Schema and validator for `config.json`, 2016-01-19, opencontainers#313) and subequent JSON Schema work. The remainder of these targets are handled by ocitools [3]. # printable/compiled-spec target The bulk of this was addressed by 4ee036f (*: printable documents, 2015-12-09, opencontainers#263). Any remaining polishing of that workflow seems like a GitHub-issue thing and not a ROADMAP thing. And publishing these to opencontainers.org certainly seems like it's outside the scope of this repository (although I think that such publishing is a good idea). [1]: /~https://github.com/opencontainers/image-spec [2]: https://groups.google.com/a/opencontainers.org/d/msg/dev/xo4SQ92aWJ8/NHpSQ19KCAAJ Subject: OCI Bundle Digests Summary Date: Wed, 14 Oct 2015 17:09:15 +0000 Message-ID: <CAD2oYtN-9yLLhG_STO3F1h58Bn5QovK+u3wOBa=t+7TQi-hP1Q@mail.gmail.com> [3]: /~https://github.com/opencontainers/ocitools Signed-off-by: W. Trevor King <wking@tremily.us>

As discussed earlier [1,2]. I'm in favor of rolling it back into config.json [3], but we aren't there yet [4]. [1]: opencontainers#231 (comment) [2]: /~https://github.com/opencontainers/specs/pull/231/files#r46735828 [3]: https://groups.google.com/a/opencontainers.org/forum/#!topic/dev/0QbyJDM9fWY Subject: Single, unified config file (i.e. rolling back specs#88) Date: Wed, 4 Nov 2015 09:53:20 -0800 Message-ID: <20151104175320.GC24652@odin.tremily.us> [4]: /~https://github.com/opencontainers/specs/blob/4a63e81a807edec3d67ed2b6bd99a6c2b288676f/bundle.md#container-format Signed-off-by: W. Trevor King <wking@tremily.us>

# digest/hashing target Most of this has spun off with [1], and I haven't heard of anyone talking about verifying the on-disk filesystem in a while. My personal take is on-disk verification doesn't add much over serialized verification unless you have a local attacker (or unreliable disk), and you'll need some careful threat modeling if you want to do anything productive about the local attacker case. For some more on-disk verification discussion, see the thread starting with [2]. # distributable-format target This spun off with [1]. # lifecycle target I think this is resolved since 7713efc (Add lifecycle for containers, 2015-10-22, opencontainers#231), which was committed on the same day as the ROADMAP entry (4859f6d, Add initial roadmap, 2015-10-22, opencontainers#230). # container-action target Addressed by 7117ede (Expand on the definition of our ops, 2015-10-13, opencontainers#225), although there has been additional discussion in a7a366b (Remove exec from required runtime functionalities, 2016-04-19, opencontainers#388) and 0430aaf (Split create and start, 2016-04-01, opencontainers#384). # validation and testing targets Validation is partly covered by cdcabde (schema: JSON Schema and validator for `config.json`, 2016-01-19, opencontainers#313) and subequent JSON Schema work. The remainder of these targets are handled by ocitools [3]. # printable/compiled-spec target The bulk of this was addressed by 4ee036f (*: printable documents, 2015-12-09, opencontainers#263). Any remaining polishing of that workflow seems like a GitHub-issue thing and not a ROADMAP thing. And publishing these to opencontainers.org certainly seems like it's outside the scope of this repository (although I think that such publishing is a good idea). [1]: /~https://github.com/opencontainers/image-spec [2]: https://groups.google.com/a/opencontainers.org/d/msg/dev/xo4SQ92aWJ8/NHpSQ19KCAAJ Subject: OCI Bundle Digests Summary Date: Wed, 14 Oct 2015 17:09:15 +0000 Message-ID: <CAD2oYtN-9yLLhG_STO3F1h58Bn5QovK+u3wOBa=t+7TQi-hP1Q@mail.gmail.com> [3]: /~https://github.com/opencontainers/ocitools Signed-off-by: W. Trevor King <wking@tremily.us>

My pull request was rejected on 2015-10-12 [1], but Mrunal filed an alternative which landed on 2015-12-09 [2]. [1]: opencontainers/runtime-spec#207 (comment) [2]: opencontainers/runtime-spec#231 (comment)

vishh reviewed Oct 23, 2015
View reviewed changes

mrunalp force-pushed the lifecycle branch from 72fa308 to afdd2ee Compare October 23, 2015 17:22

wking reviewed Oct 23, 2015
View reviewed changes

mrunalp force-pushed the lifecycle branch from afdd2ee to c5391b9 Compare October 23, 2015 19:19

duglin reviewed Oct 28, 2015
View reviewed changes

hqhq mentioned this pull request Nov 17, 2015

use the lifecycle states to define the hooks #250

Closed

vishh reviewed Nov 18, 2015
View reviewed changes

mrunalp force-pushed the lifecycle branch from c5391b9 to ec85489 Compare November 18, 2015 00:32

vishh reviewed Nov 18, 2015
View reviewed changes

vishh self-assigned this Nov 18, 2015

mrunalp force-pushed the lifecycle branch 2 times, most recently from 5d07ec8 to bd549a2 Compare November 20, 2015 23:20

wking reviewed Nov 20, 2015
View reviewed changes

mrunalp force-pushed the lifecycle branch from bd549a2 to 2ca0be2 Compare December 4, 2015 20:51

Add lifecycle for containers

7713efc

The lifecycle described is generic and should apply all platforms. It provides leeway for the runtimes to be flexible in how they implement it. Signed-off-by: Mrunal Patel <mrunalp@gmail.com>

mrunalp force-pushed the lifecycle branch from 2ca0be2 to 7713efc Compare December 4, 2015 20:52

wking referenced this pull request in mrunalp/specs Dec 4, 2015

Add lifecycle for containers

bd549a2

The lifecycle described is generic and should apply all platforms. It provides leeway for the runtimes to be flexible in how they implement it. Signed-off-by: Mrunal Patel <mrunalp@gmail.com>

wking reviewed Dec 4, 2015
View reviewed changes

wking mentioned this pull request Dec 5, 2015

Support Lifecycle Hooks #20

Closed

vbatts added a commit that referenced this pull request Dec 9, 2015

Merge pull request #231 from mrunalp/lifecycle

82cfe56

Add lifecycle for containers

vbatts merged commit 82cfe56 into opencontainers:master Dec 9, 2015

wking mentioned this pull request Dec 11, 2015

lifecycle: Mention runtime.json #268

Merged

wking mentioned this pull request Dec 11, 2015

lifecycle: Don't require /run/opencontainer/<runtime>/containers #269

Merged

This was referenced Dec 17, 2015

start: Add a --state option wking/oci-command-line-api#14

Closed

config-linux: Specify host mount namespace for namespace paths #275

Merged

wking mentioned this pull request Jan 5, 2016

Expand on the definition of our ops #225

Merged

wking mentioned this pull request Mar 16, 2016

Exec semantics #345

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add lifecycle for containers #231

Add lifecycle for containers #231

mrunalp commented Oct 22, 2015

vishh Oct 23, 2015

mrunalp Oct 23, 2015

vishh Oct 23, 2015

mrunalp commented Oct 23, 2015

wking Oct 23, 2015

mrunalp commented Oct 27, 2015

mrunalp commented Oct 27, 2015

duglin Oct 28, 2015

duglin Oct 28, 2015

wking Nov 18, 2015

vishh Nov 18, 2015

wking Nov 18, 2015

mrunalp Nov 20, 2015

duglin commented Oct 28, 2015

duglin commented Oct 28, 2015

wking commented Oct 28, 2015

crosbymichael commented Nov 9, 2015

mrunalp commented Nov 9, 2015

vishh Nov 18, 2015

vishh Nov 18, 2015

wking Nov 18, 2015

mrunalp Nov 18, 2015

vishh Nov 18, 2015

mrunalp Nov 20, 2015

mrunalp commented Nov 20, 2015

wking Nov 20, 2015

wking Nov 20, 2015

hqhq Dec 3, 2015

mrunalp Dec 4, 2015

wking Dec 4, 2015

dqminh commented Dec 8, 2015

vbatts commented Dec 9, 2015

Add lifecycle for containers #231

Add lifecycle for containers #231

Conversation

mrunalp commented Oct 22, 2015

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mrunalp commented Oct 23, 2015

Choose a reason for hiding this comment

mrunalp commented Oct 27, 2015

mrunalp commented Oct 27, 2015

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

duglin commented Oct 28, 2015

duglin commented Oct 28, 2015

wking commented Oct 28, 2015

crosbymichael commented Nov 9, 2015

mrunalp commented Nov 9, 2015

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mrunalp commented Nov 20, 2015

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

dqminh commented Dec 8, 2015

vbatts commented Dec 9, 2015