Switch buildkit containerd worker namespace to finch #71
Assignees
Comments
|
Thanks for the detailed bug report. This seems like a great addition to the finch.yaml config file. After reading your post, it does seem like defaulting to the containerd worker makes sense. |
|
I've been able to debug this one and get a workaround. It turns out Finch is already using a containerd worker for Buildkit, not an OCI one. However the containerd namespace that buildkit is using is set to The finch cli is using the containerd namespace I was able to get this working by switching the buildkit containerd worker to use the In the
to
And then restart buildkit. After this I was successfully able to build on top of previously built images. I will update the title of this issue to reflect the real issue. |
ollypom
changed the title
sam-berning
pushed a commit
that referenced
this issue
Jan 4, 2023
Signed-off-by: Olly Pomeroy <opomer@amazon.co.uk> Issue #, if available: #71 *Description of changes:* Adding the [env](/~https://github.com/lima-vm/lima/blob/master/examples/default.yaml#L345) section to the lima vm configuration, and the defining a CONTAINERND_NAMESPACE variable. The containerd-rootless-setup.sh script in nerdctl takes a CONTAINERND_NAMESPACE variable when configuring the buildkit containerd worker namespace. This variable is defined here in lima so that it is passed into the containerd-rootless-setup.sh script, used by the CIDATA scripts while provisioning the VM. This variable is passed into CIDATA [here](/~https://github.com/lima-vm/lima/blob/master/pkg/cidata/cidata.TEMPLATE.d/boot/40-install-containerd.sh#L88) and is used within the containerd-rootless-setup.sh script [here](/~https://github.com/containerd/nerdctl/blob/main/extras/rootless/containerd-rootless-setuptool.sh#L301). *Testing done:* Built locally and verified the Variable is being consumed. ``` $ buildctl --addr=unix:///run/user/504/buildkit-finch/buildkitd.sock debug workers -v ID: 34w45gxhe8bfdr76y9jxvdo16 Platforms: linux/arm64,linux/amd64 Labels: org.mobyproject.buildkit.worker.containerd.namespace: finch org.mobyproject.buildkit.worker.containerd.uuid: a6e1dad3-4843-4ab4-b595-a912b5a3e76f org.mobyproject.buildkit.worker.executor: containerd org.mobyproject.buildkit.worker.hostname: lima-finch org.mobyproject.buildkit.worker.network: host org.mobyproject.buildkit.worker.snapshotter: overlayfs GC Policy rule#0: All: false Filters: type==source.local,type==exec.cachemount,type==source.git.checkout Keep Duration: 48h0m0s Keep Bytes: 512MB GC Policy rule#1: All: false Keep Duration: 1440h0m0s Keep Bytes: 10GB GC Policy rule#2: All: false Keep Bytes: 10GB GC Policy rule#3: All: true Keep Bytes: 10GB ``` Also tested this solves the [issue](#71): ``` $ cat Dockerfile.one FROM debian RUN apt-get update $ cat Dockerfile.two FROM mydebian RUN apt-get install curl -y $ ../finch/_output/bin/finch build -t mydebian -f Dockerfile.one . [+] Building 0.9s (6/6) FINISHED => [internal] load .dockerignore 0.0s => => transferring context: 2B 0.0s => [internal] load build definition from Dockerfile.one 0.0s => => transferring dockerfile: 72B 0.0s => [internal] load metadata for docker.io/library/debian:latest 0.8s => [1/2] FROM docker.io/library/debian@sha256:c66c0e5dc607baefefda1d9e64a3b3a317e4189c540c8eac0c1a06186fe353a1 0.0s => => resolve docker.io/library/debian@sha256:c66c0e5dc607baefefda1d9e64a3b3a317e4189c540c8eac0c1a06186fe353a1 0.0s => CACHED [2/2] RUN apt-get update 0.0s => exporting to image 0.0s => => exporting layers 0.0s => => exporting manifest sha256:1ea8c95744126d9a451938b8db0fa7da10042f35e21f16a011361076975b34ba 0.0s => => exporting config sha256:bc9337720dfc88a9ee7388eb364c8b5bc7486b082577cda7d17f578fa2079c5a 0.0s => => naming to docker.io/library/mydebian:latest 0.0s => => unpacking to docker.io/library/mydebian:latest 0.0s $ ../finch/_output/bin/finch image ls REPOSITORY TAG IMAGE ID CREATED PLATFORM SIZE BLOB SIZE mydebian latest 1ea8c9574412 21 seconds ago linux/arm64 146.0 MiB 64.3 MiB $ ../finch/_output/bin/finch build -t mydebian2 -f Dockerfile.two . [+] Building 4.6s (6/6) FINISHED => [internal] load .dockerignore 0.0s => => transferring context: 2B 0.0s => [internal] load build definition from Dockerfile.two 0.0s => => transferring dockerfile: 83B 0.0s => [internal] load metadata for docker.io/library/mydebian:latest 0.9s => CACHED [1/2] FROM docker.io/library/mydebian@sha256:1ea8c95744126d9a451938b8db0fa7da10042f35e21f16a011361076975b34ba 0.5s => => resolve docker.io/library/mydebian@sha256:1ea8c95744126d9a451938b8db0fa7da10042f35e21f16a011361076975b34ba 0.5s => [2/2] RUN apt-get install curl -y 2.7s => exporting to image 0.4s => => exporting layers 0.3s => => exporting manifest sha256:b3110f4faff6ac2334d9989c7dab34cb288d03ecb6ef6bd83c2eb02255e39e44 0.0s => => exporting config sha256:570d3e1d79d7ecc8bb54fbb9603c216b39489f036d703fc76891b566a9c358ec 0.0s => => naming to docker.io/library/mydebian2:latest 0.0s => => unpacking to docker.io/library/mydebian2:latest 0.1s ``` - [ X ] I've reviewed the guidance in CONTRIBUTING.md #### License Acceptance By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license. Signed-off-by: Olly Pomeroy <opomer@amazon.co.uk>
ahsan-z-khan
pushed a commit
to ahsan-z-khan/finch
that referenced
this issue
Jan 11, 2023
Signed-off-by: Olly Pomeroy <opomer@amazon.co.uk> Issue #, if available: runfinch#71 *Description of changes:* Adding the [env](/~https://github.com/lima-vm/lima/blob/master/examples/default.yaml#L345) section to the lima vm configuration, and the defining a CONTAINERND_NAMESPACE variable. The containerd-rootless-setup.sh script in nerdctl takes a CONTAINERND_NAMESPACE variable when configuring the buildkit containerd worker namespace. This variable is defined here in lima so that it is passed into the containerd-rootless-setup.sh script, used by the CIDATA scripts while provisioning the VM. This variable is passed into CIDATA [here](/~https://github.com/lima-vm/lima/blob/master/pkg/cidata/cidata.TEMPLATE.d/boot/40-install-containerd.sh#L88) and is used within the containerd-rootless-setup.sh script [here](/~https://github.com/containerd/nerdctl/blob/main/extras/rootless/containerd-rootless-setuptool.sh#L301). *Testing done:* Built locally and verified the Variable is being consumed. ``` $ buildctl --addr=unix:///run/user/504/buildkit-finch/buildkitd.sock debug workers -v ID: 34w45gxhe8bfdr76y9jxvdo16 Platforms: linux/arm64,linux/amd64 Labels: org.mobyproject.buildkit.worker.containerd.namespace: finch org.mobyproject.buildkit.worker.containerd.uuid: a6e1dad3-4843-4ab4-b595-a912b5a3e76f org.mobyproject.buildkit.worker.executor: containerd org.mobyproject.buildkit.worker.hostname: lima-finch org.mobyproject.buildkit.worker.network: host org.mobyproject.buildkit.worker.snapshotter: overlayfs GC Policy rule#0: All: false Filters: type==source.local,type==exec.cachemount,type==source.git.checkout Keep Duration: 48h0m0s Keep Bytes: 512MB GC Policy rule#1: All: false Keep Duration: 1440h0m0s Keep Bytes: 10GB GC Policy rule#2: All: false Keep Bytes: 10GB GC Policy rule#3: All: true Keep Bytes: 10GB ``` Also tested this solves the [issue](runfinch#71): ``` $ cat Dockerfile.one FROM debian RUN apt-get update $ cat Dockerfile.two FROM mydebian RUN apt-get install curl -y $ ../finch/_output/bin/finch build -t mydebian -f Dockerfile.one . [+] Building 0.9s (6/6) FINISHED => [internal] load .dockerignore 0.0s => => transferring context: 2B 0.0s => [internal] load build definition from Dockerfile.one 0.0s => => transferring dockerfile: 72B 0.0s => [internal] load metadata for docker.io/library/debian:latest 0.8s => [1/2] FROM docker.io/library/debian@sha256:c66c0e5dc607baefefda1d9e64a3b3a317e4189c540c8eac0c1a06186fe353a1 0.0s => => resolve docker.io/library/debian@sha256:c66c0e5dc607baefefda1d9e64a3b3a317e4189c540c8eac0c1a06186fe353a1 0.0s => CACHED [2/2] RUN apt-get update 0.0s => exporting to image 0.0s => => exporting layers 0.0s => => exporting manifest sha256:1ea8c95744126d9a451938b8db0fa7da10042f35e21f16a011361076975b34ba 0.0s => => exporting config sha256:bc9337720dfc88a9ee7388eb364c8b5bc7486b082577cda7d17f578fa2079c5a 0.0s => => naming to docker.io/library/mydebian:latest 0.0s => => unpacking to docker.io/library/mydebian:latest 0.0s $ ../finch/_output/bin/finch image ls REPOSITORY TAG IMAGE ID CREATED PLATFORM SIZE BLOB SIZE mydebian latest 1ea8c9574412 21 seconds ago linux/arm64 146.0 MiB 64.3 MiB $ ../finch/_output/bin/finch build -t mydebian2 -f Dockerfile.two . [+] Building 4.6s (6/6) FINISHED => [internal] load .dockerignore 0.0s => => transferring context: 2B 0.0s => [internal] load build definition from Dockerfile.two 0.0s => => transferring dockerfile: 83B 0.0s => [internal] load metadata for docker.io/library/mydebian:latest 0.9s => CACHED [1/2] FROM docker.io/library/mydebian@sha256:1ea8c95744126d9a451938b8db0fa7da10042f35e21f16a011361076975b34ba 0.5s => => resolve docker.io/library/mydebian@sha256:1ea8c95744126d9a451938b8db0fa7da10042f35e21f16a011361076975b34ba 0.5s => [2/2] RUN apt-get install curl -y 2.7s => exporting to image 0.4s => => exporting layers 0.3s => => exporting manifest sha256:b3110f4faff6ac2334d9989c7dab34cb288d03ecb6ef6bd83c2eb02255e39e44 0.0s => => exporting config sha256:570d3e1d79d7ecc8bb54fbb9603c216b39489f036d703fc76891b566a9c358ec 0.0s => => naming to docker.io/library/mydebian2:latest 0.0s => => unpacking to docker.io/library/mydebian2:latest 0.1s ``` - [ X ] I've reviewed the guidance in CONTRIBUTING.md #### License Acceptance By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license. Signed-off-by: Olly Pomeroy <opomer@amazon.co.uk>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
What is the problem you're trying to solve?.
buildkit can either leverage a containerd worker or an OCI worker, currently the Finch default is the OCI worker. This can be frustrating as images stored within the containerd image store can not be used for builds.
Describe the feature you'd like
The finch default flipped to the containerd worker in the buildkit.toml file, therefore unblocking this common workflow. Lima has already switched to the containerd worker.
I'm not aware of a current workaround, as I don't think there is a way to customize the
buildkit.tomlfile viafinch.yaml.The text was updated successfully, but these errors were encountered: