cmd/k8s-operator,docs/k8s: run tun mode proxies in privileged containers #14262
+36
−41
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
tomhjp
approved these changes
Dec 2, 2024
We were previously relying on unintended behaviour by runc where all containers where by default given read/write/mknod permissions for tun devices. This behaviour was removed in opencontainers/runc#3468 and released in runc 1.2. Containerd container runtime, used by Docker and majority of Kubernetes distributions bumped runc to 1.2 in 1.7.24 /~https://github.com/containerd/containerd/releases/tag/v1.7.24 thus breaking our reference tun mode Tailscale Kubernetes manifests and Kubernetes operator proxies. This PR changes the all Kubernetes container configs that run Tailscale in tun mode to privileged. This should not be a breaking change because all these containers would run in a Pod that already has a privileged init container. Updates #14256 Updates #10814 Signed-off-by: Irbe Krumina <irbe@tailscale.com>
Good catch, updated
It is non-breaking so changelog should suffice + will update docs |
This was referenced Dec 11, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
We were previously relying on unintended behaviour by runc where all containers where by default given read/write/mknod permissions for tun devices.
This behaviour was removed in opencontainers/runc#3468 and released in runc 1.2.
Containerd container runtime, used by Docker and majority of Kubernetes distributions bumped runc to 1.2 in 1.7.24 /~https://github.com/containerd/containerd/releases/tag/v1.7.24 thus breaking our reference tun mode Tailscale Kubernetes manifests and Kubernetes operator proxies.
This PR changes the all Kubernetes container configs that run Tailscale in tun mode to privileged. This should not be a breaking change because all these containers would run in a Pod that already has a privileged init container.
Alternative would be to delegate tun device creation to a device plugin like described in #10814 (comment).
However, this would mean an additional prerequisite and dependency on a third party tool to get started with the operator proxies- it seems like a better way is to make the proxies privileged by default to maintain the ease of installation/quick start setup. We can then document that users can alternatively deploy the device plugin and remove privileged context from the Tailscale containers.
I have tested this with a GKE 1.31.3 cluster + manually bumped containerd to v1.74 and runc to 1.2 as well as with Talos 1.8 that bundles containerd v2.0
Long term, we should look into whether these proxies can be run in netstack mode.
Updates #14256
Updates #10814
Updates #14149