FIPS Image - Correct Configuration for User Data in Non-FIPS Regions

[aydosman]

I hope this won’t be a complicated issue—more of a clarification due to the documentation being out of date. We use Cluster AutoScaler and Karpenter with EKS on various modern Kubernetes versions (1.30/1.31 etc.) and run Bottlerocket OS.

We are interested in testing the FIPS variant in the eu-west-1 region. The problem I’m encountering is determining the precise user data configuration required to avoid using the FIPS endpoints.

I came across this PR and later discovered this discussion.

Image I'm using:
Latest Bottlerocket OS (FIPS Variant) in eu-west-1

What I expected to happen:
The node should boot up and successfully join the EKS cluster.

What actually happened:
The node boots but fails to join the cluster. Logs indicate issues with retrieving the private DNS name from EC2 and failures in initializing Kubernetes settings.

Relevant logs

[  OK  ] Finished User-specified setting generators.
         Starting Generate additional settings for Kubernetes...
[FAILED] Failed to start Send a Metricdog Ping.
See 'systemctl status metricdog.service' for details.
[  306.571770] pluto[1573]: Timed out retrieving private DNS name from EC2: deadline has elapsed
[FAILED] Failed to start Generate additional settings for Kubernetes.
See 'systemctl status pluto.service' for details.
[DEPEND] Dependency failed for Applies settings to create config files.
[DEPEND] Dependency failed for Send signal to CloudFormation Stack.
[DEPEND] Dependency failed for Bottlerocket initial configuration complete.
[DEPEND] Dependency failed for Isolates configured.target.
[DEPEND] Dependency failed for Sets the hostname.
[DEPEND] Dependency failed for Bootstrap Commands.

Unmodified UserData Example

[settings.kubernetes]
"cluster-name" = "foo-eks-foo"
"api-server" = "https://foo.foo.eu-west-1.eks.amazonaws.com"
"cluster-certificate" = "foo"
"cluster-dns-ip" = "0.0.0.0"

[settings.kubernetes.kube-reserved]
<removed>

[settings.kubernetes.node-labels]
<removed>

[settings.kubernetes.node-taints]
<removed>

Does anyone have insights on how to correctly configure userdata?

[ginglis13]

Hi @aydosman thank you for the issue and your patience as we work to update our documentation regarding this.

In the Bottlerocket FIPS variants, interactions between system services and AWS services will occur through AWS FIPS endpoints but only in the regions where EC2, EKS and ECS are supported (consult the latest AWS FIPS Endpoints By Service for details for each service).

Bottlerocket FIPS variants are available all AWS and AWS GovCloud regions. However, in regions without FIPS support (those outside of the US and Canada like eu-west-1), you must opt-out from using AWS FIPS endpoints by updating the settings.aws.confg API with the encoded AWS config to disable AWS FIPS endpoints. You can use the following user data in your nodes to opt-out from using FIPS endpoints in AWS API calls.

[settings.aws]
config = "W2RlZmF1bHRdCnVzZV9maXBzX2VuZHBvaW50PWZhbHNl"

The base64 string corresponds to the following configuration:

[default]
use_fips_endpoint=false