Support configuration using MAC instead of device name #2622

zmrow · 2022-12-01T20:15:30Z

Issue number:
Closes #2293

Description of changes:
This PR adds the ability for a user to use MAC address (something like f8:75:a4:d5:32:64) instead of the interface name (eno) when configuring interfaces. The commits messages are detailed and a summary provided below.

The PR adds a new type InterfaceId which is an enum holding either a name or MAC address. The new MacAddress type validates the string given is a valid MAC address. The InterfaceId type is used as the key in the main map of ID > interface config. Keeping the InterfaceName type is significant, as it will still be used as the key for bonds/vlans once #2596 is merged, since configuring those things using MAC isn't possible/desirable.

In Bottlerocket, the concept of the "primary" interface is important. We use this interface's IP when registering with kubelet, to determine DNS settings when using DHCP, and also set some default sysctls on this interface. However, at the time we are generating network config, udev may still be asynchronously renaming interfaces so there isn't a reliable or safe way to map a MAC to an interface name. Later in the boot process (after interfaces have been setup by wicked and have leases), when interface name is necessary we can map a MAC address to a name using sysfs (/sys/class/net/*). When using a MAC as the identifier for the primary interface, we write a new file primary_mac_address to disk instead of primary_interface. We use these files later to determine the primary interface name. Under the hood if a MAC is used, it is correlated with sysfs to determine the interface name.

One significant change this causes is that we are now setting default sysctls in the wicked "install" helper, rather than using a separate service to do this. Previously the service ran during network-pre, at which point we cant' correlate a MAC to a name. Setting the sysctls in the helper provides a useful synchronization point where we know that our interfaces have a lease and a more stable name. This helper is called when setting up new interfaces (static and DHCP) and when DHCP leases are renewed.

A simple example of using MAC, also described in the documentation:

["0e:b3:69:44:b6:33"]
dhcp4 = true

Testing done:
For each of the tests below, I made sure /etc/resolv.conf was properly written, current_ip existed, and either primary_interface or primary_mac_address had been written. Also verified the sysctls were set properly for the primary interface.

Boot an aws-k8s-1.22 node without any customization - it joins the cluster and runs pods just fine.
Boot an aws-k8s-1.22 node into an IPv6-enabled EKS cluster and ensure the node receives an ipv6 address (validating the IPv6 sysctl accept_ra is still working)
Boot an metal-dev variant using MAC address to set up the primary interface for DHCP. Also tested using MAC address with a statically configured setup.

version = 3
["3c:ec:ef:79:ae:ac"]
dhcp4 = true

Terms of contribution:

By submitting this pull request, I agree that this contribution is dual-licensed under the terms of both the Apache License, version 2.0, and the MIT license.

PROVISIONING-METAL.md

sources/api/netdog/src/interface_id.rs

sources/api/netdog/src/net_config/v3.rs

sources/api/netdog/src/wicked/mod.rs

zmrow · 2022-12-14T20:05:04Z

^ Rebased on develop and fixup changes related to the bonding/vlan PR

PROVISIONING-METAL.md

yeazelm

Great work!

PROVISIONING-METAL.md

sources/api/netdog/src/net_config/test_macros/bonding.rs

sources/api/netdog/src/net_config/v3.rs

sources/api/netdog/src/wicked/mod.rs

sources/api/netdog/test_data/wicked/net_config.toml

markusboehme · 2023-01-03T20:46:50Z

I have yet to look at the code, just wondering from the change description:

One significant change this causes is that we are now setting default sysctls in the wicked DHCP helper, rather than using a separate service to do this. Previously the service ran during network-pre, at which point we cant' correlate a MAC to a name. Setting the sysctls in the DHCP helper provides a useful synchronization point where we know that our interfaces have a lease and a more stable name.

Does this mean sysctls won't be applied for a statically configured interface that is identified by its MAC address?

zmrow · 2023-01-04T20:38:32Z

@markusboehme

Does this mean sysctls won't be applied for a statically configured interface that is identified by its MAC address?

The helper mentioned is actually an "install" helper called by wicked anytime it brings up an interface, static or not. So to answer your question, the sysctls will be set for both dhcp and statically configured interfaces. (I'll update that wording in the change description as well.)

zmrow · 2023-01-05T23:27:31Z

^ Addresses all the comments and ensures MACs can be handled upper/lowercase, and dash/colon separated.

PROVISIONING-METAL.md

zmrow · 2023-01-06T15:56:37Z

^ Uses @yeazelm 's docs suggestion - thanks!

sources/api/netdog/src/cli/generate_net_config.rs

sources/api/netdog/src/cli/install.rs

sources/api/netdog/src/cli/mod.rs

zmrow · 2023-01-09T20:42:36Z

^ Addresses @bcressey 's feedback!

zmrow · 2023-01-09T20:51:01Z

^ Removes the commit parsing extra fields from the lease. We're not using those fields now that we're using sysfs to determine interface name

sources/api/netdog/src/main.rs

bcressey · 2023-01-09T22:51:55Z

Can we drop the patch to add the MAC address to the wicked lease, if we're no longer using it?

markusboehme

Just some nit picks from my side. The generation of the wicked stanzas taught me some more Serde ($unflatten). 👍

I'm ready to approve once the reboot concern identified by Ben is addressed.

sources/api/netdog/src/main.rs

markusboehme · 2023-01-10T14:30:00Z

Can we drop the patch to add the MAC address to the wicked lease, if we're no longer using it?

Related, the commit message of [netdog] Handle using MAC address as primary interface identifier also still mentions using the lease file.

zmrow · 2023-01-10T15:11:01Z

Can we drop the patch to add the MAC address to the wicked lease, if we're no longer using it?

Related, the commit message of [netdog] Handle using MAC address as primary interface identifier also still mentions using the lease file.

Thanks! I'll fix that up

zmrow · 2023-01-10T16:25:44Z

Can we drop the patch to add the MAC address to the wicked lease, if we're no longer using it?

@bcressey : #2726

This change removes the `prepare-primary-interface` service and moves its functionality to the wicked helper `netdog install`.

This adds an additional type `InterfaceId`, which allows deserialization of a valid interface name as well as a valid MAC address from network configuration. Validation of the MAC happens during deserialization. The `WickedInterface` struct has been updated and a `WickedName` struct has been added to ensure that the "name" node in the serialized XML has the proper format based on the use of an interface name or MAC address.

The interface name is important to us in a few places: the wicked install helper and setting up resolv.conf using DHCP lease. Previously, interface names were the sole way to configure interfaces in net.toml. At network generation time, there isn't a reliable or safe way to map a MAC to an interface name since udev may still be in the process of renaming devices. Later in the boot process (after interfaces have been setup by wicked and have leases), when interface name is necessary we can map a MAC address to a name using sysfs. In order to support MAC addresses as identifiers for interfaces, a few changes had to be made: At network generation time, if a MAC address is used for the primary interface, we write a `primary_mac_address` file rather than a `primary_interface` file. The `primary_interface()` function in the `Interfaces` trait now returns an `InterfaceId` and the `write_primary_interface()` function determines which file to write based on the InterfaceId enum variant. A new function `primary_interface_name()` has been added that returns the primary interface name. Under the hood, it reads either the `primary_interface` or `primary_mac_address` file. If `primary_mac_address` is read, it crawls sysfs to determine the network interface name. The `write_resolv_conf` subcommand has been updated to use the `primary_interface_name()` function.

This change adds the ability to configure network interfaces using their MAC address rather than the name udev assigns. This makes things easier for situations where MAC is known, but final name is not. Currently this ability is limited to interface configuration due to a limitation with wicked, and may not be used in conjunction with bonds/vlans.

This commit adds the documentation to support the use of MAC addresses as an additional method for interface configuration.

zmrow · 2023-01-10T21:29:18Z

^ Addresses @bcressey and @markusboehme 's comments. Thanks!

bcressey

🥅

markusboehme

🚀

zmrow requested review from bcressey, markusboehme and yeazelm December 1, 2022 20:15

zmrow commented Dec 1, 2022

View reviewed changes

zmrow force-pushed the MAC-address-identify branch 3 times, most recently from 3b2de95 to a2accb3 Compare December 14, 2022 20:04

webern reviewed Dec 14, 2022

View reviewed changes

PROVISIONING-METAL.md Outdated Show resolved Hide resolved

PROVISIONING-METAL.md Show resolved Hide resolved

yeazelm reviewed Dec 21, 2022

View reviewed changes

zmrow marked this pull request as ready for review January 4, 2023 20:45

zmrow force-pushed the MAC-address-identify branch from a2accb3 to 3f74783 Compare January 5, 2023 23:26

yeazelm approved these changes Jan 5, 2023

View reviewed changes

PROVISIONING-METAL.md Outdated Show resolved Hide resolved

zmrow force-pushed the MAC-address-identify branch from 3f74783 to d8a9ee4 Compare January 6, 2023 15:56

bcressey reviewed Jan 6, 2023

View reviewed changes

sources/api/netdog/src/cli/generate_net_config.rs Outdated Show resolved Hide resolved

sources/api/netdog/src/cli/install.rs Outdated Show resolved Hide resolved

sources/api/netdog/src/cli/mod.rs Outdated Show resolved Hide resolved

zmrow force-pushed the MAC-address-identify branch from d8a9ee4 to e3d3ace Compare January 9, 2023 20:41

zmrow force-pushed the MAC-address-identify branch from e3d3ace to d4d1afd Compare January 9, 2023 20:50

yeazelm approved these changes Jan 9, 2023

View reviewed changes

bcressey reviewed Jan 9, 2023

View reviewed changes

sources/api/netdog/src/main.rs Outdated Show resolved Hide resolved

markusboehme reviewed Jan 10, 2023

View reviewed changes

sources/api/netdog/src/main.rs Outdated Show resolved Hide resolved

sources/api/netdog/src/main.rs Show resolved Hide resolved

zmrow mentioned this pull request Jan 10, 2023

Generate Bottlerocket default network interface configuration aws/eks-anywhere#3411

Closed

zmrow added 6 commits January 10, 2023 21:01

netdog: Write primary interface sysctls in wicked DHCP helper

78b35fe

This change removes the `prepare-primary-interface` service and moves its functionality to the wicked helper `netdog install`.

docs: Add documentation for using MAC addresses in net.toml

c46d996

This commit adds the documentation to support the use of MAC addresses as an additional method for interface configuration.

netdog: Fix new clippy lints

d33f9a4

zmrow force-pushed the MAC-address-identify branch from d4d1afd to d33f9a4 Compare January 10, 2023 21:27

bcressey approved these changes Jan 10, 2023

View reviewed changes

markusboehme approved these changes Jan 11, 2023

View reviewed changes

zmrow merged commit 517189c into bottlerocket-os:develop Jan 12, 2023

zmrow deleted the MAC-address-identify branch January 12, 2023 17:19

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Support configuration using MAC instead of device name #2622

Support configuration using MAC instead of device name #2622

zmrow commented Dec 1, 2022 •

edited

Loading

zmrow commented Dec 14, 2022

yeazelm left a comment

markusboehme commented Jan 3, 2023 •

edited

Loading

zmrow commented Jan 4, 2023

zmrow commented Jan 5, 2023

zmrow commented Jan 6, 2023

zmrow commented Jan 9, 2023

zmrow commented Jan 9, 2023 •

edited

Loading

bcressey commented Jan 9, 2023

markusboehme left a comment

markusboehme commented Jan 10, 2023

zmrow commented Jan 10, 2023

zmrow commented Jan 10, 2023

zmrow commented Jan 10, 2023

bcressey left a comment

markusboehme left a comment

Support configuration using MAC instead of device name #2622

Support configuration using MAC instead of device name #2622

Conversation

zmrow commented Dec 1, 2022 • edited Loading

zmrow commented Dec 14, 2022

yeazelm left a comment

Choose a reason for hiding this comment

markusboehme commented Jan 3, 2023 • edited Loading

zmrow commented Jan 4, 2023

zmrow commented Jan 5, 2023

zmrow commented Jan 6, 2023

zmrow commented Jan 9, 2023

zmrow commented Jan 9, 2023 • edited Loading

bcressey commented Jan 9, 2023

markusboehme left a comment

Choose a reason for hiding this comment

markusboehme commented Jan 10, 2023

zmrow commented Jan 10, 2023

zmrow commented Jan 10, 2023

zmrow commented Jan 10, 2023

bcressey left a comment

Choose a reason for hiding this comment

markusboehme left a comment

Choose a reason for hiding this comment

zmrow commented Dec 1, 2022 •

edited

Loading

markusboehme commented Jan 3, 2023 •

edited

Loading

zmrow commented Jan 9, 2023 •

edited

Loading