enable kernel lockdown in integrity mode

[bcressey]

Support for the Lockdown LSM was merged in the 5.4 kernel. We'd like to enable it in integrity mode, which is aimed at protecting the kernel from unwanted modification by userspace.

Lockdown reasons through LOCKDOWN_INTEGRITY_MAX are blocked in integrity mode.

We need to make sure that BPF functionality is not impaired.

[bcressey]

I created an admin container with bcc and bpftrace installed.

bpftrace works after mounting debugfs (/sys/kernel/debug) into the container. Some of the bcc programs do as well, but others are blocked by LOCKDOWN_DEBUGFS as they need access to /sys/kernel/debug/tracing/../kprobes/blacklist.

It seems like what we really want is to only mount tracefs (/sys/kernel/debug/tracing) into the container, and for tracefs to include the kprobes blacklist so we don't run afoul of the lockdown policy.

[bcressey]

Predictably, lockdown also blocks loading of out-of-tree modules:

[129687.022749] Lockdown: insmod: unsigned module loading is restricted; see man kernel_lockdown.7

So we'll need to solve that as well.

[bcressey]

bcc merged a fix to make the kprobes blacklist optional: iovisor/bcc@5558e36b.

[tjkirch]

For those following this issue, we've added settings.kernel.lockdown which should be available in the next release. It allows you to raise your Linux kernel lockdown level from none (the default) to integrity or confidentiality. (#1224 tracks adding further guidance in our documentation.)

This issue will remain open to discuss changing the default value.

[arnaldo2792]

• We now use settings.kernel.lockdown = integrity as the default value for new variants as k8s 1.20 (introduced in Add aws-k8s-1.20 variant with Kubernetes 1.20 support #1437)
• Update security documentation for kernel lockdown mode #1224 is already closed, and we already have documentation for the setting

[arnaldo2792]

After some discussion with @bcressey and @samuelkarp, we decided to keep this open until we change the default settings for the ecs variant

[vikas027]

* We now use `settings.kernel.lockdown = integrity` as the default value for new variants as k8s 1.20 (introduced in #1437)

* #1224 is already closed, and we already have documentation for the setting

Hey @arnaldo2792 ,

Sorry, a bit confused in the TOML usage.

The setting can be defined in any of these format, right?

settings.kernel.lockdown = "integrity"

or

[settings.kernel]
lockdown = "integrity"

[etungsten]

Hey @arnaldo2792 ,

Sorry, a bit confused in the TOML usage.

The setting can be defined in any of these format, right?

settings.kernel.lockdown = "integrity"

or

[settings.kernel]
lockdown = "integrity"

Hi @vikas027, that's correct. Both of those are valid ways of defining that setting in your userdata.

Edit: The above is incorrect.

The latter is what you should use for userdata.

[settings.kernel]
lockdown = "integrity"

[etungsten]

Sorry, I gave a bad answer! I've edited the comment above to correct myself. We recommend using the structured format when specifying settings in userdata.

[faarshad]

Predictably, lockdown also blocks loading of out-of-tree modules:

[129687.022749] Lockdown: insmod: unsigned module loading is restricted; see man kernel_lockdown.7

So we'll need to solve that as well.

@bcressey, Is there any way to use kernel lockdown integrity mode but white-list(maybe somehow sign it to make it legit etc) a particular out-of-tree kernel module?

kernel_lockdown man page says:

Only validly signed modules may be loaded (waived if the module
file being loaded is vouched for by IMA appraisal).

I tried installing falco with lockdown = "integrity" set and as expected(per kernel_lockdown), the installation did fail:

...
DKMS: build completed.

falco.ko:
Running module version sanity check.
 - Original module
   - No original module exists within this kernel
 - Installation
   - Installing to /lib/modules/5.4.129/kernel/extra/

depmod...

DKMS: install completed.
* falco module installed in dkms, trying to insmod
* Unable to insmod falco module
* Trying to dkms install falco module with GCC /usr/bin/gcc-6
DIRECTIVE: MAKE="'/tmp/falco-dkms-make'"
Module falco/17f5df52a7d9ed6bb12d3b1768460def8439936d already installed on kernel 5.4.129/x86_64
* falco module installed in dkms, trying to insmod
* Unable to insmod falco module
* Trying to dkms install falco module with GCC /usr/bin/gcc-5
DIRECTIVE: MAKE="'/tmp/falco-dkms-make'"
Module falco/17f5df52a7d9ed6bb12d3b1768460def8439936d already installed on kernel 5.4.129/x86_64
* falco module installed in dkms, trying to insmod
Consider compiling your own falco driver and loading it or getting in touch with the Falco community
* Unable to insmod falco module
Thu Sep 23 01:31:18 2021: Falco version 0.29.1 (driver version 17f5df52a7d9ed6bb12d3b1768460def8439936d)
Thu Sep 23 01:31:18 2021: Falco initialized with configuration file /etc/falco/falco.yaml
Thu Sep 23 01:31:18 2021: Loading rules from file /etc/falco/falco_rules.yaml:
Thu Sep 23 01:31:20 2021: Loading rules from file /etc/falco/falco_rules.local.yaml:
Thu Sep 23 01:31:22 2021: Loading rules from file /etc/falco/rules.d/white-lists.yaml:
Thu Sep 23 01:31:23 2021: Unable to load the driver.
Thu Sep 23 01:31:23 2021: Runtime error: error opening device /host/dev/falco0. Make sure you have root credentials and that the falco module is loaded.. Exiting.

Also, i did not see the error: Lockdown: X: Y is restricted, see man kernel_lockdown.7, so falco might be swallowing that error.

[bcressey]

Lockdown in "integrity" mode and external kernel modules are mutually exclusive for now.

This is complicating our plans for other features - nvidia drivers, Secure Boot - so it's something we're thinking about, but don't have any concrete solution or plans to share.

It's just a hard problem to make a private key available on the host for signing "good" modules, but keep it inaccessible so untrusted actors can't use it to sign "bad" modules.

For Falco specifically, the BPF implementation can be used in place of the kmod to work around this.

[faarshad]

Thank you @bcressey - Falco(0.29.1) eBPF mode is functional with kernel lockdown in integrity mode for us on bottlerocket (1.2.1 & 1.1.4) on k8s 1.19.12. Appreciate the help!