diff --git a/content/en/blog/2024/cve-2024-36129/index.md b/content/en/blog/2024/cve-2024-36129/index.md new file mode 100644 index 000000000000..3d2b2cc7ba06 --- /dev/null +++ b/content/en/blog/2024/cve-2024-36129/index.md @@ -0,0 +1,129 @@ +--- +title: Security vulnerability in confighttp (CVE-2024-36129) +linkTitle: Security vulnerability in confighttp +date: 2024-06-05 +author: + '[Juraci Paixão Kröhling](/~https://github.com/jpkrohling) (OpenTelemetry, + Grafana Labs), [Pablo Baeyens](/~https://github.com/mx-psi) (OpenTelemetry, + Datadog)' +cSpell:ignore: Baeyens configgrpc confighttp Miroslav OSTIF Stampar zstd +--- + +On our path toward graduation, the OpenTelemetry project is currently undergoing +a security audit sponsored by the [CNCF](https://www.cncf.io/), facilitated by +[OSTIF](https://ostif.org/), and performed by +[7ASecurity](https://7asecurity.com/). During this process, we have received a +few ideas about things that we could do better, like using specific compiler +flags when preparing our +[OpenTelemetry Collector binaries](/~https://github.com/open-telemetry/opentelemetry-collector-releases/releases). +On 31 May 2024, we received +[a more serious report](/~https://github.com/open-telemetry/opentelemetry-collector/security/advisories/GHSA-c74f-6mfw-mm4v): +a malicious user could cause a denial of service (DoS) when using a specially +crafted HTTP or gRPC request. The advisory was assigned the following CVE +identifier: CVE-2024-36129. + +When sending an HTTP request with a compressed payload, the Collector would +verify only whether the compressed payload is beyond a certain limit, but not +its uncompressed version. A malicious payload could then send a +“[compressed bomb](https://en.wikipedia.org/wiki/Zip_bomb)”, causing the +Collector to crash. + +Similarly, when sending a gRPC request using zstd compression, the decompression +mechanism would not respect the limits imposed by gRPC, also causing the +Collector to crash while decompressing the malicious payload. + +A few business hours after the report, on 03 June 2024, Collector developers +were able to reproduce the report related to HTTP and worked together on a fix +that +[got merged the same day](/~https://github.com/open-telemetry/opentelemetry-collector/pull/10289). +Given the high score of this issue, we’ve decided to hold the release that would +have happened on the same day, +[completing it](/~https://github.com/open-telemetry/opentelemetry-collector-releases/releases/tag/v0.102.0) +on 04 June 2024 instead. + +After the release, we got a confirmation that gRPC with zstd was also affected. +Within a few business hours of the confirmation, we worked on a fix that also +[got merged the same day](/~https://github.com/open-telemetry/opentelemetry-collector/pull/10323). +[We released v0.102.1](/~https://github.com/open-telemetry/opentelemetry-collector-releases/releases/tag/v0.102.1) +right after that. + +## Am I affected? + +You are affected by this vulnerability if you have an OpenTelemetry Collector +with one or more HTTP or gRPC receivers on a public port, such as the OTLP +Receiver with the “HTTP” or “gRPC” protocol enabled (typically on ports 4318 and +4317, respectively) AND the receiver has version 0.102.0 or below. The +vulnerability is exploitable only by attackers who can send payloads to your +HTTP/gRPC endpoint(s). This usually means that the port needs to be exposed to +the public internet or another network segment that's available to the attacker. + +Note that if you require authentication, an attacker would need to have valid +credentials in order to exploit the vulnerability using the HTTP protocol. For +gRPC, the exploitable code is executed before authentication. + +## What does it mean to me? + +If you manage a Collector that has an interface to the public internet, you +should upgrade it as soon as feasible, and consider setting the parameter +`max_request_body_size` on HTTP receivers, such as the OTLP receiver, to a value +that makes sense to your workload. Up to v0.101.0, this setting applied only to +the payload size sent by the client, which could often be compressed. + +Starting from v0.102.0, this setting applies to uncompressed, compressed, and +decompressed payload sizes and we are establishing a default value of 20 MiB for +this. This new default characterizes a breaking change, as clients sending +payloads bigger than 20 MiB will start seeing an error. While we believe most +authentic requests will be way within this limit, it’s still wise to monitor +your Collector for increased error rates after this update. Here’s an example of +a configuration setting a different limit to this field: + +```yaml +receivers: + otlp: + protocols: + http: + endpoint: localhost:4318 + max_request_body_size: 10485760 # 10 MiB +``` + +For gRPC receivers, it’s sufficient to upgrade to v0.102.1, as there’s a default +value being applied to the message size already: 4 MiB. + +If your Collector instances are receiving data only from trusted clients, like +your own applications, you are still encouraged to upgrade to the latest +Collector version but you can do it at your regular pace. + +If you are using a custom distribution and building it with the +[OpenTelemetry Collector Builder (ocb)](/docs/collector/custom-collector/), you +can add a +[“replaces”](/~https://github.com/open-telemetry/opentelemetry-collector/tree/main/cmd/builder#configuration) +entry pointing to the latest version of the +[confighttp](http://go.opentelemetry.io/collector/config/confighttp) and +[configgrpc](http://go.opentelemetry.io/collector/config/configgrpc) Go module. +If your base Collector version is at v0.96.0 or higher, we do not expect any +compatibility issues by just bumping to the latest version. + +## Lessons learned + +During this process, we found a couple of gaps in the telemetry for the +Collector, as well as to the options we provide to Collector admins. Concretely, +we noticed that we do not have a good way to verify what’s the distribution of +request sizes received by the Collector, which would have been useful to +determine whether the change would break clients for a given Collector. We also +noticed that we don’t provide a way for admins to completely disable +compression, which would be a good way to mitigate an attack without having to +upgrade the Collector. We are working to fill those gaps over the next releases. + +We are also working on stabilizing the `component.UseLocalHostAsDefaultHost` +feature gate to reduce exposure of all Collector endpoints by default. This +feature gate was motivated by a previous, +[similar vulnerability on Go's standard library](/~https://github.com/advisories/GHSA-4374-p667-p6c8) +and has been in alpha for several months. You can follow the discussion +surrounding stabilization at +[issue 8510](/~https://github.com/open-telemetry/opentelemetry-collector/issues/8510). + +## Acknowledgements + +This issue was identified by [Miroslav Stampar](/~https://github.com/stamparm/), +from 7ASecurity. We’d like to thank 7ASecurity for the responsible reporting of +this vulnerability. diff --git a/static/refcache.json b/static/refcache.json index 817859068f96..a634888a4a78 100644 --- a/static/refcache.json +++ b/static/refcache.json @@ -19,6 +19,14 @@ "StatusCode": 206, "LastSeen": "2024-01-18T08:54:18.004628-05:00" }, + "http://go.opentelemetry.io/collector/config/configgrpc": { + "StatusCode": 200, + "LastSeen": "2024-06-05T14:38:00.458726875+02:00" + }, + "http://go.opentelemetry.io/collector/config/confighttp": { + "StatusCode": 200, + "LastSeen": "2024-06-04T17:29:53.957107724+02:00" + }, "http://mypy-lang.org/": { "StatusCode": 206, "LastSeen": "2024-01-18T19:02:04.404963-05:00" @@ -39,6 +47,10 @@ "StatusCode": 206, "LastSeen": "2024-01-30T16:15:09.743434-05:00" }, + "https://7asecurity.com/": { + "StatusCode": 206, + "LastSeen": "2024-06-04T17:29:49.77219755+02:00" + }, "https://access.redhat.com/products/ansible-tower-red-hat": { "StatusCode": 200, "LastSeen": "2024-01-18T08:05:55.59597-05:00" @@ -1943,6 +1955,10 @@ "StatusCode": 200, "LastSeen": "2024-01-18T19:01:58.402439-05:00" }, + "https://en.wikipedia.org/wiki/Zip_bomb": { + "StatusCode": 200, + "LastSeen": "2024-06-04T17:29:51.894512776+02:00" + }, "https://equalitymi.org/": { "StatusCode": 200, "LastSeen": "2024-01-18T19:02:20.222589-05:00" @@ -2323,6 +2339,10 @@ "StatusCode": 200, "LastSeen": "2024-01-30T06:01:30.362583-05:00" }, + "/~https://github.com/advisories/GHSA-4374-p667-p6c8": { + "StatusCode": 200, + "LastSeen": "2024-06-04T17:29:54.745634368+02:00" + }, "/~https://github.com/aishyandapalli": { "StatusCode": 200, "LastSeen": "2024-01-18T20:06:24.492346-05:00" @@ -2923,6 +2943,10 @@ "StatusCode": 200, "LastSeen": "2024-01-18T20:05:04.146005-05:00" }, + "/~https://github.com/mx-psi": { + "StatusCode": 200, + "LastSeen": "2024-06-04T17:29:48.499621359+02:00" + }, "/~https://github.com/nemoshlag": { "StatusCode": 200, "LastSeen": "2024-01-18T20:05:15.066411-05:00" @@ -3203,6 +3227,14 @@ "StatusCode": 200, "LastSeen": "2024-01-18T19:37:01.186504-05:00" }, + "/~https://github.com/open-telemetry/opentelemetry-collector-releases/releases/tag/v0.102.0": { + "StatusCode": 200, + "LastSeen": "2024-06-04T17:29:53.270047268+02:00" + }, + "/~https://github.com/open-telemetry/opentelemetry-collector-releases/releases/tag/v0.102.1": { + "StatusCode": 200, + "LastSeen": "2024-06-05T18:14:20.086786151+02:00" + }, "/~https://github.com/open-telemetry/opentelemetry-collector-releases/releases/tag/v0.74.0": { "StatusCode": 200, "LastSeen": "2024-01-30T05:18:23.898255-05:00" @@ -3231,10 +3263,22 @@ "StatusCode": 200, "LastSeen": "2024-04-04T11:07:15.276911438-07:00" }, + "/~https://github.com/open-telemetry/opentelemetry-collector/issues/8510": { + "StatusCode": 200, + "LastSeen": "2024-06-04T17:29:55.674365003+02:00" + }, "/~https://github.com/open-telemetry/opentelemetry-collector/issues/9375": { "StatusCode": 200, "LastSeen": "2024-05-03T08:34:29.3339-07:00" }, + "/~https://github.com/open-telemetry/opentelemetry-collector/pull/10289": { + "StatusCode": 200, + "LastSeen": "2024-06-04T17:29:52.747224454+02:00" + }, + "/~https://github.com/open-telemetry/opentelemetry-collector/pull/10323": { + "StatusCode": 200, + "LastSeen": "2024-06-05T14:37:59.891183931+02:00" + }, "/~https://github.com/open-telemetry/opentelemetry-collector/pull/6140": { "StatusCode": 200, "LastSeen": "2024-01-30T05:18:24.402543-05:00" @@ -3303,6 +3347,10 @@ "StatusCode": 200, "LastSeen": "2024-01-30T15:25:16.610744-05:00" }, + "/~https://github.com/open-telemetry/opentelemetry-collector/security/advisories/GHSA-c74f-6mfw-mm4v": { + "StatusCode": 200, + "LastSeen": "2024-06-05T18:14:19.254754319+02:00" + }, "/~https://github.com/open-telemetry/opentelemetry-configuration": { "StatusCode": 200, "LastSeen": "2024-01-30T06:01:13.097575-05:00" @@ -4507,6 +4555,10 @@ "StatusCode": 200, "LastSeen": "2024-01-30T16:15:09.784881-05:00" }, + "/~https://github.com/stamparm/": { + "StatusCode": 200, + "LastSeen": "2024-06-04T17:29:57.257378271+02:00" + }, "/~https://github.com/statsd/statsd": { "StatusCode": 200, "LastSeen": "2024-01-18T19:37:16.308411-05:00" @@ -5847,6 +5899,10 @@ "StatusCode": 200, "LastSeen": "2024-01-22T14:34:52.350494+01:00" }, + "https://nvd.nist.gov/vuln/detail/CVE-2024-36129": { + "StatusCode": 200, + "LastSeen": "2024-06-04T17:29:51.244907508+02:00" + }, "https://o11y.news": { "StatusCode": 206, "LastSeen": "2024-01-18T19:10:29.899332-05:00" @@ -6051,6 +6107,10 @@ "StatusCode": 206, "LastSeen": "2024-03-19T10:16:59.755536357Z" }, + "https://opentelemetry.io/": { + "StatusCode": 206, + "LastSeen": "2024-06-04T17:29:59.050384645+02:00" + }, "https://opentelemetry.io/blog/2022/jaeger-native-otlp/": { "StatusCode": 206, "LastSeen": "2024-04-27T13:39:35.171666-04:00" @@ -6131,6 +6191,14 @@ "StatusCode": 206, "LastSeen": "2024-04-30T09:31:38.379400427Z" }, + "https://opentelemetry.io/docs/migration/opentracing/": { + "StatusCode": 206, + "LastSeen": "2024-06-04T17:29:58.687939269+02:00" + }, + "https://opentelemetry.io/docs/migration/opentracing/#language-version-support": { + "StatusCode": 206, + "LastSeen": "2024-06-04T17:29:58.393305846+02:00" + }, "https://opentelemetry.io/docs/specs/otel/glossary/#instrumentation-library": { "StatusCode": 206, "LastSeen": "2024-04-30T09:31:37.929550219Z" @@ -6139,6 +6207,34 @@ "StatusCode": 206, "LastSeen": "2024-02-24T14:33:05.630341-08:00" }, + "https://opentelemetry.io/docs/specs/otel/versioning-and-stability/#semantic-conventions-stability": { + "StatusCode": 206, + "LastSeen": "2024-06-04T17:30:02.895017798+02:00" + }, + "https://opentelemetry.io/docs/specs/otlp/": { + "StatusCode": 206, + "LastSeen": "2024-06-04T17:30:00.93746657+02:00" + }, + "https://opentelemetry.io/docs/specs/semconv/exceptions/exceptions-spans/#recording-an-exception": { + "StatusCode": 206, + "LastSeen": "2024-06-04T17:30:00.355601942+02:00" + }, + "https://opentelemetry.io/docs/specs/semconv/general/attribute-naming/": { + "StatusCode": 206, + "LastSeen": "2024-06-04T17:29:59.520526055+02:00" + }, + "https://opentelemetry.io/docs/specs/semconv/general/attribute-requirement-level/": { + "StatusCode": 206, + "LastSeen": "2024-06-04T17:29:56.977663774+02:00" + }, + "https://opentelemetry.io/docs/specs/semconv/general/metric-requirement-level/": { + "StatusCode": 206, + "LastSeen": "2024-06-04T17:29:59.855130203+02:00" + }, + "https://opentelemetry.io/docs/specs/semconv/resource/": { + "StatusCode": 206, + "LastSeen": "2024-06-04T17:30:01.040485926+02:00" + }, "https://opentelemetry.io/docs/specs/semconv/resource/k8s/": { "StatusCode": 206, "LastSeen": "2024-05-15T19:23:47.920456821+03:00" @@ -6175,6 +6271,10 @@ "StatusCode": 206, "LastSeen": "2024-01-19T13:15:33.481201+01:00" }, + "https://ostif.org/": { + "StatusCode": 200, + "LastSeen": "2024-06-04T17:29:49.319661335+02:00" + }, "https://otterize.com/": { "StatusCode": 200, "LastSeen": "2024-01-30T16:15:04.230377-05:00" @@ -8439,6 +8539,10 @@ "StatusCode": 206, "LastSeen": "2024-01-18T19:10:25.924884-05:00" }, + "https://www.cncf.io/": { + "StatusCode": 206, + "LastSeen": "2024-06-04T17:29:48.60678168+02:00" + }, "https://www.cncf.io/blog/2019/05/21/a-brief-history-of-opentelemetry-so-far/": { "StatusCode": 206, "LastSeen": "2024-01-18T20:05:14.383107-05:00"