Add the ability to drop all datapoints except a specific one.

[ceastman-r7]

Component(s)

processor/metricstransform

Is your feature request related to a problem? Please describe.

I am trying to drop all datapoints except where destination_service_name == BlackHoleCluster

the label destination_service_name can have an infinite amount of values, I just need to only retain the datapoints where it equals BlackHoleCluster``

Describe the solution you'd like

from this example: /~https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/processor/metricstransformprocessor#delete-by-label-value

something like:

# deletes all data points with the label value 'idle' of the label 'state'
include: system.cpu.usage
action: update
operations:
  - action: delete_label_value
    label: destination_service_name
    label_not_value: BlackHoleCluster

Describe alternatives you've considered

No response

Additional context

No response

[github-actions]

Pinging code owners:

• processor/metricstransform: @dmitryax

See Adding Labels via Comments if you do not have permissions to add labels yourself.

[strawgate]

FYI I believe the metrics/logs/etc transform processors are being replaced with the OTTL-based "transform processor"

Have you considered trying the filterprocessor with the context set to datapoint?
/~https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/processor/filterprocessor

processors:
  filter/ottl:
    error_mode: ignore
    metrics:
      datapoint:
          - 'attributes["destination_service_name"] != "blackhole"'

[github-actions]

Pinging code owners for processor/filter: @TylerHelmuth @boostchicken. See Adding Labels via Comments if you do not have permissions to add labels yourself.

[TylerHelmuth]

@strawgate is correct. Closing this since an answer has been provided.

[ceastman-r7]

Thank you, I will try that.

[etiennecallies]

Hi @TylerHelmuth ! I saw that you did implement this feature.

I tried it on latest version : 0.114.0

With hostmetrics receiver and metric system.filesystem.utilization we have this kind of metric and datapoints :

otelcol-contrib[401089]: InstrumentationScope github.com/open-telemetry/opentelemetry-collector-contrib/receiver/hostmetricsreceiver/internal/scraper/filesystemscraper 0.114.0
otelcol-contrib[401089]: Metric #0
otelcol-contrib[401089]: Descriptor:
otelcol-contrib[401089]:      -> Name: system.filesystem.utilization
otelcol-contrib[401089]:      -> Description: Fraction of filesystem bytes used.
otelcol-contrib[401089]:      -> Unit: 1
otelcol-contrib[401089]:      -> DataType: Gauge
otelcol-contrib[401089]: NumberDataPoints #0
otelcol-contrib[401089]: Data point attributes:
otelcol-contrib[401089]:      -> device: Str(/dev/sda1)
otelcol-contrib[401089]:      -> mode: Str(rw)
otelcol-contrib[401089]:      -> mountpoint: Str(/)
otelcol-contrib[401089]:      -> type: Str(ext4)
otelcol-contrib[401089]: StartTimestamp: 2024-11-22 21:12:38 +0000 UTC
otelcol-contrib[401089]: Timestamp: 2024-11-28 07:15:38.038988892 +0000 UTC
otelcol-contrib[401089]: Value: 0.357942
...
otelcol-contrib[401089]: NumberDataPoints #9
otelcol-contrib[401089]: Data point attributes:
otelcol-contrib[401089]:      -> device: Str(/dev/loop4)
otelcol-contrib[401089]:      -> mode: Str(ro)
otelcol-contrib[401089]:      -> mountpoint: Str(/snap/aws-cli/1064)
otelcol-contrib[401089]:      -> type: Str(squashfs)
otelcol-contrib[401089]: StartTimestamp: 2024-11-22 21:12:38 +0000 UTC
otelcol-contrib[401089]: Timestamp: 2024-11-28 07:15:38.038988892 +0000 UTC
otelcol-contrib[401089]: Value: 1.000000
otelcol-contrib[401089]:         {"kind": "exporter", "data_type": "metrics", "name": "debug"}

I want to keep only datapoints with mountpoint == "/".

So I tried this filter :

  filter/ottl:
    error_mode: ignore
    metrics:
      datapoint:
          - 'metric.name == "system.filesystem.utilization" and resource.attributes["mountpoint"] != "/"'

But it removes the whole metrics system.filesystem.utilization.

Is it expected? Am I missing something?
Many thanks

[strawgate]

      - 'metric.name == "system.filesystem.utilization" and resource.attributes["mountpoint"] != "/"'

mountpoint is an attribute, not a resource attribute (as indicated by your output above saying "data point attribute"), so none of the data points will pass the filter and all will be removed

If you change it to attribute, it should start working

[etiennecallies]

This works !! 🎉 Many thanks @strawgate 😘

  filter/ottl:
    error_mode: ignore
    metrics:
      datapoint:
          - 'metric.name == "system.filesystem.utilization" and attributes["mountpoint"] != "/"'