Add exemplar support

CHANGELOG.md

@@ -12,7 +12,7 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/).
 
 ### Added
 
--
+- Exemplars support (#51)
 
 ### Changed
 
@@ -32,8 +32,6 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/).
 
 -
 
----
-
 ## [0.5](/~https://github.com/autometrics-dev/autometrics-py/releases/tag/0.5) - 2023-05-11
 
 ### Added

README.md

@@ -1,8 +1,11 @@
 ![GitHub_headerImage](https://user-images.githubusercontent.com/3262610/221191767-73b8a8d9-9f8b-440e-8ab6-75cb3c82f2bc.png)
 
-# autometrics-py
+[![Discord Shield](https://discordapp.com/api/guilds/950489382626951178/widget.png?style=shield)](https://discord.gg/kHtwcH8As9)
 
-A Python library that exports a decorator that makes it easy to understand the error rate, response time, and production usage of any function in your code. Jump straight from your IDE to live Prometheus charts for each HTTP/RPC handler, database method, or other piece of application logic.
+> A Python port of the Rust
+> [autometrics-rs](/~https://github.com/fiberplane/autometrics-rs) library
+
+**Autometrics is a library that exports a decorator that makes it easy to understand the error rate, response time, and production usage of any function in your code.** Jump straight from your IDE to live Prometheus charts for each HTTP/RPC handler, database method, or other piece of application logic.
 
 Autometrics for Python provides:
 
@@ -22,6 +25,7 @@ See [Why Autometrics?](/~https://github.com/autometrics-dev#why-autometrics) for m
 - [🚨 Define alerts](#alerts--slos) using SLO best practices directly in your source code
 - [📊 Grafana dashboards](#dashboards) work out of the box to visualize the performance of instrumented functions & SLOs
 - [⚙️ Configurable](#metrics-libraries) metric collection library (`opentelemetry`, `prometheus`, or `metrics`)
+- [📍 Attach exemplars](#exemplars) to connect metrics with traces
 - ⚡ Minimal runtime overhead
 
 ## Using autometrics-py
@@ -130,6 +134,16 @@ The `version` is read from the `AUTOMETRICS_VERSION` environment variable, and t
 
 This follows the method outlined in [Exposing the software version to Prometheus](https://www.robustperception.io/exposing-the-software-version-to-prometheus/).
 
+## Exemplars
+
+> **NOTE** - As of writing, exemplars aren't supported by the default tracker (`AUTOMETRICS_TRACKER=OPEN_TELEMETRY`).
+> You can track the progress of this feature here: /~https://github.com/autometrics-dev/autometrics-py/issues/41
+
+Exemplars are a way to associate a metric sample to a trace by attaching `trace_id` and `span_id` to it. You can then use this information to jump from a metric to a trace in your tracing system (for example Jaeger). If you have an OpenTelemetry tracer configured, autometrics will automatically pick up the current span from it.
+
+To use exemplars, you need to first switch to a tracker that supports them by setting `AUTOMETRICS_TRACKER=prometheus` and enable
+exemplar collection by setting `AUTOMETRICS_EXEMPLARS=true`. You also need to enable exemplars in Prometheus by launching Prometheus with the `--enable-feature=exemplar-storage` flag.
+
 ## Development of the package
 
 This package uses [poetry](https://python-poetry.org) as a package manager, with all dependencies separated into three groups:

examples/README.md

@@ -5,6 +5,7 @@ You should be able to run each example by:
 - cloning this repository
 - run `poetry install --with examples`
 - and execute `poetry run python examples/<example>.py` from the root of the repo.
+- for django and starlette examples, you can find the exact commands below
 
 You can change the base url for Prometheus links via the `PROMETHEUS_URL` environment variable. So, if your local Prometheus were on a non-default port, like 9091, you would run:
 
@@ -57,3 +58,19 @@ Autometrics also tracks a label, `caller`, which is the name of the function tha
 This is an example that shows you how to use autometrics to get metrics on http handlers with FastAPI. In this case, we're setting up the API ourselves, which means we need to expose a `/metrics` endpoint manually.
 
 > Don't forget to configure Prometheus itself to scrape the metrics endpoint. Refer to the example `prometheus.yaml` file in the root of this project on how to set this up.
+
+## `django-example`
+
+This is a default Django project with autometrics configured. You can find examples of instrumenting function and class based views in `django_example/views`. To run the example, navigate to `django_example` directory and run the standard command:
+
+`python manage.py runserver 8080`
+
+> Don't forget to configure Prometheus itself to scrape the metrics endpoint. Refer to the example `prometheus.yaml` file in the root of this project on how to set this up.
+
+## `starlette-otel-exemplars.py`
+
+This app shows how to use the OpenTelemetry integration to add exemplars to your metrics. In a distributed system, it allows you to track a request as it flows through your system by adding trace/span ids to it. We can catch these ids from OpenTelemetry and expose them to Prometheus as exemplars. Do note that exemplars are an experimental feature and you need to enable it in Prometheus with a `--enable-feature=exemplar-storage` flag. Run the example with a command:
+
+`AUTOMETRICS_TRACKER=prometheus AUTOMETRICS_EXEMPLARS=true uvicorn starlette-otel-exemplars:app --port 8080`
+
+> Don't forget to configure Prometheus itself to scrape the metrics endpoint. Refer to the example `prometheus.yaml` file in the root of this project on how to set this up.

examples/starlette-otel-exemplars.py

@@ -0,0 +1,57 @@
+from opentelemetry import trace
+from autometrics import autometrics
+from prometheus_client import REGISTRY
+from prometheus_client.openmetrics.exposition import generate_latest
+from starlette import applications
+from starlette.responses import PlainTextResponse
+from starlette.routing import Route
+from opentelemetry import trace
+from opentelemetry.sdk.trace import TracerProvider
+from opentelemetry.sdk.trace.export import (
+    BatchSpanProcessor,
+    ConsoleSpanExporter,
+)
+
+# Let's start by setting up the OpenTelemetry SDK with some defaults
+provider = TracerProvider()
+processor = BatchSpanProcessor(ConsoleSpanExporter())
+provider.add_span_processor(processor)
+trace.set_tracer_provider(provider)
+
+# Now we can instrument our Starlette application
+tracer = trace.get_tracer(__name__)
+
+
+# We need to add tracer decorator before autometrics so that we see the spans
+@tracer.start_as_current_span("request")
+@autometrics
+def outer_function(request):
+    response = inner_function()
+    return PlainTextResponse(response)
+
+
+# This function will also get an exemplar because it is called within
+# the span of the outer_function
+@autometrics
+def inner_function():
+    return "Hello world!"
+
+
+def metrics(request):
+    # Exemplars are not supported by default prometheus format, so we specifically
+    # make an endpoint that uses the OpenMetrics format that supoorts exemplars.
+    body = generate_latest(REGISTRY)
+    return PlainTextResponse(body, media_type="application/openmetrics-text")
+
+
+app = applications.Starlette(
+    routes=[Route("/", outer_function), Route("/metrics", metrics)]
+)
+
+# Now, start the app (env variables are required to enable exemplars):
+# AUTOMETRICS_TRACKER=prometheus AUTOMETRICS_EXEMPLARS=true uvicorn starlette-otel-exemplars:app --port 8080
+# And make some requests to /. You should see the spans in the console.
+# With autometrics extension installed, you can now hover over the hello handler
+# and see the charts and queries associated with them. Open one of the queries
+# in Prometheus and you should see exemplars added to the metrics. Don't forget
+# to click the Show Exemplars button in Prometheus to see them!