# OpenTelemetry

Sending GitHub metrics to an OpenTelemetry collector or to Prometheus: OTLP over HTTP with the JSON encoding, and why raw is false by default.

Source: https://docs.jmrp.io/ghchronicle/sinks/otlp/

The OTLP sink pushes ghchronicle's metrics to an OpenTelemetry collector, or straight into Prometheus through its OTLP receiver.

```yaml
sinks:
  otlp:
    endpoint: http://collector:4318/v1/metrics
    service: ghchronicle
    headers:
      Authorization: Bearer ${OTLP_TOKEN}
    raw: false
    batch: 2000
```

## The wire contract

OTLP over HTTP with the **JSON encoding**, which every receiver worth using
accepts on the same endpoint. That is a deliberate trade: JSON is larger on the
wire, and it keeps a code generator, a protobuf runtime and their transitive
dependencies out of the binary. Every sink but PostgreSQL, which needs its
driver, speaks its protocol with the standard library alone.

`endpoint` is the full URL of the metrics path, not a base. `headers` go with
every request, which is where an API key or a tenant id belongs. `service`
becomes the resource attribute the backend groups by. `batch` is how many data
points go in one request, 2000 unless it is lowered for a receiver with a
smaller body limit.

Metric names use dots, following the OpenTelemetry convention:
`github.workflow.run.duration.seconds`. A receiver exporting onward to
Prometheus converts them itself; going the other way it cannot.

## `raw`

- **raw: false (default)**

  Sends the same reduced current values as the Prometheus exporter, made by
  [the same reduction](https://docs.jmrp.io/ghchronicle/sinks/prometheus/#the-reduction), so each
  mean is taken over the items that carried the field. Safe with any backend,
  including Prometheus's own OTLP receiver, because nothing in the payload is
  older than the sweep.

- **raw: true**

  Sends the dated points. Whether they survive is entirely the backend's
  decision: OTLP data points carry an explicit timestamp, so a store that
  accepts old ones keeps the history, and one that does not rejects them.

> **Prometheus's OTLP receiver is one that does not**
>
> Measured: against Prometheus 3.14 with `--web.enable-otlp-receiver` and
> `out_of_order_time_window: 30m`, a sample dated two days back comes back as
> HTTP 400. With `raw: true` pointed at Prometheus, roughly half of every sweep
> is refused. Use `raw: false` there, or send to a store that keeps the dates.

## `repeat`

A gauge asserted once and then never again fades from the dashboards, and a
family that runs every twelve hours would be a flat line with a dot in it. With
`repeat` set, the sink keeps asserting the newest value of every series it has
seen until the next sweep replaces it.

```yaml
sinks:
  otlp:
    endpoint: http://collector:4318/v1/metrics
    repeat: 1m
```

This matters for a backend that answers an instant query from the last sample
within a lookback window.

## A collector in front

The usual arrangement is a collector that receives from here and exports
onward, which is what makes this sink worth having at all: the tool speaks one
protocol and the pipeline decides where the data ends up.

```yaml
receivers:
  otlp:
    protocols:
      http:
        endpoint: 0.0.0.0:4318

exporters:
  prometheusremotewrite:
    endpoint: http://prometheus:9090/api/v1/write

service:
  pipelines:
    metrics:
      receivers: [otlp]
      exporters: [prometheusremotewrite]
```

With that pipeline, keep `raw: false`: the exporter at the far end is
Prometheus, and the constraint follows the data rather than the protocol.

## Nothing to migrate

The sink pushes today's values, in the shape of the release that runs it, so
[`-migrate`](https://docs.jmrp.io/ghchronicle/install/upgrading/#migrations) has nothing to do
here. What the receiver keeps of an old series ends where the new one begins.

## Where to go next

- [Choosing a store](https://docs.jmrp.io/ghchronicle/sinks/) compares OpenTelemetry with the others,
  and holds the write ledger every one of them shares.
- [The dashboards](https://docs.jmrp.io/ghchronicle/dashboards/) says which of the five is drawn
  against which store, and what a panel a store cannot answer becomes.