# Environment variables

The MIKROSCOPE_* variables the CLI reads, the credentials that exist only as variables, and the envlist variables the agent reads on the router.

Source: https://docs.jmrp.io/mikroscope/reference/environment/

Two programs read environment variables, and they read different ones. The
**CLI** on your machine reads `MIKROSCOPE_*` variables as flag defaults, plus a
handful of credentials that have no flag, plus `GRAFANA_URL` and
`GRAFANA_TOKEN`. The **agent** on the router reads unprefixed variables
(`RATE_HZ`, `TOKEN` …) from its container envlist, which `install` writes. The
CLI's are defined in [`cmd/mikroscope/`](https://github.com/jmrplens/mikroscope/tree/main/cmd/mikroscope), the agent's
in [`internal/agent/config.go`](https://github.com/jmrplens/mikroscope/blob/main/internal/agent/config.go).

## Precedence

- A variable sets a flag's **default**; the flag on the command line wins.
- A variable set to the empty string is treated as unset.
- The shape variables (`MIKROSCOPE_VETH`, `MIKROSCOPE_SUBNET`,
  `MIKROSCOPE_IFACE_LIST`, `MIKROSCOPE_ADDR_LIST`, `MIKROSCOPE_DISK`,
  `MIKROSCOPE_LAN_ADDRESS`, `MIKROSCOPE_REMOTE_IMAGE`) set what `install`
  writes. `status`, `upgrade` and `uninstall` take the shape of the install on
  the router instead, from its manifest or its tagged objects, and refuse a
  variable or flag that contradicts it, naming both values; `MIKROSCOPE_REMOTE_IMAGE`
  is the exception, filled in for `status` and `uninstall` and never refused.
  `.env.example` leaves them unset for that reason.
- The CLI never reads a `.env` file. Export it into the shell first:

  ```sh
  set -a; . ./.env; set +a
  ```

- Quote a URL that holds `&` when it lives in a file you `source`:
  unquoted, `&` is a shell operator. [`.env.example`](https://github.com/jmrplens/mikroscope/blob/main/.env.example) quotes
  `MIKROSCOPE_INFLUX_URL` for that reason.

Most deployment flags have **no** variable. `--rate`, `--buffer`, `--port`,
`--memory-max`, `--mem-limit-mb`, `--capture-mb`, `--triggers`, `--floor-hz`,
`--privileged`, `--ephemeral`, `--expose`, `--restart-max-count`,
`--restart-interval`, `--start-on-boot`, `--container-name` and
`--extract-timeout` are set on the command line or not at all. [CLI](https://docs.jmrp.io/mikroscope/reference/cli/) has every flag.

## Router and deployment

| Variable                  | Flag             | Read by                                               | Default in the code          | Value in [`.env.example`](https://github.com/jmrplens/mikroscope/blob/main/.env.example) |
| ------------------------- | ---------------- | ----------------------------------------------------- | ---------------------------- | ----------------------- |
| `MIKROSCOPE_ROUTER`       | `--router`       | `doctor`, `install`, `upgrade`, `uninstall`, `status` | none (required)              | set                   |
| `MIKROSCOPE_SSH_PORT`     | `--ssh-port`     | the same                                              | ssh config                   | `22`                    |
| `MIKROSCOPE_SSH_KEY`      | `--ssh-key`      | the same                                              | ssh agent or config          | empty                   |
| `MIKROSCOPE_SSH_OPTIONS` | `--ssh-option` | the same | none | commented out, `StrictHostKeyChecking=accept-new` |
| `MIKROSCOPE_ARCH` | `--arch` | the deployment verbs | `auto` | commented out, `auto` |
| `MIKROSCOPE_AGENT_TAR`    | `--agent-tar`    | `doctor`, `plan`, `install`, `upgrade`, `image`       | empty (build the agent here) | commented out           |
| `MIKROSCOPE_REMOTE_IMAGE` | `--remote-image` | the deployment verbs                                  | empty (upload a tar)         | commented out           |
| `MIKROSCOPE_NAME`         | `--name`         | the deployment verbs                                  | `mikroscope`                 | commented out           |
| `MIKROSCOPE_VETH` | `--veth` | the deployment verbs | `veth-mikroscope` | commented out |
| `MIKROSCOPE_SUBNET` | `--subnet` | the deployment verbs, `record`, `forward` | `172.30.10.0/30` | commented out |
| `MIKROSCOPE_IFACE_LIST` | `--iface-list` | the deployment verbs | `LAN`; `none` joins no list | commented out |
| `MIKROSCOPE_ADDR_LIST` | `--addr-list` | the deployment verbs | `LANs`; `none` joins no list | commented out |
| `MIKROSCOPE_DISK`         | `--disk`         | the deployment verbs                                  | empty (internal flash)       | commented out, `tmpfs`  |
| `MIKROSCOPE_LAN_ADDRESS`  | `--lan-address`  | the deployment verbs                                  | empty                        | commented out           |
| `MIKROSCOPE_TOKEN`        | `--token`        | the deployment verbs, `record`, `forward`             | empty                        | commented out           |

`MIKROSCOPE_SSH_OPTIONS` is a comma-separated list of `Key=value` pairs, the
same as repeating `--ssh-option`; the first `--ssh-option` on the command line
replaces the whole list. [CLI](https://docs.jmrp.io/mikroscope/reference/cli/) has
the keys it takes.

`MIKROSCOPE_TOKEN` is two things at once. For `install` and `upgrade` it is the
token written into the agent's envlist, which the agent then requires; the CLI
hands the command that writes it to ssh on its standard input, never on a
command line. `status` and `uninstall` do not need it. For `record`,
`forward` and `doctor`'s health section it is the token sent to the agent over
the direct transport; without it, `doctor` skips the ring of an agent that has
one. The envlist is not
a secret store: any RouterOS user with the `read` policy can list every
container's envlist over the API, which is why the token is the only
credential that goes there.

## Agent and API access

| Variable                  | Flag           | Read by                     | Meaning                                                                                            |
| ------------------------- | -------------- | --------------------------- | -------------------------------------------------------------------------------------------------- |
| `MIKROSCOPE_API_ADDR`     | `--api`        | `record`, `mark`, `forward` | RouterOS binary API `host:port`; [`.env.example`](https://github.com/jmrplens/mikroscope/blob/main/.env.example) has `192.168.88.1:8728`                            |
| `MIKROSCOPE_API_USER`     | `--api-user`   | `record`, `mark`, `forward` | the dedicated API user                                                                             |
| `MIKROSCOPE_API_PASSWORD` | none           | `record`, `mark`, `forward` | that user's password; there is deliberately no flag                                                |
| `MIKROSCOPE_INTERFACES`   | `--interfaces` | `forward`                   | comma-separated interfaces for `monitor-traffic`; [`.env.example`](https://github.com/jmrplens/mikroscope/blob/main/.env.example) has `bridge,ether1` commented out |

The relay, `--log-markers` and the API tier fail, or in `forward`'s case are
disabled with a warning, unless `--api`, `--api-user` and
`MIKROSCOPE_API_PASSWORD` are all set. [The API
user](https://docs.jmrp.io/mikroscope/security/api-user/) has the policy that user needs.

## Sinks

| Variable                   | Flag            | Default in the code | Meaning                                                                          |
| -------------------------- | --------------- | ------------------- | -------------------------------------------------------------------------------- |
| `MIKROSCOPE_INFLUX_URL`    | `--influx`      | empty               | InfluxDB 3 server, `http://host:8181`; a full write URL is still taken verbatim  |
| `MIKROSCOPE_INFLUX_DB`     | `--influx-db`   | empty               | the database `--influx` writes to, when `--influx` is a bare server               |
| `MIKROSCOPE_POSTGRES_DSN`  | `--postgres`    | empty               | PostgreSQL connection string for the connecting SQL sink                          |
| `MIKROSCOPE_LOKI_URL`      | `--loki`        | empty               | Loki push URL                                                                    |
| `MIKROSCOPE_LOKI_TENANT`   | `--loki-tenant` | empty               | `X-Scope-OrgID`                                                                  |
| `MIKROSCOPE_OTLP_URL`      | `--otlp`        | empty               | OTLP/HTTP metrics endpoint                                                       |
| `MIKROSCOPE_GRAPHITE_ADDR` | `--graphite`    | empty               | carbon plaintext `host:port`                                                     |
| `MIKROSCOPE_ELASTIC_URL`   | `--elastic`     | empty               | Elasticsearch or OpenSearch base URL                                             |
| `MIKROSCOPE_TELEGRAF_URL`  | `--telegraf`    | empty               | Telegraf listener URL (`http://`, `tcp://` or `udp://`)                          |
| `MIKROSCOPE_HOST_TAG`      | `--host-tag`    | `router`            | host tag on every point in every sink; [`.env.example`](https://github.com/jmrplens/mikroscope/blob/main/.env.example) has it commented out |

Setting a sink's URL variable is enough to turn that sink on: `forward` builds
every sink whose flag ends up non-empty, from the command line or from the
variable. Every one of these names carries the `MIKROSCOPE_` prefix: a bare
`LOKI_URL` is read by nothing.

## Credential variables

A flag is visible in `ps` and in a shell history, so every secret the CLI uses
is read from the environment only.

| Variable                    | Used for                                          |
| --------------------------- | ------------------------------------------------- |
| `MIKROSCOPE_API_PASSWORD`   | the RouterOS API user                             |
| `MIKROSCOPE_INFLUX_TOKEN`   | InfluxDB, sent as `Authorization: Bearer <token>` |
| `MIKROSCOPE_LOKI_TOKEN`     | Loki                                              |
| `MIKROSCOPE_OTLP_TOKEN`     | the OTLP endpoint                                 |
| `MIKROSCOPE_ELASTIC_AUTH`   | Elasticsearch or OpenSearch                       |
| `MIKROSCOPE_TELEGRAF_TOKEN` | the Telegraf HTTP listener                        |
| `GRAFANA_TOKEN`             | `dashboards publish`, `import` and `check`, `forward --grafana`, `uninstall --targets dashboard` |

## Grafana

`GRAFANA_TOKEN` is the one Grafana token every verb uses, with no prefix, and no flag takes it.
The Grafana itself is `--grafana`, and each verb looks for a default in a different order:

| Verb                                                  | Reads, after `--grafana`                     |
| ----------------------------------------------------- | -------------------------------------------- |
| `dashboards publish`, `uninstall --targets dashboard` | `MIKROSCOPE_GRAFANA_URL`, then `GRAFANA_URL` |
| `dashboards import`, `dashboards check`               | `GRAFANA_URL`, then `MIKROSCOPE_GRAFANA_URL` |
| `forward`                                             | `MIKROSCOPE_GRAFANA_URL` only                |

`forward` never reads `GRAFANA_URL`, so a `GRAFANA_URL` on its own does not make the collector
publish. Other Grafana tools read that name and `GRAFANA_TOKEN` too, and a collector started from a
shell set up for one of them would start writing a folder, a datasource and a dashboard into that
Grafana without anyone asking it to. The file [`.env.example`](https://github.com/jmrplens/mikroscope/blob/main/.env.example) carries every Grafana
variable, commented out, in its two Grafana blocks.

`forward`, `dashboards publish` and `uninstall` take `--grafana` and the flags below, and their
variables **do** carry the prefix, because they are collector settings rather than the
`dashboards import` and `check` ones:

| Variable                                | Flag                           | Default in the code | Meaning                                                                                                   |
| --------------------------------------- | ------------------------------ | ------------------- | --------------------------------------------------------------------------------------------------------- |
| `MIKROSCOPE_GRAFANA_URL`                | `--grafana`                    | empty               | the Grafana to publish to; empty publishes nothing                                                        |
| `MIKROSCOPE_GRAFANA_FOLDER`             | `--grafana-folder`             | `mikroscope`        | folder to publish into; an empty variable is ignored, so the General folder takes `--grafana-folder ""`  |
| `MIKROSCOPE_GRAFANA_DATASOURCE_UID`     | `--grafana-datasource-uid`     | empty               | adopt this datasource instead of creating one; one store per run                                          |
| `MIKROSCOPE_GRAFANA_DATASOURCE_URL`     | `--grafana-datasource-url`     | empty               | the address Grafana queries, for the sinks that cannot know it; one store per run                         |
| `MIKROSCOPE_GRAFANA_DATASOURCE_SSLMODE` | `--grafana-datasource-sslmode` | empty               | sslmode for the PostgreSQL datasource: `disable`, `require`, `verify-ca` or `verify-full`, nothing else |

`uninstall` reads only `MIKROSCOPE_GRAFANA_URL` and `MIKROSCOPE_GRAFANA_DATASOURCE_UID` of these,
and ignores the others.

A run that publishes more than one store and sets `MIKROSCOPE_GRAFANA_DATASOURCE_UID` or
`MIKROSCOPE_GRAFANA_DATASOURCE_URL` is refused before any request: each names one datasource.
Publish the store that needs one on its own with `dashboards publish` ([Grafana
publishing](https://docs.jmrp.io/mikroscope/reference/cli/#publishing-to-grafana)).

Publishing refuses to write without `GRAFANA_TOKEN`: some Grafanas accept an anonymous request, and
one that did would write as whoever the server thinks is asking. `--grafana-dry-run` writes nothing
and sends no request, so it runs without the token.

## Agent envlist

The agent reads these from its container's environment when it starts. An
invalid value makes it print one line starting
`mikroscope-agent: bad configuration:`, which RouterOS puts in its log under
the `container` topic, and exit with status 2.

| Variable               | Agent default                                      | Accepted                    | Meaning                                                                                                                                     |
| ---------------------- | -------------------------------------------------- | --------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------- |
| `RATE_HZ`              | `10`                                               | 1–100                       | sampler rate                                                                                                                                |
| `BUFFER_S`             | `60`                                               | 10–3600                     | ring length in seconds                                                                                                                      |
| `PORT`                 | `9123`                                             | 1–65535                     | HTTP port                                                                                                                                   |
| `ADDR`                 | empty                                              | an address                  | bind address. Empty binds `:PORT`, every address in the container's network namespace; `install` always sets it to the agent's veth address |
| `TOKEN`                | empty                                              |                             | when set, every path but `/healthz` requires `Authorization: Bearer <token>`                                                                |
| `IRQ_TOP_K`            | `8`                                                | 0–64                        | interrupt lines kept per sample, busiest first; `0` ships no interrupt lines and no `irq_total`                                             |
| `MEM_LIMIT_MB`         | `14`                                               | 8–1024                      | Go soft memory limit in MiB                                                                                                                 |
| `FLOOR_HZ`             | `0`                                                | 0–1000                      | `0` keeps the per-source floors; N puts every level source on N Hz and turns off emit-on-change                                             |
| `SOURCES`              | empty (every source detected)                      | comma-separated names       | restrict the optional sources to this list; `/proc/stat` is always read                                                                     |
| `TRIGGERS`             | `softnet-drop,oom,kmsg<=3,reset,irq-err,flash-bad` | conditions, comma-separated | capture trigger conditions                                                                                                                  |
| `CAPTURE_MB`           | `4`                                                | 0–256                       | pinned-bytes budget for captures; `0` turns them off                                                                                        |
| `CAPTURE_PRE_S`        | `5`                                                | 1–60                        | seconds kept before the sample a trigger fires on                                                                                           |
| `CAPTURE_POST_S`       | `5`                                                | 1–60                        | seconds kept after it                                                                                                                       |
| `CAPTURE_POLICY`       | `first`                                            | `first` or `last`           | on a full budget: `first` refuses the new capture, `last` evicts the oldest                                                                 |
| `TRIGGER_REFRACTORY_S` | `10`                                               | 0–3600                      | quiet time per condition after it fires                                                                                                     |
| `PROC_ROOT`            | `/proc`                                            | a path                      | where `/proc` is read from; the kernel log is `/dev/kmsg` only when this is `/proc`, otherwise `dev/kmsg` beside the given directory        |
| `SYS_ROOT`             | `/sys`                                             | a path                      | where `/sys` is read from                                                                                                                   |

The names `SOURCES` accepts are the optional sources `/capabilities` reports:
`meminfo`, `loadavg`, `softnet`, `softirqs`, `interrupts`, `vmstat`, `psi`,
`schedstat`, `self`, `buddyinfo`, `yaffs`, `diskstats`, `slabinfo`, `kmsg`,
`thermal`, `cpufreq`, `mtd` and `perf`. The trigger conditions and what each
compares are on [Triggered capture](https://docs.jmrp.io/mikroscope/record/triggers/).

Before it listens, the agent checks the ring against the container's own
`memory.max`: `RATE_HZ × BUFFER_S × 3 456 B` plus `CAPTURE_MB` must fit, or it
refuses to start and names the three variables and `--memory-max`. If that
figure is more than half of `MEM_LIMIT_MB`, it starts and logs a warning with
the limit to raise to. 3 456 B is not itself a measurement: it is the Go
allocator size class that serves a line with every source on
(3 230 B, 4 cores, `IRQ_TOP_K` 8), because that class
is what the heap is charged. A board with more cores or more interrupt lines
writes longer lines, so the check errs open. When the
agent cannot read its `memory.max`, the refusal check does not run.

### Written by install

`install` and `upgrade` write the envlist `<name>-env` with these entries, in
this order, and nothing else. The install's shape (its lists, disk and
exposure) is not in the envlist: `install` records it in the install manifest,
`mikroscope/<name>.manifest.txt` on the install's disk, which holds no secret
either.

The entries install writes into the agent's envlist:

| Key | Written | From | Holds |
| --- | --- | --- | --- |
| `MIKROSCOPE_TAG` | always | `--name` | the ownership marker `mikroscope:<name> (managed by mikroscope)`, written first and removed last; the agent ignores it |
| `RATE_HZ` | always | `--rate`, default `10`, 1–100 | the sampler rate, in Hz |
| `BUFFER_S` | always | `--buffer`, default `60`, 10–3600 | the ring's length, in seconds |
| `PORT` | always | `--port`, default `9123`, 1–65535 | the agent's HTTP port |
| `ADDR` | always | `--subnet` | the agent's address, the `.2` of the /30; the agent binds only there |
| `MEM_LIMIT_MB` | always | `--mem-limit-mb`, 8–1024 | the agent's Go soft memory limit, in MiB; derived from the ring (rate × buffer × line, × 2.5, at least 16 MiB, at most three quarters of `--memory-max` while that still holds the ring) unless `--mem-limit-mb` gives it |
| `FLOOR_HZ` | only when above 0 | `--floor-hz`, default `0`, 0–1000 | one cadence for every level source, in Hz |
| `CAPTURE_MB` | always | `--capture-mb`, default `4`, 0–256 | the triggered-capture budget, in MiB; `0` turns captures off |
| `TRIGGERS` | only when set | `--triggers` | the trigger conditions; unset, the agent uses its default set |
| `TOKEN` | only when set | `--token` | the bearer token the agent requires, from `--token` or `MIKROSCOPE_TOKEN`, with or without `--expose` |

> **The agent's defaults and install's defaults are not the same**
>
> An agent started with no envlist entry uses a 14 MiB soft memory limit. The ring is **60 s** by
> default. What it buys is how long the collector may be absent before samples are lost, and it is
> not a window anybody reads: the collector drains it twice a second. 60 s covers a restart of
> either side on a LAN and costs 2.0 MiB of ring at 10 Hz. A link that drops for longer raises it
> with `--buffer`, and the memory limit follows because it is derived from the ring.
>
> `install` **derives** `MEM_LIMIT_MB` from the ring: rate × buffer × the line size, times 2.5,
> floored at 16 MiB and capped at three quarters of `memory-max`. The default install writes
> `MEM_LIMIT_MB=16` for a 60 s ring at 10 Hz. A fixed number cannot be right for every rate, and
> below about twice the ring the Go collector runs so often that the CPU per sample climbs steeply
> ([Agent cost](https://docs.jmrp.io/mikroscope/cost/#size-the-memory-limit-to-the-data)).
>
> `IRQ_TOP_K`, `CAPTURE_PRE_S`, `CAPTURE_POST_S`, `CAPTURE_POLICY`, `TRIGGER_REFRACTORY_S`,
> `SOURCES`, `PROC_ROOT` and `SYS_ROOT` have no flag, so an installed agent runs with their
> defaults.

## See also

- [CLI](https://docs.jmrp.io/mikroscope/reference/cli/): every flag, including the ones no variable
  sets.
- [API user](https://docs.jmrp.io/mikroscope/security/api-user/): the RouterOS user behind `MIKROSCOPE_API_USER`.
- [Source cadences](https://docs.jmrp.io/mikroscope/limits/source-floors/): what `FLOOR_HZ` overrides.
- [Agent cost](https://docs.jmrp.io/mikroscope/cost/): why the memory limit is sized to the ring.