E2E coverage
What the live end-to-end suite exercised in its last recorded full run: every catalog action on each of the three tool surfaces, against real Arcane v2.17.2 instances in the Docker-in-Docker estate. The figures come from test/e2e/coverage-record.json, which make e2e-coverage-record folds from the ledger the suite writes while it runs; make check-e2e-coverage checks the record, this page and the suite’s sources against the catalog without an estate. Testing explains how the suite works.
Recorded run
Section titled “Recorded run”| Date | 2026-11-02 |
| Commit | 245296129b6a |
| Working tree | clean |
| Arcane | v2.17.2 |
| Go | 1.27.3 |
| Duration | 18 min 13 s |
| Scenario subtests | 186: 186 passed, 0 failed, 0 skipped |
Summary
Section titled “Summary”455 of 461 actions in 52 domains succeeded on all three surfaces. Every action has exactly one status:
| Status | Actions | Meaning |
|---|---|---|
| covered | 455 | A call succeeded on the dynamic, meta and individual surfaces, in a test that passed. |
| partial | 0 | Succeeded on some surfaces only. |
| cleanups only | 0 | Succeeded only as an end-of-scenario cleanup, whose result no scenario asserts. |
| failure paths only | 0 | Called, but every call failed. |
| host-dependent | 0 | Failed only with an error declared as the host’s network, not the action (for example a public registry this host cannot reach). |
| untested | 6 | Listed in test/e2e/untested.yaml: the estate cannot exercise it, for the reason given below. |
| not exercised | 0 | No scenario call reached it. |
Besides succeeding, 89 actions have a failure path asserted too: a MustFail call that had to fail did (a missing resource, a refused request, a guard).
By domain
Section titled “By domain”| Domain | Actions | Covered | Partial | Cleanups only | Failure paths only | Host-dependent | Untested | Not exercised |
|---|---|---|---|---|---|---|---|---|
activity | 4 | 4 | – | – | – | – | – | – |
api_key | 8 | 8 | – | – | – | – | – | – |
app_image | 4 | 4 | – | – | – | – | – | – |
auth | 9 | 9 | – | – | – | – | – | – |
backup | 17 | 16 | – | – | – | – | 1 | – |
build | 6 | 6 | – | – | – | – | – | – |
container | 23 | 23 | – | – | – | – | – | – |
customize | 2 | 2 | – | – | – | – | – | – |
dashboard | 1 | 1 | – | – | – | – | – | – |
diagnostics | 12 | 12 | – | – | – | – | – | – |
environment | 15 | 15 | – | – | – | – | – | – |
event | 4 | 4 | – | – | – | – | – | – |
federated_credential | 6 | 6 | – | – | – | – | – | – |
git_repository | 9 | 9 | – | – | – | – | – | – |
gitops | 13 | 13 | – | – | – | – | – | – |
image | 18 | 17 | – | – | – | – | 1 | – |
image_update | 7 | 7 | – | – | – | – | – | – |
job | 11 | 10 | – | – | – | – | 1 | – |
mobile_push | 6 | 6 | – | – | – | – | – | – |
network | 9 | 9 | – | – | – | – | – | – |
notification | 6 | 6 | – | – | – | – | – | – |
oidc | 10 | 10 | – | – | – | – | – | – |
passkey | 21 | 21 | – | – | – | – | – | – |
podman | 6 | 6 | – | – | – | – | – | – |
podman_pod | 8 | 8 | – | – | – | – | – | – |
podman_quadlet | 7 | 7 | – | – | – | – | – | – |
podman_secret | 5 | 5 | – | – | – | – | – | – |
port | 1 | 1 | – | – | – | – | – | – |
project | 26 | 26 | – | – | – | – | – | – |
registry | 11 | 11 | – | – | – | – | – | – |
role | 8 | 8 | – | – | – | – | – | – |
s3_destination | 10 | 10 | – | – | – | – | – | – |
server | 3 | 3 | – | – | – | – | – | – |
settings | 5 | 5 | – | – | – | – | – | – |
stream | 1 | 1 | – | – | – | – | – | – |
swarm | 13 | 13 | – | – | – | – | – | – |
swarm_config | 5 | 5 | – | – | – | – | – | – |
swarm_node | 12 | 12 | – | – | – | – | – | – |
swarm_secret | 5 | 5 | – | – | – | – | – | – |
swarm_service | 9 | 9 | – | – | – | – | – | – |
swarm_stack | 9 | 9 | – | – | – | – | – | – |
swarm_task | 1 | 1 | – | – | – | – | – | – |
system | 11 | 8 | – | – | – | – | 3 | – |
template | 15 | 15 | – | – | – | – | – | – |
updater | 5 | 5 | – | – | – | – | – | – |
upload | 4 | 4 | – | – | – | – | – | – |
user | 6 | 6 | – | – | – | – | – | – |
variable | 6 | 6 | – | – | – | – | – | – |
volume | 13 | 13 | – | – | – | – | – | – |
volume_backup | 14 | 14 | – | – | – | – | – | – |
vulnerability | 16 | 16 | – | – | – | – | – | – |
webhook | 5 | 5 | – | – | – | – | – | – |
| Total | 461 | 455 | 0 | 0 | 0 | 0 | 6 | 0 |
By surface
Section titled “By surface”Each action on each surface is one cell, judged from the calls the scenarios made through that surface:
| State | dynamic | meta | individual |
|---|---|---|---|
| Succeeded: a call that was not a cleanup succeeded in a test that did not fail | 455 | 455 | 455 |
| Test failed: such a call succeeded, but only in tests that failed | 0 | 0 | 0 |
| Cleanup only: only end-of-scenario cleanups succeeded | 0 | 0 | 0 |
| Failure only: every call failed | 4 | 4 | 4 |
| Not called: no call | 2 | 2 | 2 |
Actions not covered on every surface
Section titled “Actions not covered on every surface”The reasons are quoted from test/e2e/untested.yaml and from the host-dependent declarations in cmd/audit_e2e_coverage.
| Action | Status | Surfaces | Reason |
|---|---|---|---|
backup.restore | untested | dynamic: failure onlymeta: failure onlyindividual: failure only | Arcane finds its own container through its cgroup before restoring. The estate managers do not run in the daemon they manage, and an Arcane run inside the estate’s Docker-in-Docker resolves the outer dind container, which the inner daemon does not know (‘inspect Arcane container: No such container’). TestPlatform_PrivateManager covers everything up to it and calls restore, checking the result if it ever succeeds. |
image.patch | untested | dynamic: failure only (expected failure)meta: failure only (expected failure)individual: failure only (expected failure) | Needs a daemon on the containerd image store (Copacetic uses BuildKit’s docker exporter); the estate’s docker:dind daemons use overlay2, so Arcane refuses before patching. The core suite asserts that refusal. |
job.resolve_run | untested | dynamic: failure only (expected failure)meta: failure only (expected failure)individual: failure only (expected failure) | Deprecated mixed-version compatibility route (removed in Arcane v3): only needs_attention runs can be resolved, and Arcane (v2.16.1 to v2.17.2) records every such outcome as failed (pkg/scheduler/runs/actors.go), so it only succeeds for a run of an agent older than v2.16 (verified against a v2.14.0 agent, plan/upstream-bugs/job-resolve-run-unreachable). The estate runs v2.17.1 agents only; the suite asserts the refusal. |
system.update_all | untested | dynamic: not calledmeta: not calledindividual: not called | Same as system.upgrade, for every environment starting with the manager. |
system.update_all_status | untested | dynamic: failure onlymeta: failure onlyindividual: failure only | Answers 404 ‘no update-all job found’ until system.update_all has run, which the estate cannot do (see system.update_all). The core suite accepts the 404. |
system.upgrade | untested | dynamic: not calledmeta: not calledindividual: not called | Upgrades the Arcane server itself by replacing its own container, which it must find on the daemon it manages; the estate managers run outside the daemons they manage, and replacing the server under test would end the run. |
Skipped subtests
Section titled “Skipped subtests”None.