# Cadences The three layers of the every block, the built-in cadence of every family, the warning when a configuration speeds one up, and the heartbeat. Source: https://docs.jmrp.io/ghchronicle/configuration/cadences/ Each family of GitHub metrics is collected on its own interval, and the `every` block is where those intervals are changed. ```yaml every: default: 15m groups: ci: 1m feeds: 10m families: actions: 30s ``` ## The three layers Most specific wins. A family's own entry beats its group's, a group's beats `default`, and `default` beats the built-in value. Anything omitted falls through to the layer below it, so a file with only `families:` in it behaves exactly as it always did. The value is a Go duration: `30s`, `15m`, `2h`, `36h`. | Layer | Reaches | Written as | | ------------------ | ----------------------------------- | ----------------------- | | `every.families` | one family | `families: {keys: 24h}` | | `every.groups` | every family of one group | `groups: {ci: 1m}` | | `every.default` | every family neither above names | `default: 15m` | | the built-in table | every family none of the above name | nothing | They are three keys of a nested block rather than three flat ones because a flat map cannot hold this vocabulary. `security` and `account` are each both a family name **and** a group name, so `every: {security: 1m}` has two readings and no way to choose between them. Under `families:` the word is the family; under `groups:` it is the group; there is nowhere left for the question to arise. The same nesting is what makes `default` and `groups` safe as words: they are fields of a fixed structure, and an unknown key is refused at start-up, so no family can ever shadow a layer and no layer can ever shadow a family. > **The two broad layers never switch a family on** > > `default` and `groups` cannot reach a family whose built-in cadence is `0`. > `deps`, `history` and `joblogs` ship switched off and are enabled by naming > them under `families:` and by nothing else. `deps` alone is 1.8 MB of SBOM per > repository, and a default written to speed up the fast families must not also > switch on three families you never mentioned. It is the same rule `groups` > already follows. Turning it around gives the shortest way to collect a little: switch everything off with `default`, then name what you want back. ```yaml every: default: 0 families: traffic: 6h actions: 15m ``` ## Every family, its group and its built-in cadence The built-in values are not round numbers picked for tidiness. They came out of a costed audit, and the reason each one is what it is lives next to the number in the code. That is the source, and this table is pinned to it: a test in `internal/config` compares every group, every duration and every reason below against the code in both directions, so a family added, moved or re-costed without this table following it fails the build. `ghchronicle -groups` prints the same membership. | Group | Family | Default | Why that value | | ----------- | -------------- | ------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `account` | `account` | `1h` | the contribution calendar moves with every contribution and the profile's counts with every follow and star, and a pass is one GraphQL point and seven REST requests, of which only the profile is charged: GitHub all but never answered it with a 304, once in 48 conditional reads, and the six package listings answer one until a package changes | | `account` | `achievements` | `1h` | the badges on the public profile page, which no API lists, and how far each tiered one is from its next tier; a pass transfers the page, 36 KB and off the budget, and the pull requests merged since the day before for two GraphQL points, so a tier reached in the afternoon shows in the afternoon, and the whole history is walked once a week, 24 MB | | `account` | `billing` | `1h` | the month in progress moves while continuous integration runs: it had changed at every one of 46 six-hourly reads measured, and a pass is two requests, that month and the one before, of which only the first is charged | | `account` | `history` | `0` | off until asked for by name: it walks every past year and the year so far, and the rows are idempotent | | `account` | `keys` | `24h` | an SSH or GPG key changes when somebody changes it, and what matters is its expiry date, not the hour it was noticed | | `account` | `outbound` | `1h` | stars given, work in other people's repositories and the answers accepted there, for about nine GraphQL points a pass, so the hour costs nothing that shows and an answer accepted this afternoon is on the dashboard this afternoon | | `account` | `profile` | `12h` | packages, gists and social accounts, all of them edited by hand | | `account` | `totals` | `1h` | the lifetime numbers move with every star, fork, merge and push, and a pass is one GraphQL point for the account, one per ten repositories and one per twenty-five archived ones set aside, and one request of the search budget | | `audience` | `forks` | `12h` | the whole list fits in one page, and a fork is a rare event | | `audience` | `stars` | `1h` | a star can land at any hour, and after the one full stargazer walk a pass is one GraphQL point per ten repositories for the newest hundred and one conditional request per repository for the daily history, which answered a free 304 to 184 of 185 measured | | `audience` | `traffic` | `6h` | the fourteen-day window is rewritten whole each time, so a missed sweep repairs itself on the next one | | `ci` | `actions` | `15m` | a workflow run is over in minutes, and its queue time is only worth watching while it is happening | | `ci` | `artifacts` | `1h` | artifacts appear with the run that made them and expire on a scale of days | | `ci` | `deployments` | `30m` | the surface a delivery dashboard reads, and the newest page is one GraphQL point per five repositories, which brought up to four new rows a pass measured, so the half hour costs little | | `ci` | `joblogs` | `0` | off until asked for by name: it is text rather than a measurement, it costs a request per failure, and it only makes sense with a log store attached | | `collector` | `ratelimit` | `15m` | free, and worth having at the resolution of the busiest family | | `feeds` | `activity` | `15m` | the repository log holds a hundred entries, which covered twenty-six hours on the busiest repository measured, and a pass is one or two conditional requests per repository, which answered a free 304 to 2,320 of 2,356 measured | | `feeds` | `events` | `15m` | one core request a pass, and the feed had moved in 43 of 46 half hours measured, so the quarter hour is how soon an event reaches the dashboard; the window, the last three hundred events of the past thirty days, is far wider than that | | `feeds` | `notifs` | `15m` | one core request a pass and twenty once a day for the whole inbox, and a thread shows only its latest move, so a move overtaken before the next read is never seen; half the passes measured brought a new row | | `repos` | `branches` | `24h` | branches are created and deleted all day, but the question the row answers is which are stale right now, which is a daily one | | `repos` | `deps` | `0` | off until asked for by name: the SBOM is 1.8 MB per repository and has its own budget of a hundred a minute | | `repos` | `inventory` | `24h` | four core requests per repository, for settings that change only when somebody changes them | | `repos` | `policyfiles` | `24h` | SECURITY.md, CODEOWNERS, dependabot.yml and FUNDING.yml move about once a quarter | | `repos` | `repo` | `1h` | stars, forks, languages and topics move slowly, and a pass is three REST requests per repository, most of them a free 304, and two GraphQL points per ten repositories | | `repos` | `rulesets` | `24h` | a ruleset is edited a few times a year and every version keeps its own date; both requests answer a free 304 until somebody edits one, the first pass after a restart included, since the ETag cache is kept beside the state file | | `repos` | `settings` | `6h` | webhooks, rulesets, environments and deploy keys change only when somebody changes them | | `security` | `analyses` | `1h` | GitHub prunes code scanning analyses and every scanned push adds some, and a pass is one conditional request per repository with code scanning, three of them charged in the median pass measured, the refusals of the rest remembered for a day | | `security` | `security` | `1h` | an alert is something to act on today, and the list of open ones is short | | `work` | `commits` | `1h` | one GraphQL point per repository whose default branch has a commit from the last two cadences, which 38 of 999 answers measured had, and one per twenty-five repositories, shared with issues and issueevents, to ask which, so the hour is how soon a push is charted | | `work` | `discussions` | `1h` | a discussion is answered over hours or days and few repositories have a forum, and a pass is two GraphQL points for each that does and nothing for the rest | | `work` | `issueevents` | `1h` | the timeline of what moved in two cadences, one GraphQL point for each repository where an issue or a pull request moved and nothing for the rest, so the hour is how soon a transition is worth seeing | | `work` | `issues` | `1h` | one or two GraphQL points per repository whose issues or pull requests moved in two cadences, nothing for the rest, and once a day a point or more for its open items and up to four a page for what moved in the day, so the hour is how soon a review or a merge is charted | | `work` | `planning` | `6h` | labels and milestones are edited by hand, a few times a week at most | | `work` | `stats` | `12h` | GitHub recomputes these slowly anyway, so asking more often returns the same numbers | ## The warning when a cadence is faster than the value is worth A broad layer is a cheap way to slow everything down and an expensive way to speed everything up. `default: 15m` asks GitHub for the account's SSH keys ninety-six times a day for a value that changes twice a year, and a group cadence does the same thing one level down: `work` holds families the audit measured at `1h` and at `12h`, so one number there is one number over six different answers. So every family a configuration collects **four times more often or more** than its built-in cadence is named at start-up, with the key that set it, both numbers, and the reason that number is what it is: ```text level=WARN msg="every.default sets keys to 15m against a built-in 24h, 96 times more often: an SSH or GPG key changes when somebody changes it, and what matters is its expiry date, not the hour it was noticed" level=WARN msg="every.groups.work sets stats to 1h against a built-in 12h, 12 times more often: GitHub recomputes these slowly anyway, so asking more often returns the same numbers" ``` It is a warning and never a refusal, and it is always per family, never per group. A line saying "the group `work` is too fast" would name nothing you can act on: the reason a cadence is what it is belongs to the family, so the warning has to as well. Four is the threshold because the built-in values stand on a ladder, `15m` `30m` `1h` `2h` `6h` `12h` `24h`, and the widest gap between two neighbouring rungs is three, from `2h` to `6h`. No family ships at `2h` since `discussions` moved to the hour, but the rung is still where one step down from `6h` lands. Four is therefore the smallest factor no single step down that ladder can reach. Moving one rung is a deliberate adjustment made by somebody looking at that family and stays quiet; four or more can only be a broad layer landing somewhere it was never chosen for, or a number typed without reading this table. Nothing warns for going slower. This is about waste, not about taste. ## Groups: collecting less than everything `every` sets how often a family runs. `groups` sets which families exist for this deployment at all. Omit the key and it collects for every group, which is the default and is what "every metric GitHub exposes" has always meant. ```yaml groups: [audience, account, repos, security] ``` Naming any group turns the rest off: their families are never requested and their measurements are never written. `ghchronicle -groups` prints the list, which is: | Group | Families | | ----------- | ---------------------------------------------------------------------------------------- | | `audience` | `forks`, `stars`, `traffic` | | `account` | `account`, `achievements`, `billing`, `history`, `keys`, `outbound`, `profile`, `totals` | | `repos` | `branches`, `deps`, `inventory`, `policyfiles`, `repo`, `rulesets`, `settings` | | `work` | `commits`, `discussions`, `issueevents`, `issues`, `planning`, `stats` | | `ci` | `actions`, `artifacts`, `deployments`, `joblogs` | | `security` | `analyses`, `security` | | `feeds` | `activity`, `events`, `notifs` | | `collector` | `ratelimit` | The top-level `groups` and `every.groups` are different questions about the same eight names: the first decides whether a group is collected at all, the second how often. A cadence under `every.groups` for a group the top-level `groups` leaves out is legal and warns, rather than failing: narrowing a deployment should not also mean pruning an `every` block you tuned last year. The two axes both have to say yes, and only one of them can say no. A group that is not named switches its families off whatever `every` says, and naming a group never resurrects a family whose cadence is zero. That is the whole of how a family ships switched off while the default is everything. Writing `groups: []` is refused rather than read as "collect nothing": omitting the key is how you ask for everything, so an empty list can only be a mistake. What GitHub deletes cannot be recovered later, whatever you do afterwards, and it sits in three different groups: the event feed, which keeps three hundred events and none older than thirty days, and the inbox, which keeps notifications for three months unless they are saved (`feeds`); the fourteen-day traffic window (`audience`); and job logs, deleted after the repository's retention period, ninety days by default, along with, from 1 October 2026, the workflow runs themselves (`ci`). Whatever GitHub dropped before ghchronicle read it does not exist anywhere. Everything else can be filled in later with `-backfill`. > **Panels for a group you switched off will show an error** > > A measurement that is never written does not exist as a table, and InfluxDB 3 > answers a query naming a table it does not have with an error rather than with > no rows. The dashboards are generated once, for the full set of metrics, so > every panel fed by a group you turned off shows that error. It is expected. > They are a demonstration of what the collector can chart, not a view that > reshapes itself around your configuration. PostgreSQL and Elasticsearch behave > the same way; Prometheus and Graphite have no schema to miss, so the same > panels there read No data instead. The dashboard sections are not aligned to the groups either, so a group switched off empties some panels of several sections rather than one section cleanly. `Stars and forks` reads `gh_repo` from `repos` as well as `gh_star` and `gh_star_day` from `audience`; `Code` reads `gh_workflow_run` from `ci` and `gh_repo_activity` from `feeds` alongside its own commits; `Inventory` reads from `repos` and `account`, and its licence panels read `gh_dependency_license`, which is family `deps` and ships switched off whether or not `repos` is selected. ## What `0` means `0` switches off every family the layer it is written on reaches. It is not "as often as possible" and not "use the default": those families never run, write nothing and cost nothing. ```yaml every: families: billing: 0 # no billing data at all artifacts: 0 # no artifact rows ``` Three families ship off and are enabled by giving them any duration, under `families:` and nowhere else: - **`joblogs`** is the tail of every failed job's log. It is text rather than a measurement, it costs a request per failure, and it only makes sense with a log store attached. The InfluxDB sink excludes it by default. - **`history`** walks every past year's contribution calendar, one GraphQL point per year, back to the day the account was created, and the year in progress from January to now. The past years never change and the rows are idempotent; the current year's row is a snapshot marked `partial`, so a daily cadence keeps it current and a single sweep leaves it frozen on the day it ran. - **`deps`** is the dependency graph. The SBOM is 1.8 MB per repository and has its own budget of a hundred a minute. A configuration that leaves nothing at all enabled says so at start-up rather than running an empty loop in silence. > **An unknown name is fatal at start-up** > > ```text > every.families.trafic: unknown collector (known: account, achievements, actions, activity, ...) > every.families.feeds: "feeds" is a group, not a family; every.groups.feeds is where a whole group's cadence lives > every.groups.actions: "actions" is a family, not a group; it is in group "ci", and every.families.actions is where its cadence lives > ``` > > Every name is checked against the known collectors and the known groups before > the first API call. A typo would otherwise mean a family silently running at its > default forever, and a name written under the wrong layer is told which layer it > belonged under. ## `heartbeat`: the loop's tick, which is not a cadence `heartbeat` forces how often the sweep loop wakes to ask which families are due. It gives no family an interval, it is compared against nothing in the table above, and it earns none of the warnings on this page. ```yaml heartbeat: 15s ``` Omit it and the loop ticks at the shortest cadence configured, held between one minute and one hour, which is what a real deployment wants: a family that runs every quarter of an hour is not delayed by one that runs every twelve hours. The upper end matters to a configuration whose cadences are all slower than an hour: the search for the shortest one starts at an hour, so the loop still wakes hourly and finds nothing due. Set it when you want the loop itself under control, which is almost always a test run. It is the only way to turn the loop faster than that one minute floor, because the floor exists to survive a mistyped cadence and an explicit heartbeat is not one. A family is due on the first tick at which its cadence has elapsed, to within half a tick. The loop reads the clock a few milliseconds after it wakes, by an amount that changes from tick to tick, and without that margin a family whose cadence is a whole number of ticks waited one tick more about half the time: until 2.6.0, which shipped the fix written for 2.5.2, the quarter hour families ran every 24 minutes on average and the hourly ones every 69. Half a tick absorbs that and never lets a family run a tick early. A heartbeat **longer** than the shortest cadence holds that family back, and start-up says so: ```text level=WARN msg="heartbeat is 1h and the shortest cadence is 15m (actions), so no family can run more often than every 1h" ``` ## The slow families take turns The running service starts one family whose cadence is six hours or more in each sweep, or the fewest that still keep every cadence when one cannot (below), and leaves any other that is due for the next tick. Families that run in the same sweep are marked with the same instant in the state file, so they come due together again at every cadence, for ever. A group like that forms whenever many families are marked at once: a fresh install, a `-once` or a backfill run before the service, a family switched on. Until 2.6.0 the production account's eight daily families ran in the first sweep of the UTC day, every day, and its five twelve-hour ones together in one sweep of their own. On 2026-09-26 the daily sweep took 294 billable `core` requests and 24.3 MB against 17.5 requests for the median sweep, and with the twelve-hour sweep 45 minutes after it, its hour was eight times the median hour in `core` requests. The calendar put them there, not the work. The family that starts is the one that has been due the longest, counted from the later of its last run and the last time the process let it start. The second half matters for a family whose every pass fails: a failed pass is not marked as run, so by its last run alone it would be the most overdue family on every sweep and would take every turn. Counted this way it goes behind the others. A family waits for each of the others at most once, so the worst wait is one tick for each other slow family, and only the last of a group that formed waits that long. At the quarter-hour tick the built-in cadences give: | Schedule | Families of 6h or more | Longest wait | | --------------------------------------------- | ---------------------- | ----------------- | | the built-in cadences | 11 | 10 ticks, `2h30m` | | with `deps` and `history` at `24h` | 13 | 12 ticks, `3h` | | with `deps`, `history` and `joblogs` at a day | 14 | 13 ticks, `3h15m` | That wait is paid once. Once two families have run in different sweeps they come due in different sweeps, and nothing moves them again: a simulated week of the thirteen, all due in one sweep at the start, had every family start exactly a cadence after its previous start from the first day on. The log says which family started and which are waiting, so a slow family missing from a sweep it was due in is accounted for: ```text level=INFO msg="slow families due together take turns" starting=planning waiting=settings,traffic,forks,profile,stats,branches,inventory,keys,policyfiles,rulesets ``` One a sweep holds while it can keep every cadence, and a configuration with a longer tick can have more slow families than that. `every.default: 6h` ticks hourly, because nothing in it runs more often, with thirty-one families at six hours, and one a sweep would start each of them every thirty-one hours. A sweep there starts the fewest that fit, six, and no family waits as long as its cadence. Only the service takes turns. `-once` runs every family that is due, because it has no next tick to leave one for: run once a day by a scheduler, it would leave it for a day. A backfill, a card and the first sweep of a service that primes the [Prometheus exporter](https://docs.jmrp.io/ghchronicle/sinks/prometheus/#scraping-it) run every family whatever the state file says. The primed sweep then records only the families that were due, so a restart does not put the others back on one instant, and each keeps the turn it had. ## Making them slower If `core` is the budget that runs short, lengthen `artifacts` and then `actions`, the two largest REST costs, which grow with how busy the repositories are. If it is `graphql`, lengthen `issues`, `issueevents` and `commits`, which read only the repositories where something moved and so cost more the more of them move. They are not the only ones that follow activity rather than size: `activity` is charged only for a repository whose log moved, and `outbound` a point more for each further hundred of what it reads. See [what scales with activity, not with size](https://docs.jmrp.io/ghchronicle/api/cost/#what-scales-with-activity-not-with-size). The symptom of cadences that are too fast is a warning every sweep: ```text level=WARN msg="rate limit reserve reached, family skipped" family=actions ``` ## Cadence is not resolution Lengthening a cadence does not coarsen the history, because the points are dated by the thing that happened rather than by the sweep. Collecting workflow runs every hour instead of every fifteen minutes still records each run at the second it finished. What a longer cadence risks is missing a window entirely: `events` reads the last three hundred events, `notifs` only the latest move of each thread and `activity` the last hundred entries of each repository's log, so whatever leaves one of them between two sweeps is gone.