Keel

Grounding report.

Every verification edge in keel, classified by who produces the signal and whether the actor being verified can write to that producer.

targetkeel
revision44f7e8d6cbdfaee7d0e4ff281d59a236ecf0c747
generated2026-07-25T01:28:17.000Z
0.36

anchored / (anchored + self_referential + unknown) = 5 / 14

anchored 5 self_referential 9 unknown 0 not_a_check 11 excluded from the denominator

Coverage (judged)

node kindjudged
ci_step13
script12

Nodes by kind, over the 25 node(s) carried in this report. Surfaces the gatherer cannot read are absent here rather than counted as unknown, which makes non-coverage Keel's own shoppable class — so it is stated, not implied.

ε-audit — not run

No verdict in this report carries a readable audit block, so nothing was re-decided and there is no agreement rate to report. This is not agreement, and it is not full agreement either: a rate over zero comparisons would state a measurement that was never taken. (“Readable” is load-bearing: a block can be present and state no comparison, and this state covers that too — see below if so.)

This run carries 0 probe-decided verdict(s), none of them contributing a readable comparison. (“Readable” rather than “audited”, because a block can arrive and state nothing — if any did, they are counted below.) Audit coverage is itself shoppable — audit none of the library and no disagreement can ever be found — so the absence is printed here rather than left to be inferred from a missing section.

This run judged 25 of 32 gathered edges. Every number on this page describes that sample, not the whole surface — 7 gathered edges are absent from all of it. A cap is disclosed rather than smoothed: an undisclosed sample is how a ratio gets shopped.

Scope. Keel measures the shape of verification, not its quality. A repo can be 100% anchored with terrible tests. Anchoring says the signal comes from outside; it does not say the signal is sufficient.

Node graph — 25 node(s), in gather order

anchored 5 self_referential 9 not_a_check 11

1 mark is ringed in the unknown hue: an anchored verdict asserted below confidence 0.6. The ring says the claim is closer to unknown than its colour suggests. Hover a mark for its node.

anchored — 5

The producer sits outside the write boundary of the actor being verified.

portability-check
anchored
Makefile:52 · script

I appended `/Users/somebody/secret/path` to README.md and the target went from exit 0 to exit 1, naming README.md:73; removing it restored exit 0. The signal is grep's match against committed bytes, so a contributor who commits a machine-specific path cannot make the scan report clean without editing scripts/portability-check.sh itself — and in the CI lane the scan runs on a fresh ubuntu-latest checkout with none of the author's tooling present. Documented limit: the oracle (which paths count, and the is_exempt list at lines 40-46 covering docs/handoffs/*, docs/decisions/* and the script itself) is authored inside the write boundary, so quietly widening an exemption shrinks what gets scanned without turning the gate red.

producer bash + grep -nE scanning the actual bytes of every file in `git ls-files`, with the script's `fail` flag driving `exit 1` — actor can write to it: no

  • Makefile:52-53 — `@bash scripts/portability-check.sh`
  • MEASURED (negative control): appended `/Users/somebody/...` to README.md -> exit 1, output `README.md:73 absolute home path: /Users/somebody` + `[portability] FAIL`; restored -> exit 0
  • scripts/portability-check.sh:69-70 — `grep -nE '/Users/[A-Za-z0-9._-]+|/home/[A-Za-z0-9._-]+'` over `git ls-files` output
  • scripts/portability-check.sh:86-94 — `if [ "$fail" -ne 0 ]; then ... exit 1`
  • .github/workflows/portability.yml:19-26 — job `paths` on ubuntu-latest, step `run: bash scripts/portability-check.sh`; the step's exit code is the job conclusion
  • LIMIT: scripts/portability-check.sh:40-46 is_exempt() — exemption list is in-repo and author-editable

decided by agent · confidence 0.88

design-audit
anchored
Makefile:55 · script

I injected `.x{color:#abcdef}` into a <style> block in site/index.html and the target flipped from exit 0 to exit 1, reporting `site/index.html:19 raw hex color`; reverting restored exit 0 over 14 governed files. The exit code comes from the interpreter reading the files on disk, so someone adding a raw literal cannot make the scan report clean without editing scripts/design-audit.py. Documented limits: the rules are authored in-repo, checks 1-2 carry an `audit-ok:` per-line escape hatch (line 133), check 3's expected bytes can be regenerated wholesale by `make design-sync` — the --update-snapshots shape, where the oracle is manufactured by the thing under test — and no workflow invokes this target, so it blocks only a human ticking the PR-template box.

producer python3 executing design-audit.py — regex scans and byte comparisons over the real governed files, returning 1 from main() when the failures list is non-empty — actor can write to it: no

  • Makefile:55-56 — `@python3 scripts/design-audit.py`
  • MEASURED (negative control): injected raw hex into site/index.html -> exit 1, `design-audit: 1 violation(s)` / `site/index.html:19 raw hex color`; reverted -> exit 0, `design-audit: 14 governed file(s) clean.`
  • scripts/design-audit.py:388-399 — `if failures: ... return 1` else `return 0`; `raise SystemExit(main())` at 403
  • scripts/design-audit.py:131-153 (check_literals), 269-307 (check_verdict_classes parses the frozen GroundingClass union out of skills/keel/schemas/keel.ts), 355-363 (check_emoji)
  • LIMIT — escape hatch: scripts/design-audit.py:133-134 `if "audit-ok:" in line: continue`
  • LIMIT — regenerable oracle: scripts/design-audit.py:156-179 check_sync compares site/ copies to skills/keel/design/, and Makefile:58-59 `design-sync` rewrites exactly those copies
  • LIMIT — not in the pipeline path: grep of .github/workflows/ finds no design-audit invocation; only .github/PULL_REQUEST_TEMPLATE.md:18 references it as a checkbox

decided by agent · confidence 0.78

design-sync
anchored
Makefile:58 · script

The residual falsifiable path is real: an off-palette hex in skills/keel/design/mark.svg, or a missing canonical mark, makes check_brand_marks append to failures and `--fix-sync` returns 1 (line 377), and that comparison is python reading token values out of tokens.css rather than any value this target writes. Heavy documented limit, and the reason confidence is only 0.5: everything else the target does is mutation — it overwrites the site/ copies and derives the -on-light variants, which is precisely the expected output that design-audit checks 3 and 8 later compare against, so running it manufactures the agreement rather than testing it. Only the ink-validation half can go red; the sync half passes by construction. Not filed not_a_check because 'a formatter that only rewrites' is factually false here — it does execute one falsifiable assertion.

producer python3 running design-audit.py --fix-sync; in fix mode check_sync only writes, but check_brand_marks still validates every hex in the canonical SVGs against --k-ink-0/--k-accent read out of tokens.css, and main() returns 1 if that appends a failure — actor can write to it: no

  • Makefile:58-59 — `@python3 scripts/design-audit.py --fix-sync`
  • scripts/design-audit.py:374-377 — `if "--fix-sync" in sys.argv: check_brand_marks(fix=True); check_sync(fix=True); return 0 if not failures else 1`
  • scripts/design-audit.py:246-251 — the hex-vs-token loop runs in fix mode too and appends a failure for any ink that is not --k-ink-0 / --k-accent
  • scripts/design-audit.py:167-172 — in fix mode check_sync calls `dst.write_bytes(want)` and never appends a failure
  • scripts/design-audit.py:371-373 — the author's own note: '--fix-sync is a fixer, not a gate'
  • SMELL: low confidence recorded deliberately — the dominant effect is writing the files that Makefile#design-audit later compares

decided by agent · confidence 0.5 · confidence below 0.6 on an anchored verdict

actions/deploy-pages@v4
anchored
.github/workflows/pages.yml:32 · ci_step

The pass/fail comes from a third-party system actually ingesting the artifact and publishing it: repository contributors have no write path to the Pages backend's deployment status, so a rejected or failed publish cannot be edited into a green from inside the repo. Documented limit — this anchors only the fact of publication (GitHub accepted and served the bundle); nothing here fetches the deployed URL or asserts anything about the page's content, so it is not evidence the site renders or is correct. Second, smaller limit: I did not read the marketplace action's source, so the polling behavior is taken from its documented contract.

producer GitHub's Pages service — the action creates a Pages deployment from the uploaded artifact and polls GitHub's deployment-status API until it reports success or error; the step's exit code is that reported status — actor can write to it: no

  • .github/workflows/pages.yml:32-33 — `- id: deployment / uses: actions/deploy-pages@v4`
  • .github/workflows/pages.yml:22-24 — `environment: github-pages` with `url: ${{ steps.deployment.outputs.page_url }}`, i.e. the URL is produced by the service, not authored in-repo
  • .github/workflows/pages.yml:11-14 — `id-token: write` for the OIDC exchange the Pages service verifies
  • No post-deploy fetch/smoke step exists in the workflow

decided by agent · confidence 0.7

Scan committed files for absolute home paths
anchored
.github/workflows/portability.yml:25 · ci_step

The script greps every tracked file for /Users/<name> or /home/<name> (portability-check.sh:69-70) and for ~/broomva or $HOME/broomva in operational files (:76), sets fail=1 and exits 1 (:93); that non-zero fails the step and the job. A contributor who commits a machine-specific path cannot make the scan report clean by asserting otherwise — the verdict is a deterministic read of the committed bytes. Documented limit: the exemption list (portability-check.sh:40-46 exempts itself plus docs/handoffs/* and docs/decisions/*) and the `grep -vE '/home/runner'` filter live in the same repo, so the SCOPE oracle is inside the write boundary — moving a file under docs/handoffs/ suppresses the signal without touching the scanner. Execution axis is anchored; the scope is co-authored.

producer exit status of bash+grep in scripts/portability-check.sh, executed over the bytes listed by `git ls-files` on an ubuntu-latest runner — actor can write to it: no

  • .github/workflows/portability.yml:25-26 — `run: bash scripts/portability-check.sh`
  • scripts/portability-check.sh:31 `mapfile -t FILES < <(git ls-files)`; :69-70 grep -nE '/Users/...|/home/...'; :86-93 `if [ "$fail" -ne 0 ] ... exit 1`
  • scripts/portability-check.sh:40-46 is_exempt() — self, docs/handoffs/*, docs/decisions/*
  • gh run list --workflow=portability.yml — runs 30135018165 / 30133685962 executed on pull_request, so this edge is in the pipeline path

decided by agent · confidence 0.85

self_referential — 9

The producer sits inside the write boundary of the actor being verified.

bstack-check
self_referential
Makefile:48 · script

Of the 100 [ok] results measured here, the bulk are of the form "P1 row present", "AGENTS.md has '### Bridge' section", "primitive count header reads Twenty" — the way to make the gate green is to type the row into the markdown the same authors own, so the claim and its check are the same write. The scanner is not a fixed third party either: BSTACK is resolved at run time by scanning $HOME/.claude/skills/bstack and two sibling roots (Makefile:16-22) with no lockfile, checksum or vendored copy, and bstack is the same author's repo, so an agent in this workspace can edit the producer as easily as the file being scanned. It also never runs in CI — portability.yml:79 deliberately invokes `make BSTACK= bstack-l3-trust` instead. Counter-argument recorded: on a pure execution axis the exit code is honestly produced and falsifiable (2 gaps -> exit 1 as measured), which is why this is 0.6 and not higher.

producer bash executing `doctor.sh --strict` from an unpinned local bstack snapshot; its exit code is a count of grep misses over this repo's own CLAUDE.md, AGENTS.md and .control/policy.yaml — actor can write to it: yes

  • Makefile:48-50 — `@bash "$(BSTACK)/scripts/doctor.sh" --strict`
  • Makefile:16-22 — BSTACK discovered at run time across $HOME/.claude/skills/bstack, $HOME/.agents/skills/bstack, $HOME/.local/share/bstack, $(CURDIR)/.bstack; no version pin or integrity check; all three roots present on this machine
  • MEASURED: `doctor.sh --strict` -> exit 1 with 2 gaps, 100 [ok]
  • MEASURED [ok] sample: 'CLAUDE.md', 'AGENTS.md', 'primitive count header reads Twenty/20', 'P1 row present' ... 'P8 row present' — prose-presence assertions
  • doctor.sh:135,148,156,197,272,477,488 — gap() calls keyed on missing files, missing table rows, missing markdown sections, missing YAML blocks
  • .github/workflows/portability.yml:79 — the only bstack-adjacent CI step runs with BSTACK deliberately emptied
  • grounding-classes.md:28 — 'A status field the actor sets, gating on that field | The claim and the check are the same write.'

decided by agent · confidence 0.6

bstack-primitive-lint
self_referential
Makefile:67 · script

The script opens only AGENTS.md and CLAUDE.md and compares them to each other: each `### P-N` section must contain the literal strings `**What**/**How**/**Why**/**Invariant**`, each must have a `| P-N |` row in CLAUDE.md's table, and the spelled-out count word in CLAUDE.md's header must equal the number of sections. Both operands AND the rule list itself (REQUIRED_SECTIONS, HOOK_ENFORCED, MECHANISM_ONLY at lines 33-40) are edited by the same governance PR this gate is policing, so the way to green is to type 'Twenty' or paste an empty `**What**` heading — no executed behaviour and no outside party is consulted. Execution is honest and the signal genuinely varies (I mutated a scratch copy's header word and it printed FAIL and returned 1), but what varies is agreement between two pieces of paperwork the author owns.

producer python3 executing scripts/bstack-primitive-lint.py; the exit code is the interpreter's, but every byte it reads is two in-repo markdown files — actor can write to it: yes

  • Makefile:67-68 — `bstack-primitive-lint: @python3 scripts/bstack-primitive-lint.py`
  • scripts/bstack-primitive-lint.py:30-34 — AGENTS_MD / CLAUDE_MD are the only inputs; REQUIRED_SECTIONS is a literal tuple of markdown bold strings
  • scripts/bstack-primitive-lint.py:131-152 — table-row cross-check and 'header says N / AGENTS.md has M' count check
  • scripts/bstack-primitive-lint.py:167-172 — return 1 on any error, return 0 otherwise (falsifiable, not vacuous)
  • Empirical (repo HEAD): exit 0 — 'Primitives found 20 / table 20 / header 20', 4 warnings
  • Empirical falsifiability: copied AGENTS.md+CLAUDE.md+script to a scratch tree, changed 'Twenty irreducible'→'Nineteen irreducible' → '[ERROR] CLAUDE.md header says 19 ... but AGENTS.md has 20' → FAIL
  • Reaches CI transitively via .github/workflows/portability.yml `gates` job (`make BSTACK= bstack-l3-trust`)
  • skills/keel/references/grounding-classes.md:26 — 'A doc validated against another doc | Paperwork against paperwork'

decided by agent · confidence 0.7

bstack-rule-of-three
self_referential
Makefile:70 · script

Its intended oracle is AGENTS.md's `### P-N:` titles matched against a ledger table in research/entities/pattern/bstack-engine.md, and 'this pattern recurred 3+ times' is established by regexing a string like '30+ instances' out of a markdown cell — the promoter authors both the promotion and the evidence for it, so the claim and its check are the same write. Today the ledger path does not exist at all, so main() returns 1 before parsing anything and no target or workflow consumes the result (bstack-l3-trust deliberately drops it, Makefile:73-93). I weighed not_a_check on the grounds that a permanently red signal carries no information, but the fork point here is fully traceable and sits inside the write boundary, and the node would still be paperwork-vs-paperwork the day the ledger lands — dropping it out of the denominator would raise Keel's own ratio on a technicality, which is the shoppable move, not the honest one.

producer python3 executing scripts/bstack-rule-of-three.py — but in this repo the exit code comes from a missing-file precondition guard (line 149), never from the audit itself — actor can write to it: yes

  • Makefile:70-71 — `bstack-rule-of-three: @python3 scripts/bstack-rule-of-three.py`
  • scripts/bstack-rule-of-three.py:29-30 — inputs are AGENTS.md and research/entities/pattern/bstack-engine.md, both in-repo markdown
  • scripts/bstack-rule-of-three.py:102-118 — has_three_or_more_instances() regexes 'N+' / 'N instances' / 'meets rule-of-three' out of a table cell the promoter typed
  • scripts/bstack-rule-of-three.py:148-150 — `if not BSTACK_ENGINE.exists(): return 1`
  • Empirical: `python3 scripts/bstack-rule-of-three.py` → '[FAIL] <repo>/research/entities/pattern/bstack-engine.md not found', exit=1
  • `ls research/entities/pattern/` → No such file or directory (the whole research/ tree is absent)
  • Makefile:73-93 — documented scope-out; the target is not a prerequisite of bstack-l3-trust and appears in no workflow (.github/workflows/ = l3-stability.yml, pages.yml, portability.yml)

decided by agent · confidence 0.6

bstack-l3-trust
self_referential
Makefile:94 · script

This is an aggregate, so the rule is to descend rather than classify the wrapper: it has exactly one child, bstack-primitive-lint, whose signal is AGENTS.md compared against CLAUDE.md — the same two files a governance PR edits, i.e. inside the write boundary of the change the gate is supposed to police. The three echoes then print unconditionally once the prerequisite succeeds, including 'L3 trust gates passed — governance change is structurally valid' and a line declaring G-L3-2 not-run, so the pack narrates broader coverage than its single child established. GitHub's runner executing `make BSTACK= bstack-l3-trust` in portability.yml anchors that the command really ran on a clean clone; it does not move the fork point of what that command compared.

producer make's prerequisite chain — the target's exit status is bstack-primitive-lint's python3 exit code; the target's own recipe is three unconditional `@echo` lines — actor can write to it: yes

  • Makefile:94-97 — prerequisite `bstack-primitive-lint`, recipe is three `@echo` lines only
  • Make semantics: a non-zero prerequisite aborts the target, so the exit code is entirely the child's
  • Child verdict: Makefile#bstack-primitive-lint = self_referential (AGENTS.md vs CLAUDE.md)
  • .github/workflows/portability.yml — `gates` job step 'L3 trust gate: run: make BSTACK= bstack-l3-trust' (job conclusion depends on it)
  • Empirical: `make BSTACK= bstack-l3-trust` → child OK, then the three echoes, exit 0
  • skills/keel/references/grounding-classes.md:41-43 — 'Do not classify the wrapper; descend'

decided by agent · confidence 0.68

test
self_referential
package.json · script

`bun test` compiles and runs tests/grounding-ratio.test.ts, which imports the real groundingRatio() from skills/keel/schemas/keel.ts and compares its COMPUTED output (0.5 with and without not_a_check entries, notACheck=2, empty input → 0) rather than echoing a constant; the runtime decides pass/fail and no prose in the repo talks a failed assertion into a zero exit. Two limits belong on the record and neither is the execution axis: (1) the oracle — that not_a_check leaves the denominator and unknown fails closed — was authored in this same repo alongside the implementation, so a green run proves the code does what the test says, not that the test says the right thing; (2) nothing runs this at merge time — .github/workflows/ holds only l3-stability.yml, pages.yml and portability.yml, and test.yml is still only a plan — so the merge-time surrogate is a hand-ticked `- [ ] bun test` box in the PR template, which is a status field the author sets.

producer the Bun test-runner process — a thrown expect() failure inside bun:test sets the process exit code — actor can write to it: yes

  • package.json — "test": "bun test"
  • tests/grounding-ratio.test.ts:15 — imports groundingRatio from ../skills/keel/schemas/keel.ts (real implementation, not a fixture)
  • tests/grounding-ratio.test.ts:30-41 — asserts with_.ratio === without.ratio and === 0.5, both computed by the function under test
  • Empirical: `bun test` → '3 pass, 0 fail, 7 expect() calls', exit=0
  • `ls .github/workflows/` → l3-stability.yml, pages.yml, portability.yml — no lane executes bun test
  • docs/plans/w1-e-tests-ci.md:36 — .github/workflows/test.yml is planned, not present
  • .github/PULL_REQUEST_TEMPLATE.md:17 — '- [ ] `bun test`' self-attested checkbox
  • skills/keel/references/grounding-classes.md:68-81 — the execution-vs-oracle limit

decided by agent · confidence 0.68

Compute lambda (RCS stability budget)
self_referential
.github/workflows/l3-stability.yml:52 · ci_step

Every input to the inequality is a hand-declared constant in `.control/rcs-parameters.toml`, a file inside the PR's own tree — and that file is itself one of the paths that triggers this workflow, so an author whose lambda goes non-positive can restore a green by editing the very numbers the gate reads. The arithmetic executes honestly, but the parameters have no external referent (they are asserted, not measured), so the claim and its check are the same write. Secondary limit: the script is fetched at run time from an unpinned clone, so I traced it through a local bstack snapshot (~/.claude/skills/bstack/scripts/compute-lambda.sh), not the exact revision CI will execute.

producer python3 inside compute-lambda.sh evaluating lambda_i = gamma_i - L_theta_i*rho_i - L_d_i*eta_i - beta_i*tau_bar_i - ln(nu_i)/tau_a_i over the numbers in the repo's own .control/rcs-parameters.toml — actor can write to it: yes

  • .github/workflows/l3-stability.yml:52-66 — `out=$(bash $BSTACK/scripts/compute-lambda.sh --human)`, status captured to $GITHUB_OUTPUT, step forced to `exit 0`
  • ~/.claude/skills/bstack/scripts/compute-lambda.sh — config lookup order puts `$WORKSPACE/.control/rcs-parameters.toml` first; exit 1 iff any lambda_i <= 0
  • <repo>/.control/rcs-parameters.toml:47-64 — literal gamma/L_theta/rho/L_d/eta/beta/tau_bar/nu/tau_a per level, hand-set
  • .github/workflows/l3-stability.yml:16-21 — workflow triggers on changes to `.control/rcs-parameters.toml` itself

decided by agent · confidence 0.85

Run L3 rate gate
self_referential
.github/workflows/l3-stability.yml:68 · ci_step

The counted channel is the repository's own commit history — the actor's writes — and the threshold (tau_a for level L3, plus the [gates.l3_paths] pattern list) is read from an in-repo TOML the same PR is allowed to edit, so widening the window or dropping a path from the pattern list turns EXCEEDED into OK without changing any governance behavior. Additional limit: the step runs under `set +e` and ends in `exit 0`, and `Set final status` reads only `steps.compute.outputs.status`, so `steps.rate.outputs.status` reaches nothing but the PR comment text — this edge cannot fail the job at all.

producer l3-rate-gate.sh counting `git log --diff-filter=M --since=@CUTOFF -- <L3 paths>` in the checked-out repo, against a window tau_a read from .control/rcs-parameters.toml — actor can write to it: yes

  • .github/workflows/l3-stability.yml:68-81 — `set +e`, `bash $BSTACK/scripts/l3-rate-gate.sh`, terminal `exit 0`
  • .github/workflows/l3-stability.yml:125-133 — final status branches only on `steps.compute.outputs.status`; comment at 128 says 'L3 rate exceeded is informational'
  • ~/.claude/skills/bstack/scripts/l3-rate-gate.sh — reads `gates.l3_paths.patterns` and L3 `tau_a` from `$WORKSPACE/.control/rcs-parameters.toml`, counts via `git log --diff-filter=M`
  • <repo>/.control/rcs-parameters.toml — the parameter file lives in the same tree as the change under test

decided by agent · confidence 0.82

Comment on PR
self_referential
.github/workflows/l3-stability.yml:83 · ci_step

The step renders an authoritative-looking merge recommendation ('composite system unstable; do not merge' / 'All gates pass') that a human reviewer may act on, but both the claim and its evidence are authored by this one workflow run out of numbers taken from the PR's own .control/rcs-parameters.toml — the comment adds a publication surface, not an independent observer. Its own success only means the GitHub API accepted a comment body, which is orthogonal to whether the body is true.

producer actions/github-script@v7 calling issues.createComment with a verdict string this same job computed from steps.compute/steps.rate outputs — actor can write to it: yes

  • .github/workflows/l3-stability.yml:83-123 — verdict built from `steps.compute.outputs.status` / `steps.rate.outputs.status`, posted via `github.rest.issues.createComment`
  • .github/workflows/l3-stability.yml:27-29 — job holds `pull-requests: write`, i.e. the pipeline authors the artifact a reviewer reads
  • No step or gate in the repo reads this comment back

decided by agent · confidence 0.72

Set final status
self_referential
.github/workflows/l3-stability.yml:125 · ci_step

This is the only edge in the job that can turn the check red, so it is the gate — but it is a wrapper, and descending it lands on the same actor-authored constants: exit 1 fires iff lambda_i <= 0 for numbers the PR author typed into a file this workflow itself watches. Nothing about the change under review is measured. A further limit on its force: `gh api repos/{owner}/{repo}/branches/main/protection` returns 404, so main has no branch protection and this red X blocks no merge today.

producer a shell string comparison on steps.compute.outputs.status, which is the exit code compute-lambda.sh returned over the repo's own .control/rcs-parameters.toml — actor can write to it: yes

  • .github/workflows/l3-stability.yml:125-133 — `if [ "${{ steps.compute.outputs.status }}" != "0" ]; then ... exit 1`
  • .github/workflows/l3-stability.yml:10-11 — header notes the check 'can be made required via branch protection rules'
  • `gh api repos/{owner}/{repo}/branches/main/protection` → 404 Not Found (no protection configured)
  • <repo>/.control/rcs-parameters.toml — source of every term in the inequality

decided by agent · confidence 0.85

not_a_check — 11

Asserts nothing about correctness, so it is excluded from the denominator. This is the one shoppable class: mis-filing a real check here shrinks the denominator and inflates the score, so each row carries the same burden of argument as any other verdict.

help
not_a_check
Makefile:40 · script

The target's entire output is the Makefile's own `## ` comments re-printed by awk, so the only thing the pipeline can report is text a contributor typed two lines above. Nothing about the repository's behaviour is compared against anything, and the exit status is awk's formatting status, so no state of the world can flip it.

producer grep -hE over $(MAKEFILE_LIST) piped to awk, which reformats the `## ` doc-comments into a listing — actor can write to it: yes

  • Makefile:40-42 — `@grep -hE '^[a-zA-Z_-]+:.*?## ' $(MAKEFILE_LIST) | awk ... printf`
  • No comparison, no threshold, no assertion in the recipe body
  • Not referenced by any workflow in .github/workflows/

decided by agent · confidence 0.95

doctor
not_a_check
Makefile:44 · script

Measured directly on this machine: doctor.sh printed two [gap] lines and still exited 0, because the `exit 1` at doctor.sh:1441 is guarded by --strict and this target does not pass it. The exit status is therefore constant regardless of what the scan finds, so the edge cannot carry information — the gaps land in stdout that nothing reads. The Makefile's own help text concedes this ("always exits 0"). Filed here on falsifiability, not on difficulty; the wrapped content is separately reachable via bstack-check.

producer bash executing an external bstack doctor.sh, which prints [ok]/[gap] lines and then reaches an unconditional `exit 0` at line 1445 — actor can write to it: yes

  • Makefile:44-46 — `doctor: ## ... always exits 0` then `@bash "$(BSTACK)/scripts/doctor.sh"`
  • MEASURED: `bash ~/.claude/skills/bstack/scripts/doctor.sh` -> exit 0 with 2 `[gap]` lines present in output
  • ~/.claude/skills/bstack/scripts/doctor.sh:1439-1445 — `# Strict mode: exit non-zero if any gap (CI usage)` ... `exit 1` inside the STRICT guard; bare `exit 0` at 1445
  • REQUIRE_BSTACK (Makefile:24-34) can exit 1, but only on the tool being absent — an environment fact, not a claim about this repo

decided by agent · confidence 0.85

janitor
not_a_check
Makefile:61 · script

The target answers a housekeeping question — which local branches have already landed in origin/main and which worktrees are orphaned — and prints the list. Whether it finds twenty stale branches or none, control reaches `exit 0` at branch-janitor.sh:214, so no finding can turn the target red. Nothing here asserts that the repository's code is correct; the git state it reads is also state this repo pushes, so even the input is author-written.

producer git plumbing (merge-base --is-ancestor, commit-tree + cherry) over origin/main, printing `[dry-run drop]` lines; the script then reaches an unconditional `exit 0` — actor can write to it: yes

  • Makefile:61-62 — `@bash scripts/branch-janitor.sh` (no --apply, so APPLY=0)
  • scripts/branch-janitor.sh:168-170 — dry-run branch prints `[dry-run drop]` and continues
  • scripts/branch-janitor.sh:206-214 — summary block then unconditional `exit 0`
  • MEASURED shell semantics: `set -euo pipefail; APPLY=1; [ "$APPLY" = "0" ] && echo ...; exit 0` -> exit 0 (the trailing test does not trip set -e)
  • scripts/branch-janitor.sh:9-15 — documented as cleanup tooling, `--dry-run by default (safe)`

decided by agent · confidence 0.85

janitor-apply
not_a_check
Makefile:64 · script

This target performs deletions rather than reporting a property: it removes squash-merged branches and prunes dead worktrees, then falls through to `exit 0` at line 214 whether it dropped ten branches or zero. A delete failure is swallowed into a `[skip] ... (delete failed)` message (line 166) rather than a non-zero status, so there is no proposition the exit code could falsify. Mutation, not assertion.

producer git branch -D and git worktree remove --force executed by branch-janitor.sh --apply; the script exits 0 unconditionally — actor can write to it: yes

  • Makefile:64-65 — `@bash scripts/branch-janitor.sh --apply`
  • scripts/branch-janitor.sh:161-167 — `git branch -D` on success prints `[drop]`; on failure prints `[skip] ... (delete failed)` and continues
  • scripts/branch-janitor.sh:189-193 — `git worktree remove --force` inside the APPLY branch
  • scripts/branch-janitor.sh:203 — `git worktree prune 2>/dev/null || true`
  • scripts/branch-janitor.sh:214 — `exit 0`

decided by agent · confidence 0.85

Checkout PR head
not_a_check
.github/workflows/l3-stability.yml:31 · ci_step

The step materializes the working tree (fetch-depth: 50) and writes no pass/fail claim about the code — nothing later in the job branches on its result other than by not existing. Its only failure mode is transport/credential failure, which reports on the runner's network, not on whether the change under review is correct.

producer actions/checkout@v4 fetching the PR head tree from GitHub's git server — actor can write to it: not established

  • .github/workflows/l3-stability.yml:31-34 — `uses: actions/checkout@v4` with `fetch-depth: 50`
  • No step in the job reads an output of this step; `Set final status` (l3-stability.yml:125-133) reads only `steps.compute.outputs.status`

decided by agent · confidence 0.86

Set up Python 3.11
not_a_check
.github/workflows/l3-stability.yml:36 · ci_step

This is environment provisioning: it puts a python3 with tomllib on PATH so that compute-lambda.sh and l3-rate-gate.sh can parse TOML (both exit 4 / degrade without Python >= 3.11). It emits no verdict about the repository; the assertions live downstream in the steps that run those scripts.

producer actions/setup-python@v5 installing a CPython 3.11 toolchain onto the runner — actor can write to it: not established

  • .github/workflows/l3-stability.yml:36-39 — `uses: actions/setup-python@v5`, python-version 3.11
  • ~/.claude/skills/bstack/scripts/compute-lambda.sh — 'exit 4 — python3 or tomllib unavailable (Python >= 3.11 required)'

decided by agent · confidence 0.88

Install bstack (skill snapshot)
not_a_check
.github/workflows/l3-stability.yml:41 · ci_step

The step only resolves and installs the scripts the next two steps execute; it asserts nothing about this repo's correctness. Worth recording as a supply-chain limit rather than a class: this repo has no `.agents/skills/bstack`, so the branch is always the clone, and it floats on bstack's default branch with no SHA pin — the checker code executed at gate time is not fixed by anything in this repo, and it is owned by the same author as this repo.

producer git clone of https://github.com/broomva/bstack.git (unpinned, --depth 1 of default branch), exporting BSTACK into $GITHUB_ENV — actor can write to it: not established

  • .github/workflows/l3-stability.yml:41-50 — conditional on `.agents/skills/bstack`, else `git clone --depth 1 https://github.com/broomva/bstack.git /tmp/bstack`
  • `ls -d <repo>/.agents/skills/bstack` → No such file or directory (the clone branch is the live path)
  • No commit SHA or tag pin on the clone

decided by agent · confidence 0.8

actions/checkout@v4
not_a_check
.github/workflows/pages.yml:27 · ci_step

It exists so `site/` is on disk for the upload step; it evaluates nothing and emits no pass/fail claim about the site or the repo. Failure here means the runner could not reach GitHub, which is a statement about the runner.

producer actions/checkout@v4 fetching the main-branch tree onto the runner — actor can write to it: not established

  • .github/workflows/pages.yml:27 — `- uses: actions/checkout@v4`
  • .github/workflows/pages.yml:29-31 — the artifact upload consumes the checked-out `site` directory

decided by agent · confidence 0.88

actions/configure-pages@v5
not_a_check
.github/workflows/pages.yml:28 · ci_step

It configures the publication target and hands base path variables to the build; it runs no assertion over the site contents and gates nothing — no later step branches on its outputs in this workflow. It is setup on the deploy path, not a verification edge.

producer actions/configure-pages@v5 reading/enabling the repository's Pages settings via the GitHub API and exporting base_url — actor can write to it: not established

  • .github/workflows/pages.yml:28 — `- uses: actions/configure-pages@v5`, no `with:` and no `id:`
  • .github/workflows/pages.yml:11-14 — job permissions `pages: write`, `id-token: write` (publication plumbing)

decided by agent · confidence 0.75

actions/upload-pages-artifact@v3
not_a_check
.github/workflows/pages.yml:29 · ci_step

Packaging, not evaluation: it asserts nothing about whether the pages are correct, well-formed, or non-broken — there is no HTML validation, link check, or build step anywhere in this workflow. It can fail if `site/` is missing, but that is an incidental precondition of the tar, not a correctness claim the repo is being held to.

producer actions/upload-pages-artifact@v3 tarring the `site` directory and uploading it to the Actions artifact store — actor can write to it: not established

  • .github/workflows/pages.yml:29-31 — `uses: actions/upload-pages-artifact@v3` with `path: site`
  • .github/workflows/pages.yml:20-33 — the deploy job has exactly four steps; none builds, lints, or validates the site

decided by agent · confidence 0.72

actions/checkout@v4
not_a_check
.github/workflows/portability.yml:23 · ci_step

The step only places the repo's bytes on the runner so the next step (bash scripts/portability-check.sh) has something to scan; it evaluates nothing and emits no pass/fail claim about the code. Its failure mode is provisioning (network/auth), not 'the repo is wrong'. I am not using unknown for the action's opacity: opacity would matter if it made a correctness assertion I could not descend into, and it makes none.

producer actions/checkout@v4 on a GitHub-hosted runner — a git fetch into the workspace — actor can write to it: not established

  • .github/workflows/portability.yml:23 — `- uses: actions/checkout@v4` in job `paths`
  • .github/workflows/portability.yml:25-26 — the job's only assertion is the following run step
  • no `with:` block; no output consumed by any later step or `if:` condition

decided by agent · confidence 0.85

Crystallization curve

Keel crystallization curve Crystallization curve over 15 runs from <repo>/reports. falls — total fitted change -66.5% of the mean across 15 runs (R^2 0.28, so the line explains less than half the variance — read the raw squares) CRYSTALLIZATION CURVE 15 sequential runs · 345 nodes judged of 1838 gathered · 124 anchored Keel corpus — 15 repositories, 2026-07-24 — measured corpus (declared on the command line) estimated tokens per node estimated tokens / judged node 0 300 600 run 0 · keel · estimated tokens per node = 498 · judged 25 of 32 gathered run 1 · anthropic-sdk-python · estimated tokens per node = 369 · judged 25 of 41 gathered run 2 · openai-python · estimated tokens per node = 398 · judged 25 of 95 gathered run 3 · vercel-ai · estimated tokens per node = 410 · judged 25 of 1014 gathered run 4 · aider · estimated tokens per node = 204 · judged 25 of 62 gathered run 5 · browser-use · estimated tokens per node = 378 · judged 25 of 104 gathered run 6 · mcp-python-sdk · estimated tokens per node = 592 · judged 25 of 121 gathered run 7 · simonw-llm · estimated tokens per node = 189 · judged 25 of 38 gathered run 8 · tiktoken · estimated tokens per node = 240 run 9 · requests · estimated tokens per node = 290 · judged 25 of 106 gathered run 10 · flask · estimated tokens per node = 408 · judged 25 of 59 gathered run 11 · sinatra · estimated tokens per node = 265 · judged 25 of 59 gathered run 12 · commander-js · estimated tokens per node = 316 run 13 · anthropic-quickstarts · estimated tokens per node = 310 · judged 25 of 62 gathered run 14 · tiktoken · estimated tokens per node = 53 0* 1* 2* 3* 4* 5* 6* 7* 8 9* 10* 11* 12 13* 14 seconds per node measured s / judged node 0.0 60.0 120.0 run 0 · keel · seconds per node = 0.1 · judged 25 of 32 gathered run 1 · anthropic-sdk-python · seconds per node = 15.8 · judged 25 of 41 gathered run 2 · openai-python · seconds per node = 15.4 · judged 25 of 95 gathered run 3 · vercel-ai · seconds per node = 105.9 · judged 25 of 1014 gathered run 4 · aider · seconds per node = 21.3 · judged 25 of 62 gathered run 5 · browser-use · seconds per node = 21.7 · judged 25 of 104 gathered run 6 · mcp-python-sdk · seconds per node = 20.1 · judged 25 of 121 gathered run 7 · simonw-llm · seconds per node = 11.8 · judged 25 of 38 gathered run 8 · tiktoken · seconds per node = 23.7 run 9 · requests · seconds per node = 16.8 · judged 25 of 106 gathered run 10 · flask · seconds per node = 20.5 · judged 25 of 59 gathered run 11 · sinatra · seconds per node = 16.9 · judged 25 of 59 gathered run 12 · commander-js · seconds per node = 18.7 run 13 · anthropic-quickstarts · seconds per node = 20.4 · judged 25 of 62 gathered run 14 · tiktoken · seconds per node = 0.1 0* 1* 2* 3* 4* 5* 6* 7* 8 9* 10* 11* 12 13* 14 probe-decided share measured share of decided nodes 0.00 0.50 1.00 run 0 · keel · probe-decided share = 0.00 · judged 25 of 32 gathered run 1 · anthropic-sdk-python · probe-decided share = 0.00 · judged 25 of 41 gathered run 2 · openai-python · probe-decided share = 0.16 · judged 25 of 95 gathered run 3 · vercel-ai · probe-decided share = 0.08 · judged 25 of 1014 gathered run 4 · aider · probe-decided share = 0.56 · judged 25 of 62 gathered run 5 · browser-use · probe-decided share = 0.32 · judged 25 of 104 gathered run 6 · mcp-python-sdk · probe-decided share = 0.00 · judged 25 of 121 gathered run 7 · simonw-llm · probe-decided share = 0.56 · judged 25 of 38 gathered run 8 · tiktoken · probe-decided share = 0.58 run 9 · requests · probe-decided share = 0.28 · judged 25 of 106 gathered run 10 · flask · probe-decided share = 0.00 · judged 25 of 59 gathered run 11 · sinatra · probe-decided share = 0.28 · judged 25 of 59 gathered run 12 · commander-js · probe-decided share = 0.19 run 13 · anthropic-quickstarts · probe-decided share = 0.24 · judged 25 of 62 gathered run 14 · tiktoken · probe-decided share = 1.00 0* 1* 2* 3* 4* 5* 6* 7* 8 9* 10* 11* 12 13* 14 probe library size measured probes in library 0 15 30 run 0 · keel · probe library size = 0 · judged 25 of 32 gathered run 1 · anthropic-sdk-python · probe library size = 3 · judged 25 of 41 gathered run 2 · openai-python · probe library size = 5 · judged 25 of 95 gathered run 3 · vercel-ai · probe library size = 8 · judged 25 of 1014 gathered run 4 · aider · probe library size = 10 · judged 25 of 62 gathered run 5 · browser-use · probe library size = 12 · judged 25 of 104 gathered run 6 · mcp-python-sdk · probe library size = 13 · judged 25 of 121 gathered run 7 · simonw-llm · probe library size = 15 · judged 25 of 38 gathered run 8 · tiktoken · probe library size = 17 run 9 · requests · probe library size = 19 · judged 25 of 106 gathered run 10 · flask · probe library size = 22 · judged 25 of 59 gathered run 11 · sinatra · probe library size = 23 · judged 25 of 59 gathered run 12 · commander-js · probe library size = 26 run 13 · anthropic-quickstarts · probe library size = 29 · judged 25 of 62 gathered run 14 · tiktoken · probe library size = 29 0* 1* 2* 3* 4* 5* 6* 7* 8 9* 10* 11* 12 13* 14 x axis: run index in the recorded corpus order (listed below). Squares are the raw per-run values; the dashed line is an ordinary-least-squares fit and is never shown without them. The fit is clipped to the panel, and withheld entirely (with the panel saying so) where a straight line would predict values the points cannot take — R^2 for every fit is in the trend notes below. * on run 0, 1, 2, 3, 4, 5, 6, 7, 9, 10, 11, 13 = judged fewer nodes than gathered; per-node values are per JUDGED node. Coverage (judged) ci_step 224 · script 64 · test_target 46 · review_gate 10 · deploy_gate 1 Trend estimated tokens per node: falls — total fitted change -66.5% of the mean across 15 runs (R^2 0.28, so the line explains less than half the variance — read the raw squares). raw first 498 -> last 53. seconds per node: falls — total fitted change -72.9% of the mean across 15 runs (R^2 0.04, so the line explains less than half the variance — read the raw squares). raw first 0.1 -> last 0.1. probe-decided share: rises — total fitted change 144.9% of the mean across 15 runs (R^2 0.21, so the line explains less than half the variance — read the raw squares). raw first 0.00 -> last 1.00. probe library size: rises — total fitted change 187.7% of the mean across 15 runs (R^2 0.99). raw first 0 -> last 29. Run order run order derived from Report.generatedAt (ascending, filename as tiebreak) — no order.json manifest present 0 keel -> 1 anthropic-sdk-python -> 2 openai-python -> 3 vercel-ai -> 4 aider -> 5 browser-use -> 6 mcp-python-sdk -> 7 simonw-llm -> 8 tiktoken -> 9 requests -> 10 flask -> 11 sinatra -> 12 commander-js -> 13 anthropic-quickstarts -> 14 tiktoken Shuffle check shuffle check INCOMPLETE — the curve direction ran, the ratio direction did not Permutation (400 draws, seed 20260724): median |delta normalized slope| 0.633, 92% of draws move it by >= 0.15. A permutation reorders ALREADY-RECORDED runs; it cannot reproduce what a genuine re-run in a different order would have cost, because the probe library would have accumulated differently. It is a sanity signal on order-dependence, never a substitute for the empirical re-run. Ratio stability under shuffle was NOT checked: no re-run supplied (--shuffled) and none declared in corpus.meta.json. Permuting recorded runs cannot move a per-target ratio, so the permutation result below says nothing about it. Supply a re-run with --shuffled <dir>. Disclosures - Skipped 3 file(s) that are not usable run reports: corpus-summary.json (no string "target"); curve.json (no string "target"); keel.bindings.json (no "nodes" array). - Run 0 (keel) judged 25 of 32 gathered nodes — a cap. Per-node cost below is per JUDGED node. - Run 1 (anthropic-sdk-python) judged 25 of 41 gathered nodes — a cap. Per-node cost below is per JUDGED node. - Run 2 (openai-python) judged 25 of 95 gathered nodes — a cap. Per-node cost below is per JUDGED node. - Run 3 (vercel-ai) judged 25 of 1014 gathered nodes — a cap. Per-node cost below is per JUDGED node. - Run 4 (aider) judged 25 of 62 gathered nodes — a cap. Per-node cost below is per JUDGED node. - Run 5 (browser-use) judged 25 of 104 gathered nodes — a cap. Per-node cost below is per JUDGED node. - Run 6 (mcp-python-sdk) judged 25 of 121 gathered nodes — a cap. Per-node cost below is per JUDGED node. - Run 7 (simonw-llm) judged 25 of 38 gathered nodes — a cap. Per-node cost below is per JUDGED node. - Run 9 (requests) judged 25 of 106 gathered nodes — a cap. Per-node cost below is per JUDGED node. - Run 10 (flask) judged 25 of 59 gathered nodes — a cap. Per-node cost below is per JUDGED node. - Run 11 (sinatra) judged 25 of 59 gathered nodes — a cap. Per-node cost below is per JUDGED node. - Run 13 (anthropic-quickstarts) judged 25 of 62 gathered nodes — a cap. Per-node cost below is per JUDGED node. - Run 14 (tiktoken): decidedByProbe + decidedByAgent = 10 but nodesSampled = 12. Probe-decided share uses the decided total (10). - Provenance "measured" was declared on the COMMAND LINE (--provenance), not in <repo>/reports/corpus.meta.json. The declaration therefore lives in the invocation and is only as trustworthy as the run sheet that records it. - Token counts are ESTIMATES in 15 of 15 run(s) (RunEconomics.tokensEstimated). No API exposes session token usage to a skill, so the token axis reads "estimated tokens" and nothing here claims a measured token count. Scope. Keel measures the shape of verification, not its quality. A repo can be 100% anchored with terrible tests. Anchoring says the signal comes from outside; it does not say the signal is sufficient.

Run economics

0 decided by probe
25 decided by agent
0% probe-decided share
25 / 32 nodes judged / gathered
2,927 estimated tokens in
9,511 estimated tokens out
1.8 s wall clock
0 probes minted
0 probe library size

Token counts are estimated, and labelled so: a skill running inside an agent session has no API for its own usage, so the figure is ceil(chars/4) over the judgment payloads and responses. Wall clock and probe-decided share are measured directly. 25 of 32 gathered edges were judged — the cap is printed here and beside the ratio, never applied silently. The probe library is empty, so every decision on this run cost a model call. That is the left-hand end of the crystallization curve, not a defect.