Keel

Grounding report.

Every verification edge in anthropic-quickstarts, classified by who produces the signal and whether the actor being verified can write to that producer.

targetanthropic-quickstarts
revision370e18d4a20ff5fd4bc1a6bf11d5105b2383977c
generated2026-07-25T03:58:05.194Z
1.00

anchored / (anchored + self_referential + unknown) = 10 / 10

anchored 10 self_referential 0 unknown 0 not_a_check 15 excluded from the denominator

Coverage (judged)

node kindjudged
ci_step10
script9
test_target5
review_gate1

Nodes by kind, over the 25 node(s) carried in this report. Surfaces the gatherer cannot read are absent here rather than counted as unknown, which makes non-coverage Keel's own shoppable class — so it is stated, not implied.

ε-audit — not run

No verdict in this report carries a readable audit block, so nothing was re-decided and there is no agreement rate to report. This is not agreement, and it is not full agreement either: a rate over zero comparisons would state a measurement that was never taken. (“Readable” is load-bearing: a block can be present and state no comparison, and this state covers that too — see below if so.)

This run carries 6 probe-decided verdict(s), none of them contributing a readable comparison. (“Readable” rather than “audited”, because a block can arrive and state nothing — if any did, they are counted below.) Audit coverage is itself shoppable — audit none of the library and no disagreement can ever be found — so the absence is printed here rather than left to be inferred from a missing section.

This run judged 25 of 62 gathered edges. Every number on this page describes that sample, not the whole surface — 37 gathered edges are absent from all of it. A cap is disclosed rather than smoothed: an undisclosed sample is how a ratio gets shopped.

Scope. Keel measures the shape of verification, not its quality. A repo can be 100% anchored with terrible tests. Anchoring says the signal comes from outside; it does not say the signal is sufficient.

Node graph — 25 node(s), in gather order

anchored 10 not_a_check 15

3 marks are ringed in the unknown hue: an anchored verdict asserted below confidence 0.6. The ring says the claim is closer to unknown than its colour suggests. Hover a mark for its node.

anchored — 10

The producer sits outside the write boundary of the actor being verified.

build
anchored
customer-support-agent/package.json · script

The exit code comes from the TypeScript compiler and ESLint executing against the committed sources; an author cannot make a type-incorrect file report clean without editing the file or editing the committed tsconfig.json / .eslintrc.json (both of which are diffable). This edge is on a real gating path: amplify.yml's build phase runs `npm run build`, and a non-zero exit fails the AWS Amplify deploy. Documented limit — it is a compile-and-lint gate only; no test executes here, and `next build` says nothing about runtime behaviour.

producer the `next build` process — Next 14.2.5 runs `tsc` over tsconfig.json (`strict: true`, no `typescript.ignoreBuildErrors` escape in next.config.mjs) and ESLint over .eslintrc.json before emitting, and exits non-zero on the first type or lint error — actor can write to it: no

  • customer-support-agent/package.json:10
  • customer-support-agent/tsconfig.json:6 ("strict": true), :7 ("noEmit": true)
  • customer-support-agent/next.config.mjs:1-15 (no typescript.ignoreBuildErrors / eslint.ignoreDuringBuilds)
  • customer-support-agent/amplify.yml:9
  • customer-support-agent/package.json:34 (next 14.2.5)

decided by agent · confidence 0.8

lint
anchored
customer-support-agent/package.json · script

ESLint parses the committed .ts/.tsx files and exits non-zero on any error-level rule violation; the author cannot make a violating file report clean without changing the file or the committed .eslintrc.json. Documented limits, and they are why confidence is low: (a) `next lint` is verify-mode, not `--fix`, so it does not rewrite; (b) nothing in the repository invokes this script — no workflow covers customer-support-agent (build.yaml and tests.yaml both filter `paths:` to `.github/**` and `computer-use-demo/**`) and amplify.yml runs only `npm run build`; the same ESLint pass does, however, run inside `next build`, which IS on the Amplify deploy path.

producer the ESLint process invoked by `next lint`, resolving the committed .eslintrc.json (`extends: next/core-web-vitals`) — actor can write to it: no

  • customer-support-agent/package.json:12
  • customer-support-agent/.eslintrc.json:2
  • .github/workflows/tests.yaml:4-12 (path filter excludes customer-support-agent)
  • .github/workflows/build.yaml:5-14 (same path filter)

decided by agent · confidence 0.6

build:full
anchored
customer-support-agent/package.json · script

Descending the alias: this re-enters package.json:10, so the signal has exactly the causal path of `build` — the TypeScript compiler and ESLint decide the exit code from the committed files under a committed tsconfig.json with `strict: true`, and Amplify's build phase fails the deploy on non-zero. Same documented limit: compile-and-lint only, no behavioural assertion.

producer `npm run build` → the `next build` process, i.e. tsc + ESLint over the committed sources — actor can write to it: no

  • customer-support-agent/package.json:17
  • customer-support-agent/package.json:10 (the aliased target)
  • customer-support-agent/amplify.yml:9

decided by agent · confidence 0.75

pre-commit hooks (.pre-commit-config.yaml)
anchored
.pre-commit-config.yaml · review_gate

The hook exit codes come from analyzers running over the committed source at pinned revisions, so an author cannot make a lint error or a type error report clean without changing the file or changing a pinned rev in a reviewable diff. Two documented limits: (a) three of the six hooks (end-of-file-fixer, trailing-whitespace, `ruff --fix-only`, `ruff format`) are rewriters — pre-commit still fails when they modify a file, so they are falsifiable, but they assert formatting, not correctness; (b) enforcement is local-only — no workflow runs `pre-commit run --all-files` and there is no pre-commit.ci config, so a developer who never installs the hook, or who passes `git commit --no-verify`, is never touched by any of it.

producer the ruff (v0.6.7) and pyright (v1.1.384) processes that pre-commit installs from pinned upstream revs and executes against the staged computer-use-demo files; check-yaml additionally parses each YAML file with a real parser — actor can write to it: no

  • .pre-commit-config.yaml:10-24 (ruff v0.6.7, pyright v1.1.384)
  • .pre-commit-config.yaml:1 (files: ^computer-use-demo/)
  • .github/workflows/tests.yaml (no pre-commit job)
  • computer-use-demo/dev-requirements.txt:3 (pre-commit==3.8.0, installed locally)

decided by agent · confidence 0.7

linter/type-checker config ([tool.pyright])
anchored
pyproject.toml:1 · ci_step

This is configuration, not an invocation, so I traced its consumer: the only thing that reads THIS file is the pyright pre-commit hook (.pre-commit-config.yaml:21-24), which runs from the repo root. The CI pyright job does not read it — jakebailey/pyright-action@v1 runs with `working-directory: computer-use-demo` and therefore resolves computer-use-demo/pyproject.toml instead. So the producer is a real type checker whose verdict the author cannot forge, but the edge this config feeds is a bypassable local hook, not the merge gate. Confidence is deliberately low: anchored-with-low-confidence is the honest shape for a config block whose only consumer runs on developer machines. Second limit: `useLibraryCodeForTypes = false` suppresses inference from untyped dependencies, narrowing what the checker can detect.

producer the pyright type checker, when invoked from the repository root — its exit code is decided by static analysis of the computer-use-demo sources against the interpreter environment named by venvPath/venv — actor can write to it: no

  • pyproject.toml:1-4
  • .pre-commit-config.yaml:21-24 (pyright hook, run from repo root)
  • .github/workflows/tests.yaml:40-42 (CI pyright uses working-directory: computer-use-demo)
  • computer-use-demo/pyproject.toml:1-8 (the config CI actually resolves)

decided by agent · confidence 0.5 · confidence below 0.6 on an anchored verdict

astral-sh/ruff-action@v1
anchored
.github/workflows/tests.yaml:21 · ci_step

The step's exit code is the linter's exit code over the checked-out source, and it is the only step in the `ruff` job that can fail, so it is the job's verdict on a pull request. A contributor cannot make a violating file pass without editing the file or adding a `[tool.ruff]` table (computer-use-demo/pyproject.toml currently declares none, so ruff runs its default E/F rule set). Documented limit, and the reason confidence is not higher: the action is referenced at the mutable tag `@v1`, not a commit SHA, so the implementation that actually runs is not pinned by this repository and is not present in the tree — I am reading the action's contract from its inputs (`src:`) rather than from its code.

producer the ruff binary the action installs, executing `ruff check` over the `computer-use-demo` source tree on the GitHub-hosted runner — actor can write to it: no

  • .github/workflows/tests.yaml:21-23
  • .github/workflows/tests.yaml:14-19 (the ruff job has only checkout + this step)
  • .github/workflows/tests.yaml:3-6 (runs on pull_request touching computer-use-demo/**)
  • computer-use-demo/pyproject.toml (no [tool.ruff] table — default rules)

decided by agent · confidence 0.7

jakebailey/pyright-action@v1
anchored
.github/workflows/tests.yaml:40 · ci_step

The exit code comes from static analysis of the committed Python sources; a contributor cannot make a type error disappear without changing the code or changing a reviewable config key. It is the final and only assertion step of the `pyright` job, so it is what turns that job red on a pull request touching computer-use-demo/**. Two documented limits: the action is at the mutable tag `@v1` rather than a SHA, so its body is neither in the tree nor pinned; and the effective config sets `useLibraryCodeForTypes = false` and `reportPrivateImportUsage = false` in default (basic) mode, which narrows what the checker will flag.

producer the pyright type checker executing over computer-use-demo, resolving computer-use-demo/pyproject.toml's [tool.pyright] table and the .venv built two steps earlier — actor can write to it: no

  • .github/workflows/tests.yaml:40-42
  • .github/workflows/tests.yaml:24-28 (job defaults working-directory: computer-use-demo)
  • computer-use-demo/pyproject.toml:1-8
  • .github/workflows/tests.yaml:35-39 (venv + PATH the checker runs against)

decided by agent · confidence 0.7

pytest config ([tool.pytest.ini_options])
anchored
computer-use-demo/pyproject.toml:10 · test_target

I traced this config to a real invocation: tests.yaml:59 runs pytest in computer-use-demo on every pull request touching that directory, and the runner's exit code — decided by whether the committed loop_test.py / streamlit_test.py / tools tests raise — turns the job red. The config itself only sets pythonpath and asyncio_mode; it does not weaken any assertion. Documented limit (the oracle axis): the assertions were authored by the same contributors as the implementation, so execution independence holds while the specification does not — the suite proves the code does what the tests say, not that the tests say the right thing.

producer the CPython interpreter running the committed tests under pytest 8.3.3 — `pytest tests --junitxml=junit/test-results.xml` in the `pytest` job — actor can write to it: no

  • computer-use-demo/pyproject.toml:10-12
  • .github/workflows/tests.yaml:59
  • .github/workflows/tests.yaml:43-47
  • computer-use-demo/dev-requirements.txt:4-5 (pytest==8.3.3, pytest-asyncio==0.23.6)
  • computer-use-demo/tests/ (loop_test.py, streamlit_test.py, tools/)

decided by agent · confidence 0.75

pytest config ([tool.pytest.ini_options])
anchored
computer-use-best-practices/pyproject.toml:46 · test_target

The suite is real — tests/ holds 16 test_*.py modules with executed assertions, and pytest's exit code is decided by the interpreter, which no contributor can write to without changing the code or the tests. The limit is enforcement, and it is severe enough that I am recording it here rather than in prose: NOTHING in this repository invokes it. Both workflows filter `paths:` to `.github/**` and `computer-use-demo/**`, the pre-commit config filters `files: ^computer-use-demo/`, and there is no Makefile, tox.ini or noxfile — the only invocation anywhere is `python -m pytest` in the subproject README. So the producer is anchored while the edge gates no merge; low confidence is the honest expression of that gap, and I have deliberately NOT filed it under not_a_check, which would have removed it from the denominator and inflated the score.

producer the CPython interpreter running the 16 committed test modules under pytest, when a developer invokes it by hand — actor can write to it: no

  • computer-use-best-practices/pyproject.toml:46-47
  • computer-use-best-practices/tests/ (16 test_*.py modules)
  • computer-use-best-practices/README.md:427 (`python -m pytest`, manual)
  • .github/workflows/tests.yaml:4-12 and .github/workflows/build.yaml:5-14 (path filters exclude this directory)
  • .pre-commit-config.yaml:1 (files: ^computer-use-demo/)

decided by agent · confidence 0.45 · confidence below 0.6 on an anchored verdict

pytest config (pytest.ini)
anchored
browser-use-demo/pytest.ini · test_target

Four committed test modules are discovered by this config and their assertions are decided by the runtime; `--strict-markers` additionally makes an undeclared marker a hard error rather than a warning, so the config is falsifiable on its own terms. Same enforcement limit as computer-use-best-practices, verified the same way: no workflow covers browser-use-demo (both workflows filter to `.github/**` and `computer-use-demo/**`), build.sh only runs `docker build`, and there is no Makefile or tox config — so this suite gates no merge and its only documented invocation is `pytest tests/` in tests/README.md. Confidence is low for exactly that reason. Second limit: `--disable-warnings` and blanket `filterwarnings` ignores suppress deprecation signal the suite would otherwise surface.

producer the CPython interpreter running tests/test_*.py under pytest with `--strict-markers`, when a developer invokes it by hand — actor can write to it: no

  • browser-use-demo/pytest.ini:1-40
  • browser-use-demo/tests/ (test_integration.py, test_message_renderer.py, test_sampling_loop.py, test_streamlit_helpers.py)
  • browser-use-demo/tests/README.md:19 (`pytest tests/`, manual)
  • browser-use-demo/build.sh:5 (docker build only)
  • .github/workflows/tests.yaml:4-12 (path filter excludes browser-use-demo)

decided by agent · confidence 0.45 · confidence below 0.6 on an anchored verdict

not_a_check — 15

Asserts nothing about correctness, so it is excluded from the denominator. This is the one shoppable class: mis-filing a real check here shrinks the denominator and inflates the score, so each row carries the same burden of argument as any other verdict.

dev
not_a_check
customer-support-agent/package.json · script

The script starts Next.js in development mode and blocks serving requests until interrupted; it never terminates with a pass/fail verdict about the committed source. The only variation in its exit status comes from how the operator kills it, so no property of the code can make it succeed or fail. Nothing in amplify.yml or the three workflow files invokes it either.

producer the `next dev` development server process (a long-running HTTP server), plus two NEXT_PUBLIC_* env assignments — actor can write to it: not established

  • customer-support-agent/package.json:9
  • customer-support-agent/amplify.yml:6-13 (deploy runs `npm ci` then `npm run build`, never `dev`)

decided by agent · confidence 0.9

start
not_a_check
customer-support-agent/package.json · script

It serves an already-built .next directory over HTTP and runs until killed. It compares nothing, asserts nothing about correctness, and produces no terminating verdict; a broken application starts exactly as happily as a correct one.

producer the `next start` production server process — actor can write to it: not established

  • customer-support-agent/package.json:11

decided by agent · confidence 0.9

dev:full
not_a_check
customer-support-agent/package.json · script

Descending one hop: this is a pure alias for the dev-server script at package.json:9. A blocking development server terminates on operator interrupt, not on a property of the code, so it cannot report a verdict about anything.

producer `npm run dev`, which re-enters the `dev` script and starts the `next dev` server — actor can write to it: not established

  • customer-support-agent/package.json:13
  • customer-support-agent/package.json:9 (the aliased target)

decided by agent · confidence 0.9

dev:left
not_a_check
customer-support-agent/package.json · script

Identical causal path to `dev`: a blocking HTTP dev server started with a different NEXT_PUBLIC_* flag combination. Feature-flag permutation changes what is rendered, not whether anything is evaluated — nothing is compared, so nothing can fail.

producer the `next dev` development server process, with the right sidebar feature flag off — actor can write to it: not established

  • customer-support-agent/package.json:14

decided by agent · confidence 0.9

dev:right
not_a_check
customer-support-agent/package.json · script

Same as `dev:left` — a long-running development server under a different NEXT_PUBLIC_* flag combination. It renders an application; it evaluates no property of the committed source and emits no pass/fail signal.

producer the `next dev` development server process, with the left sidebar feature flag off — actor can write to it: not established

  • customer-support-agent/package.json:15

decided by agent · confidence 0.9

dev:chat
not_a_check
customer-support-agent/package.json · script

The fourth permutation of the same dev-server invocation. A server that runs until interrupted produces no terminating verdict, so there is no signal here to trace to a producer inside or outside the write boundary.

producer the `next dev` development server process, with both sidebar feature flags off — actor can write to it: not established

  • customer-support-agent/package.json:16

decided by agent · confidence 0.9

actions/checkout@v4
not_a_check
.github/workflows/tests.yaml:20 · ci_step

The step's whole effect is to put files or a toolchain on the runner so later steps have something to act on. It reads nothing about the repository's correctness and emits no pass/fail verdict about the artefact — a failure here means the runner could not fetch or install, not that the code is wrong. The check in such a job is whatever executes afterwards, so this edge must not enter the ratio in either direction.

producer a GitHub Actions provisioning action (repository checkout, toolchain setup, or dependency cache restore) — actor can write to it: not established

  • the step body is a bare `uses:` of a checkout/setup/cache action with no command of its own

decided by probe (ci-provisioning-action) · confidence 0.9

actions/checkout@v4
not_a_check
.github/workflows/tests.yaml:30 · ci_step

The step's whole effect is to put files or a toolchain on the runner so later steps have something to act on. It reads nothing about the repository's correctness and emits no pass/fail verdict about the artefact — a failure here means the runner could not fetch or install, not that the code is wrong. The check in such a job is whatever executes afterwards, so this edge must not enter the ratio in either direction.

producer a GitHub Actions provisioning action (repository checkout, toolchain setup, or dependency cache restore) — actor can write to it: not established

  • the step body is a bare `uses:` of a checkout/setup/cache action with no command of its own

decided by probe (ci-provisioning-action) · confidence 0.9

actions/setup-python@v5
not_a_check
.github/workflows/tests.yaml:31 · ci_step

The step's whole effect is to put files or a toolchain on the runner so later steps have something to act on. It reads nothing about the repository's correctness and emits no pass/fail verdict about the artefact — a failure here means the runner could not fetch or install, not that the code is wrong. The check in such a job is whatever executes afterwards, so this edge must not enter the ratio in either direction.

producer a GitHub Actions provisioning action (repository checkout, toolchain setup, or dependency cache restore) — actor can write to it: not established

  • the step body is a bare `uses:` of a checkout/setup/cache action with no command of its own

decided by probe (ci-provisioning-action) · confidence 0.9

|
not_a_check
.github/workflows/tests.yaml:35 · ci_step

The command installs declared dependencies so that later steps can run. Without a lockfile-freezing flag the resolver will refresh a stale lock rather than error, so the step cannot report a discrepancy between the manifest and the lock; its non-zero exits mean the network or the index failed, not that the repository is wrong. The assertion in such a job lives in whatever executes against the installed environment afterwards.

producer a package manager's dependency resolver populating an environment — actor can write to it: not established

  • the run command is a bare dependency install with no --locked/--frozen equivalent

decided by probe (unfrozen-dependency-install) · confidence 0.8

echo "$PWD/.venv/bin" >> $GITHUB_PATH
not_a_check
.github/workflows/tests.yaml:39 · ci_step

The step's whole effect is to prepend the virtualenv's bin directory to PATH so the following pyright/pytest step resolves the installed tools. It reads no property of the committed source and performs no comparison — a string append to a runner-managed file succeeds regardless of whether the code is correct. Any assertion in this job is made by the step that runs afterwards.

producer the runner's `echo` builtin appending a directory string to the GITHUB_PATH file — actor can write to it: not established

  • .github/workflows/tests.yaml:39
  • .github/workflows/tests.yaml:35-38 (the venv it is pointing at)
  • .github/workflows/tests.yaml:40 (the step that consumes the PATH)

decided by agent · confidence 0.9

actions/checkout@v4
not_a_check
.github/workflows/tests.yaml:49 · ci_step

The step's whole effect is to put files or a toolchain on the runner so later steps have something to act on. It reads nothing about the repository's correctness and emits no pass/fail verdict about the artefact — a failure here means the runner could not fetch or install, not that the code is wrong. The check in such a job is whatever executes afterwards, so this edge must not enter the ratio in either direction.

producer a GitHub Actions provisioning action (repository checkout, toolchain setup, or dependency cache restore) — actor can write to it: not established

  • the step body is a bare `uses:` of a checkout/setup/cache action with no command of its own

decided by probe (ci-provisioning-action) · confidence 0.9

actions/setup-python@v5
not_a_check
.github/workflows/tests.yaml:50 · ci_step

The step's whole effect is to put files or a toolchain on the runner so later steps have something to act on. It reads nothing about the repository's correctness and emits no pass/fail verdict about the artefact — a failure here means the runner could not fetch or install, not that the code is wrong. The check in such a job is whatever executes afterwards, so this edge must not enter the ratio in either direction.

producer a GitHub Actions provisioning action (repository checkout, toolchain setup, or dependency cache restore) — actor can write to it: not established

  • the step body is a bare `uses:` of a checkout/setup/cache action with no command of its own

decided by probe (ci-provisioning-action) · confidence 0.9

pytest config (conftest.py)
not_a_check
computer-use-demo/tests/conftest.py · test_target

The whole file is one `mock.patch.dict` fixture and its yield — it contains no assert, no `def test_`, and no collection hook that can fail. It cannot turn the suite red; every assertion in this job lives in the test modules the fixture merely sets up. Worth naming rather than crediting: what it does do is substitute fabricated screen dimensions for the real environment, so it narrows what the tests it supports can observe.

producer an autouse pytest fixture that patches os.environ with HEIGHT/WIDTH/DISPLAY_NUM for the duration of every test — actor can write to it: not established

  • computer-use-demo/tests/conftest.py:1-15
  • computer-use-demo/tests/conftest.py:8-12 (mock.patch.dict of HEIGHT/WIDTH/DISPLAY_NUM)

decided by agent · confidence 0.75

pytest config (conftest.py)
not_a_check
browser-use-demo/tests/conftest.py · test_target

The file defines fixtures and mutates sys.path; it contains no assert and no test function, so it can never turn a run red — the assertions live in the four test modules it feeds. It is also worth naming that these fixtures replace BrowserTool and every streamlit call with mocks, so the suite they support never exercises the real Playwright or Streamlit paths; that weakens the oracle of the tests downstream, but it is not itself a check.

producer pytest fixtures that patch streamlit's module surface and BrowserTool with MagicMock objects, plus a sys.path insert — actor can write to it: not established

  • browser-use-demo/tests/conftest.py:1-60
  • browser-use-demo/tests/conftest.py (mock_streamlit and mock_browser_tool fixtures; patch of browser_use_demo.tools.BrowserTool)

decided by agent · confidence 0.8

Crystallization curve

Keel crystallization curve Crystallization curve over 15 runs from <repo>/reports. falls — total fitted change -66.5% of the mean across 15 runs (R^2 0.28, so the line explains less than half the variance — read the raw squares) CRYSTALLIZATION CURVE 15 sequential runs · 345 nodes judged of 1838 gathered · 124 anchored Keel corpus — 15 repositories, 2026-07-24 — measured corpus (declared on the command line) estimated tokens per node estimated tokens / judged node 0 300 600 run 0 · keel · estimated tokens per node = 498 · judged 25 of 32 gathered run 1 · anthropic-sdk-python · estimated tokens per node = 369 · judged 25 of 41 gathered run 2 · openai-python · estimated tokens per node = 398 · judged 25 of 95 gathered run 3 · vercel-ai · estimated tokens per node = 410 · judged 25 of 1014 gathered run 4 · aider · estimated tokens per node = 204 · judged 25 of 62 gathered run 5 · browser-use · estimated tokens per node = 378 · judged 25 of 104 gathered run 6 · mcp-python-sdk · estimated tokens per node = 592 · judged 25 of 121 gathered run 7 · simonw-llm · estimated tokens per node = 189 · judged 25 of 38 gathered run 8 · tiktoken · estimated tokens per node = 240 run 9 · requests · estimated tokens per node = 290 · judged 25 of 106 gathered run 10 · flask · estimated tokens per node = 408 · judged 25 of 59 gathered run 11 · sinatra · estimated tokens per node = 265 · judged 25 of 59 gathered run 12 · commander-js · estimated tokens per node = 316 run 13 · anthropic-quickstarts · estimated tokens per node = 310 · judged 25 of 62 gathered run 14 · tiktoken · estimated tokens per node = 53 0* 1* 2* 3* 4* 5* 6* 7* 8 9* 10* 11* 12 13* 14 seconds per node measured s / judged node 0.0 60.0 120.0 run 0 · keel · seconds per node = 0.1 · judged 25 of 32 gathered run 1 · anthropic-sdk-python · seconds per node = 15.8 · judged 25 of 41 gathered run 2 · openai-python · seconds per node = 15.4 · judged 25 of 95 gathered run 3 · vercel-ai · seconds per node = 105.9 · judged 25 of 1014 gathered run 4 · aider · seconds per node = 21.3 · judged 25 of 62 gathered run 5 · browser-use · seconds per node = 21.7 · judged 25 of 104 gathered run 6 · mcp-python-sdk · seconds per node = 20.1 · judged 25 of 121 gathered run 7 · simonw-llm · seconds per node = 11.8 · judged 25 of 38 gathered run 8 · tiktoken · seconds per node = 23.7 run 9 · requests · seconds per node = 16.8 · judged 25 of 106 gathered run 10 · flask · seconds per node = 20.5 · judged 25 of 59 gathered run 11 · sinatra · seconds per node = 16.9 · judged 25 of 59 gathered run 12 · commander-js · seconds per node = 18.7 run 13 · anthropic-quickstarts · seconds per node = 20.4 · judged 25 of 62 gathered run 14 · tiktoken · seconds per node = 0.1 0* 1* 2* 3* 4* 5* 6* 7* 8 9* 10* 11* 12 13* 14 probe-decided share measured share of decided nodes 0.00 0.50 1.00 run 0 · keel · probe-decided share = 0.00 · judged 25 of 32 gathered run 1 · anthropic-sdk-python · probe-decided share = 0.00 · judged 25 of 41 gathered run 2 · openai-python · probe-decided share = 0.16 · judged 25 of 95 gathered run 3 · vercel-ai · probe-decided share = 0.08 · judged 25 of 1014 gathered run 4 · aider · probe-decided share = 0.56 · judged 25 of 62 gathered run 5 · browser-use · probe-decided share = 0.32 · judged 25 of 104 gathered run 6 · mcp-python-sdk · probe-decided share = 0.00 · judged 25 of 121 gathered run 7 · simonw-llm · probe-decided share = 0.56 · judged 25 of 38 gathered run 8 · tiktoken · probe-decided share = 0.58 run 9 · requests · probe-decided share = 0.28 · judged 25 of 106 gathered run 10 · flask · probe-decided share = 0.00 · judged 25 of 59 gathered run 11 · sinatra · probe-decided share = 0.28 · judged 25 of 59 gathered run 12 · commander-js · probe-decided share = 0.19 run 13 · anthropic-quickstarts · probe-decided share = 0.24 · judged 25 of 62 gathered run 14 · tiktoken · probe-decided share = 1.00 0* 1* 2* 3* 4* 5* 6* 7* 8 9* 10* 11* 12 13* 14 probe library size measured probes in library 0 15 30 run 0 · keel · probe library size = 0 · judged 25 of 32 gathered run 1 · anthropic-sdk-python · probe library size = 3 · judged 25 of 41 gathered run 2 · openai-python · probe library size = 5 · judged 25 of 95 gathered run 3 · vercel-ai · probe library size = 8 · judged 25 of 1014 gathered run 4 · aider · probe library size = 10 · judged 25 of 62 gathered run 5 · browser-use · probe library size = 12 · judged 25 of 104 gathered run 6 · mcp-python-sdk · probe library size = 13 · judged 25 of 121 gathered run 7 · simonw-llm · probe library size = 15 · judged 25 of 38 gathered run 8 · tiktoken · probe library size = 17 run 9 · requests · probe library size = 19 · judged 25 of 106 gathered run 10 · flask · probe library size = 22 · judged 25 of 59 gathered run 11 · sinatra · probe library size = 23 · judged 25 of 59 gathered run 12 · commander-js · probe library size = 26 run 13 · anthropic-quickstarts · probe library size = 29 · judged 25 of 62 gathered run 14 · tiktoken · probe library size = 29 0* 1* 2* 3* 4* 5* 6* 7* 8 9* 10* 11* 12 13* 14 x axis: run index in the recorded corpus order (listed below). Squares are the raw per-run values; the dashed line is an ordinary-least-squares fit and is never shown without them. The fit is clipped to the panel, and withheld entirely (with the panel saying so) where a straight line would predict values the points cannot take — R^2 for every fit is in the trend notes below. * on run 0, 1, 2, 3, 4, 5, 6, 7, 9, 10, 11, 13 = judged fewer nodes than gathered; per-node values are per JUDGED node. Coverage (judged) ci_step 224 · script 64 · test_target 46 · review_gate 10 · deploy_gate 1 Trend estimated tokens per node: falls — total fitted change -66.5% of the mean across 15 runs (R^2 0.28, so the line explains less than half the variance — read the raw squares). raw first 498 -> last 53. seconds per node: falls — total fitted change -72.9% of the mean across 15 runs (R^2 0.04, so the line explains less than half the variance — read the raw squares). raw first 0.1 -> last 0.1. probe-decided share: rises — total fitted change 144.9% of the mean across 15 runs (R^2 0.21, so the line explains less than half the variance — read the raw squares). raw first 0.00 -> last 1.00. probe library size: rises — total fitted change 187.7% of the mean across 15 runs (R^2 0.99). raw first 0 -> last 29. Run order run order derived from Report.generatedAt (ascending, filename as tiebreak) — no order.json manifest present 0 keel -> 1 anthropic-sdk-python -> 2 openai-python -> 3 vercel-ai -> 4 aider -> 5 browser-use -> 6 mcp-python-sdk -> 7 simonw-llm -> 8 tiktoken -> 9 requests -> 10 flask -> 11 sinatra -> 12 commander-js -> 13 anthropic-quickstarts -> 14 tiktoken Shuffle check shuffle check INCOMPLETE — the curve direction ran, the ratio direction did not Permutation (400 draws, seed 20260724): median |delta normalized slope| 0.633, 92% of draws move it by >= 0.15. A permutation reorders ALREADY-RECORDED runs; it cannot reproduce what a genuine re-run in a different order would have cost, because the probe library would have accumulated differently. It is a sanity signal on order-dependence, never a substitute for the empirical re-run. Ratio stability under shuffle was NOT checked: no re-run supplied (--shuffled) and none declared in corpus.meta.json. Permuting recorded runs cannot move a per-target ratio, so the permutation result below says nothing about it. Supply a re-run with --shuffled <dir>. Disclosures - Skipped 3 file(s) that are not usable run reports: corpus-summary.json (no string "target"); curve.json (no string "target"); keel.bindings.json (no "nodes" array). - Run 0 (keel) judged 25 of 32 gathered nodes — a cap. Per-node cost below is per JUDGED node. - Run 1 (anthropic-sdk-python) judged 25 of 41 gathered nodes — a cap. Per-node cost below is per JUDGED node. - Run 2 (openai-python) judged 25 of 95 gathered nodes — a cap. Per-node cost below is per JUDGED node. - Run 3 (vercel-ai) judged 25 of 1014 gathered nodes — a cap. Per-node cost below is per JUDGED node. - Run 4 (aider) judged 25 of 62 gathered nodes — a cap. Per-node cost below is per JUDGED node. - Run 5 (browser-use) judged 25 of 104 gathered nodes — a cap. Per-node cost below is per JUDGED node. - Run 6 (mcp-python-sdk) judged 25 of 121 gathered nodes — a cap. Per-node cost below is per JUDGED node. - Run 7 (simonw-llm) judged 25 of 38 gathered nodes — a cap. Per-node cost below is per JUDGED node. - Run 9 (requests) judged 25 of 106 gathered nodes — a cap. Per-node cost below is per JUDGED node. - Run 10 (flask) judged 25 of 59 gathered nodes — a cap. Per-node cost below is per JUDGED node. - Run 11 (sinatra) judged 25 of 59 gathered nodes — a cap. Per-node cost below is per JUDGED node. - Run 13 (anthropic-quickstarts) judged 25 of 62 gathered nodes — a cap. Per-node cost below is per JUDGED node. - Run 14 (tiktoken): decidedByProbe + decidedByAgent = 10 but nodesSampled = 12. Probe-decided share uses the decided total (10). - Provenance "measured" was declared on the COMMAND LINE (--provenance), not in <repo>/reports/corpus.meta.json. The declaration therefore lives in the invocation and is only as trustworthy as the run sheet that records it. - Token counts are ESTIMATES in 15 of 15 run(s) (RunEconomics.tokensEstimated). No API exposes session token usage to a skill, so the token axis reads "estimated tokens" and nothing here claims a measured token count. Scope. Keel measures the shape of verification, not its quality. A repo can be 100% anchored with terrible tests. Anchoring says the signal comes from outside; it does not say the signal is sufficient.

Run economics

6 decided by probe
19 decided by agent
24% probe-decided share
25 / 62 nodes judged / gathered
2,612 estimated tokens in
5,144 estimated tokens out
8m 30s wall clock
3 probes minted
29 probe library size

Token counts are estimated, and labelled so: a skill running inside an agent session has no API for its own usage, so the figure is ceil(chars/4) over the judgment payloads and responses. Wall clock and probe-decided share are measured directly. 25 of 62 gathered edges were judged — the cap is printed here and beside the ratio, never applied silently.