Keel

Grounding report.

Every verification edge in browser-use, classified by who produces the signal and whether the actor being verified can write to that producer.

targetbrowser-use
revisionb909fbfba0ae42790f8402dd3daecc8f265be246
generated2026-07-25T02:46:11.681Z
1.00

anchored / (anchored + self_referential + unknown) = 7 / 7

anchored 7 self_referential 0 unknown 0 not_a_check 18 excluded from the denominator

Coverage (judged)

node kindjudged
ci_step22
test_target2
review_gate1

Nodes by kind, over the 25 node(s) carried in this report. Surfaces the gatherer cannot read are absent here rather than counted as unknown, which makes non-coverage Keel's own shoppable class — so it is stated, not implied.

ε-audit — not run

No verdict in this report carries a readable audit block, so nothing was re-decided and there is no agreement rate to report. This is not agreement, and it is not full agreement either: a rate over zero comparisons would state a measurement that was never taken. (“Readable” is load-bearing: a block can be present and state no comparison, and this state covers that too — see below if so.)

This run carries 8 probe-decided verdict(s), none of them contributing a readable comparison. (“Readable” rather than “audited”, because a block can arrive and state nothing — if any did, they are counted below.) Audit coverage is itself shoppable — audit none of the library and no disagreement can ever be found — so the absence is printed here rather than left to be inferred from a missing section.

This run judged 25 of 104 gathered edges. Every number on this page describes that sample, not the whole surface — 79 gathered edges are absent from all of it. A cap is disclosed rather than smoothed: an undisclosed sample is how a ratio gets shopped.

Scope. Keel measures the shape of verification, not its quality. A repo can be 100% anchored with terrible tests. Anchoring says the signal comes from outside; it does not say the signal is sufficient.

Node graph — 25 node(s), in gather order

anchored 7 not_a_check 18

1 mark is ringed in the unknown hue: an anchored verdict asserted below confidence 0.6. The ring says the claim is closer to unknown than its colour suggests. Hover a mark for its node.

anchored — 7

The producer sits outside the write boundary of the actor being verified.

pre-commit hooks (.pre-commit-config.yaml)
anchored
.pre-commit-config.yaml · review_gate

This hook set is not decorative: .github/workflows/lint.yml:43 runs `uv run --no-sync pre-commit run --all-files --show-diff-on-failure` on every pull_request, so the gate is on the merge path. Each hook is a separate process fetched from an external repository at a declared rev and run over the files as committed; a violation makes that process exit non-zero and pre-commit propagates the failure, and a rewriting hook (ruff-format, end-of-file-fixer, fix-byte-order-marker) also fails the run because pre-commit treats a modified file as a failed hook. An author cannot make check-ast accept a syntax error, or detect-private-key ignore a key, without changing the file or changing the pinned hook config. Documented limits: the revs are mutable tags (v0.12.10, v1.1.404, v6.0.0) rather than SHAs, so the upstream owner — not the PR author — could move what runs; and the hook LIST itself is author-controlled, so several checks are commented out (add-trailing-comma, check-docstring-first, name-tests-test) and simply do not run.

producer exit codes of pinned third-party programs (ruff, pyright, codespell, pyupgrade, yesqa, pre-commit-hooks) executed by the pre-commit runner against the committed working tree — actor can write to it: no

  • .pre-commit-config.yaml
  • .github/workflows/lint.yml:43
  • .github/workflows/lint.yml:16

decided by agent · confidence 0.8

pytest config ([tool.pytest.ini_options])
anchored
pyproject.toml:190 · test_target

Unlike the linter tables above, this node is the declaration of the test suite itself (`testpaths = ["tests"]`, `python_files`), and that suite is executed on the merge path: .github/workflows/test.yaml:182 runs `pytest "tests/ci/<file>.py"` as a matrix job over every discovered test file, with `max_attempts: 1` and no continue-on-error, so a non-zero pytest exit fails the required job. Nothing an author writes in the module under test can turn a raised AssertionError or a segfaulting browser process into a pass without changing the test or the code. Documented limits, on the axis this does NOT cover: the oracle is co-authored — the assertions live in the same repository as the implementation, so the suite proves the code does what the tests say and not that the tests say the right thing; `addopts = "-svx"` stops at the first failure; `timeout = 300` and the job's 4-minute cap mean a slow suite is truncated by the clock; and tests/ci/conftest.py replaces the LLM with an AsyncMock returning canned JSON, so agent-behaviour tests exercise a scripted oracle rather than a real model.

producer the pytest process exit code — the CPython runtime executing the declared suite and the assertions in it — actor can write to it: no

  • pyproject.toml:190
  • .github/workflows/test.yaml:182
  • .github/workflows/test.yaml:100
  • tests/ci/conftest.py

decided by agent · confidence 0.6

Verify release environment is protected (fail-closed)
anchored
.github/workflows/publish.yml:88 · ci_step

The step asks GitHub for the release environment's protection_rules and exits 1 unless a required_reviewers rule exists AND prevent_self_review is true; `set -euo pipefail` plus the explicit `exit 1` on a failed API call make it fail closed if the environment is missing or unreadable. The value being read is not in the repository and is not writable by this job: GITHUB_TOKEN here carries contents/id-token/actions only, and changing environment protection needs repository-admin access through the settings UI. Nothing a PR author commits, and nothing an earlier step in this pipeline writes, can make an unprotected environment report as protected. Documented limit: a repository admin can turn the protection off — but then this step fails rather than silently passing, which is the property that makes it a real gate on the publish job at publish.yml:76.

producer GitHub's environments API (`GET /repos/{repo}/environments/release`) — repository settings stored in GitHub's control plane, outside the git tree — actor can write to it: no

  • .github/workflows/publish.yml:88-109
  • .github/workflows/publish.yml:79
  • .github/workflows/publish.yml:80-83

decided by agent · confidence 0.8

uv run --no-sync ruff check --no-fix --select PLE # quick ch
anchored
.github/workflows/publish.yml:117 · ci_step

`--no-fix` means ruff reports instead of rewriting, so a PLE-class error (Python syntax / pyflakes-error rules) yields a non-zero exit, and with no `continue-on-error` and no `|| true` that failure stops the publish job before `uv build` at line 118 and before `uv publish` at line 137. The author cannot make a file with a syntax error report clean without editing the file or editing the pinned ruff configuration. Documented limit: `--select PLE` is a deliberately tiny rule set — this is a fast syntax tripwire, not the repository's lint gate, which lives in lint.yml.

producer the ruff process exit code — a linter binary parsing the committed source and returning non-zero on a violation — actor can write to it: no

  • .github/workflows/publish.yml:117
  • .github/workflows/publish.yml:118
  • .github/workflows/publish.yml:137

decided by agent · confidence 0.9

uv build
anchored
.github/workflows/publish.yml:118 · ci_step

The build backend resolves the project metadata and assembles the sdist and wheel; malformed pyproject metadata, an unresolvable version, or a missing declared file makes the process exit non-zero, and with no failure suppression that stops the job before `uv publish` at line 137. The verdict is produced by the packaging toolchain executing over the committed tree, not by anything the authors assert about it. Documented limit — narrow scope, and it should never be read as more: a Python wheel build neither imports nor executes browser_use, so this says the package can be assembled, not that the code in it works.

producer the uv/PEP 517 build-backend process exit code, running against pyproject.toml and the source tree — actor can write to it: no

  • .github/workflows/publish.yml:118
  • .github/workflows/publish.yml:137

decided by agent · confidence 0.6

uv publish --trusted-publishing always
anchored
.github/workflows/publish.yml:137 · ci_step

`--trusted-publishing always` forces the OIDC path: the job exchanges a GitHub-signed id-token (permissions: id-token: write, publish.yml:82) for a short-lived PyPI credential, and PyPI decides whether the token's repository/workflow/environment claims match a configured trusted publisher. Neither the repository contents nor any earlier step in this pipeline can forge that token or overrule PyPI's decision, and PyPI also rejects a version that already exists or metadata it will not accept — a non-zero exit here stops the stable-branch push at line 138. Documented limit, and it is a real one: what the third party validates is publisher identity, version uniqueness and package metadata. It says nothing about whether the code in the wheel behaves. Confidence is held low deliberately because this step's primary role is the release action itself; the assertion is a by-product of it.

producer PyPI's upload API accepting or rejecting the distribution, with the uploader identity minted as an OIDC token by GitHub's identity provider — actor can write to it: no

  • .github/workflows/publish.yml:137
  • .github/workflows/publish.yml:82
  • .github/workflows/publish.yml:138

decided by agent · confidence 0.55 · confidence below 0.6 on an anchored verdict

uv run ruff check --no-fix --select PLE
anchored
.github/workflows/lint.yml:31 · ci_step

This is the `lint-syntax` job's only real step and lint.yml triggers on `pull_request` (lint.yml:16), so its exit code is on the merge path. `--no-fix` keeps it reporting rather than rewriting, so committing a file with a Python syntax error or a pyflakes-error-class defect makes ruff exit non-zero and fails the job; no `|| true`, no continue-on-error, nothing to suppress it. The author would have to change the file or the pinned ruff configuration to get a clean result. Documented limit: `--select PLE` restricts this to the error-class subset, so it is a syntax tripwire; the broader rule set only runs through the pre-commit job at lint.yml:43.

producer the ruff process exit code — the linter binary analysing the checked-out source and returning non-zero on a violation — actor can write to it: no

  • .github/workflows/lint.yml:31
  • .github/workflows/lint.yml:16
  • .github/workflows/lint.yml:23

decided by agent · confidence 0.9

not_a_check — 18

Asserts nothing about correctness, so it is excluded from the denominator. This is the one shoppable class: mis-filing a real check here shrinks the denominator and inflates the score, so each row carries the same burden of argument as any other verdict.

linter/type-checker config ([tool.ruff])
not_a_check
pyproject.toml:119 · ci_step

`line-length = 130` and `fix = true` are parameters handed to a linter; the table is never executed and no exit code is derived from it. The signal-producing edges that read it are classified separately (.github/workflows/lint.yml:31 `ruff check --no-fix --select PLE`, and the ruff-check/ruff-format pre-commit hooks run at lint.yml:43). Filing it here is deliberate rather than convenient: counting a config table as a check would either add a phantom anchored edge or invent a producer that does not exist. Worth recording as shape, not as verdict: `fix = true` makes autofix the default, which is why the CI invocations must pass `--no-fix` to keep the run assertive rather than rewriting.

producer nothing — this is a TOML table read by ruff, not a process that emits a pass/fail — actor can write to it: no

  • pyproject.toml:119
  • .github/workflows/lint.yml:31

decided by agent · confidence 0.7

linter/type-checker config ([tool.ruff.lint])
not_a_check
pyproject.toml:123 · ci_step

`select`/`ignore`/`unfixable` set which rules ruff enforces; the exit code that can block a merge is produced by the ruff process at .github/workflows/lint.yml:31 and by the ruff-check hook under lint.yml:43, both classified on their own. The table asserts nothing and cannot fail. It is, however, the actor-writable lever on the checks that do fire, and the narrowing here is real and should be read next to those verdicts: E501, F841 (unused variable), E731, E402, W291 and ASYNC109 are all ignored, and the standalone lint job at lint.yml:31 overrides `select` down to PLE alone, so the full rule set only gates via the pre-commit job.

producer nothing — a TOML rule-selection table consumed by ruff; it emits no verdict of its own — actor can write to it: no

  • pyproject.toml:123
  • .github/workflows/lint.yml:31
  • .github/workflows/lint.yml:43

decided by agent · confidence 0.7

linter/type-checker config ([tool.ruff.format])
not_a_check
pyproject.toml:136 · ci_step

This block chooses single quotes, tab indentation and LF line endings. It emits no signal, and what it configures is a rewriter: the ruff-format hook's job is to change files, not to make a claim about whether the program is correct. It does become failure-capable inside `pre-commit run --all-files --show-diff-on-failure` at .github/workflows/lint.yml:43 — a file the formatter rewrites fails the hook — but that gate is a statement about typography, not about correctness, and it is already carried by the .pre-commit-config.yaml verdict rather than by this table.

producer nothing — a TOML style table (quote-style, indent-style, line-ending) consumed by the ruff formatter — actor can write to it: no

  • pyproject.toml:136
  • .pre-commit-config.yaml
  • .github/workflows/lint.yml:43

decided by agent · confidence 0.65

linter/type-checker config ([tool.pyright])
not_a_check
pyproject.toml:144 · ci_step

`typeCheckingMode`, `include`, `exclude`, `venv` are inputs to pyright, not a verdict. The edges that actually gate are `uv run --no-sync pyright` at .github/workflows/lint.yml:54 and the pyright pre-commit hook at lint.yml:43, and those are where the anchored/self-referential question is decided. Recorded as shape because it directly bounds those edges: mode is `basic` (not strict), `reportMissingTypeStubs` is off, and the exclude list removes real product code from analysis — notably `browser_use/beta/service.py` together with `tests/ci/test_beta_agent.py`, plus browser_use/llm/oci_raw/* and two llm test modules. That narrowing is written by the same authors the checker is supposed to constrain.

producer nothing — a TOML scope/strictness table read by pyright; the type checker's exit code is produced by the invocations elsewhere — actor can write to it: no

  • pyproject.toml:144
  • .github/workflows/lint.yml:54
  • .github/workflows/lint.yml:43

decided by agent · confidence 0.7

pytest config (conftest.py)
not_a_check
tests/ci/conftest.py · test_target

This module sets env vars (SKIP_LLM_API_KEY_VERIFICATION, ANONYMIZED_TELEMETRY=false, placeholder cloud URLs), makes a temp config dir, disables blocking socket shutdown, and builds fixtures. It is harness provisioning: nothing here can fail in a way that says anything about the correctness of browser_use. The assertions it supports run under the pytest edge classified at pyproject.toml#tool-pytest-ini-options. What it does deserve recording is where it puts the write boundary for those assertions: `create_mock_llm` returns an AsyncMock whose responses are hard-coded JSON agent outputs authored in this file, so every test that drives the agent loop through `mock_llm` is scoring the code against a script the test author wrote, not against a model's behaviour — execution stays anchored, the oracle does not.

producer nothing — pytest fixtures and environment setup; the file contains no test and emits no pass/fail — actor can write to it: no

  • tests/ci/conftest.py
  • .github/workflows/test.yaml:182

decided by agent · confidence 0.7

actions/stale@5bef64f19d7facfb25b37b414482c7164d639639 # v9
not_a_check
.github/workflows/stale-bot.yml:15 · ci_step

Every input to this step is an inactivity timer (days-before-stale 60/45, days-before-close 14) and every output is a label, a comment, or a close. It makes no claim about whether any code, artifact or deployment is correct, and there is no pass/fail it could contribute to a merge decision — `debug-only: false` just means it acts for real. It is repository housekeeping, so it is excluded from the ratio rather than counted as a weak check. The pin is a full commit SHA, so the implementation is at least identified even though it lives outside this repo — but that is moot here, since the step asserts nothing regardless of what the action does internally.

producer actions/stale reading issue/PR timestamps from the GitHub API and writing labels and comments back — actor can write to it: no

  • .github/workflows/stale-bot.yml:15

decided by agent · confidence 0.9

actions/checkout@v4
not_a_check
.github/workflows/publish.yml:38 · ci_step

The step's whole effect is to put files or a toolchain on the runner so later steps have something to act on. It reads nothing about the repository's correctness and emits no pass/fail verdict about the artefact — a failure here means the runner could not fetch or install, not that the code is wrong. The check in such a job is whatever executes afterwards, so this edge must not enter the ratio in either direction.

producer a GitHub Actions provisioning action (repository checkout, toolchain setup, or dependency cache restore) — actor can write to it: not established

  • the step body is a bare `uses:` of a checkout/setup/cache action with no command of its own

decided by probe (ci-provisioning-action) · confidence 0.9

Create pre-release tag
not_a_check
.github/workflows/publish.yml:39 · ci_step

The step's purpose is mutation: compute the next rc version and `git tag` / `git push origin $new_tag`. It does contain `exit 1` branches, but every one of them fires on a property of the tag namespace the same maintainers push (no tag matched the version regex, or the rc counter hit 99) — never on a property of the code being released. So it asserts nothing about correctness and is excluded from the ratio rather than counted; the reason `actorCanWrite` is true here is recorded because the only thing it could be said to validate is an input the actor authored, which is why treating it as a check would have been the wrong call in the other direction.

producer bash + git, reading the repository's own tag list and writing a new tag back to origin — actor can write to it: yes

  • .github/workflows/publish.yml:39-74
  • .github/workflows/publish.yml:35

decided by agent · confidence 0.65

actions/checkout@v4
not_a_check
.github/workflows/publish.yml:110 · ci_step

The step's whole effect is to put files or a toolchain on the runner so later steps have something to act on. It reads nothing about the repository's correctness and emits no pass/fail verdict about the artefact — a failure here means the runner could not fetch or install, not that the code is wrong. The check in such a job is whatever executes afterwards, so this edge must not enter the ratio in either direction.

producer a GitHub Actions provisioning action (repository checkout, toolchain setup, or dependency cache restore) — actor can write to it: not established

  • the step body is a bare `uses:` of a checkout/setup/cache action with no command of its own

decided by probe (ci-provisioning-action) · confidence 0.9

astral-sh/setup-uv@v6
not_a_check
.github/workflows/publish.yml:111 · ci_step

The step's whole effect is to put files or a toolchain on the runner so later steps have something to act on. It reads nothing about the repository's correctness and emits no pass/fail verdict about the artefact — a failure here means the runner could not fetch or install, not that the code is wrong. The check in such a job is whatever executes afterwards, so this edge must not enter the ratio in either direction.

producer a GitHub Actions provisioning action (repository checkout, toolchain setup, or dependency cache restore) — actor can write to it: not established

  • the step body is a bare `uses:` of a checkout/setup/cache action with no command of its own

decided by probe (ci-provisioning-action) · confidence 0.9

uv sync
not_a_check
.github/workflows/publish.yml:115 · ci_step

The command installs declared dependencies so that later steps can run. Without a lockfile-freezing flag the resolver will refresh a stale lock rather than error, so the step cannot report a discrepancy between the manifest and the lock; its non-zero exits mean the network or the index failed, not that the repository is wrong. The assertion in such a job lives in whatever executes against the installed environment afterwards.

producer a package manager's dependency resolver populating an environment — actor can write to it: not established

  • the run command is a bare dependency install with no --locked/--frozen equivalent

decided by probe (unfrozen-dependency-install) · confidence 0.8

uvx playwright install chrome
not_a_check
.github/workflows/publish.yml:130 · ci_step

Pure machine preparation — it fetches a browser binary so later steps have one. Its only failure modes are network or CDN availability, which are properties of the runner's environment rather than of the repository, and no assertion about browser-use's correctness is derived from it. Note it is doing nothing for this job in particular: the test invocation it would have supported (`uv run pytest tests/ci/test_tools.py`) is commented out at publish.yml:134, so the download feeds no check at all in this workflow.

producer the playwright CLI downloading a Chrome build into the runner's cache — actor can write to it: no

  • .github/workflows/publish.yml:130
  • .github/workflows/publish.yml:134

decided by agent · confidence 0.9

uvx playwright install chromium
not_a_check
.github/workflows/publish.yml:131 · ci_step

Same shape as the chrome install one line above: a binary fetch that provisions the runner. It emits no verdict about the artifact being published, its failure modes are environmental, and the only step in this job that would have consumed the browser is commented out at publish.yml:134, so nothing downstream reads it.

producer the playwright CLI downloading a Chromium build into the runner's cache — actor can write to it: no

  • .github/workflows/publish.yml:131
  • .github/workflows/publish.yml:134

decided by agent · confidence 0.9

Push to stable branch (if stable release)
not_a_check
.github/workflows/publish.yml:138 · ci_step

`git checkout -b stable` followed by `git push origin -f stable` is state mutation with no predicate: it force-moves a branch pointer to whatever commit the release was cut from. There is nothing here that can fail on a property of the code, and the `if:` guard is a condition on the triggering event, not a verification. It is excluded from the ratio; recorded separately is that this is the step which makes the `stable` branch a claim the release process writes about itself, so anything downstream that trusts `stable` is trusting an actor-set pointer rather than an independent signal.

producer git, writing a ref to origin — actor can write to it: yes

  • .github/workflows/publish.yml:138-142

decided by agent · confidence 0.85

actions/checkout@v4
not_a_check
.github/workflows/lint.yml:27 · ci_step

The step's whole effect is to put files or a toolchain on the runner so later steps have something to act on. It reads nothing about the repository's correctness and emits no pass/fail verdict about the artefact — a failure here means the runner could not fetch or install, not that the code is wrong. The check in such a job is whatever executes afterwards, so this edge must not enter the ratio in either direction.

producer a GitHub Actions provisioning action (repository checkout, toolchain setup, or dependency cache restore) — actor can write to it: not established

  • the step body is a bare `uses:` of a checkout/setup/cache action with no command of its own

decided by probe (ci-provisioning-action) · confidence 0.9

astral-sh/setup-uv@v5
not_a_check
.github/workflows/lint.yml:28 · ci_step

The step's whole effect is to put files or a toolchain on the runner so later steps have something to act on. It reads nothing about the repository's correctness and emits no pass/fail verdict about the artefact — a failure here means the runner could not fetch or install, not that the code is wrong. The check in such a job is whatever executes afterwards, so this edge must not enter the ratio in either direction.

producer a GitHub Actions provisioning action (repository checkout, toolchain setup, or dependency cache restore) — actor can write to it: not established

  • the step body is a bare `uses:` of a checkout/setup/cache action with no command of its own

decided by probe (ci-provisioning-action) · confidence 0.9

actions/checkout@v4
not_a_check
.github/workflows/lint.yml:37 · ci_step

The step's whole effect is to put files or a toolchain on the runner so later steps have something to act on. It reads nothing about the repository's correctness and emits no pass/fail verdict about the artefact — a failure here means the runner could not fetch or install, not that the code is wrong. The check in such a job is whatever executes afterwards, so this edge must not enter the ratio in either direction.

producer a GitHub Actions provisioning action (repository checkout, toolchain setup, or dependency cache restore) — actor can write to it: not established

  • the step body is a bare `uses:` of a checkout/setup/cache action with no command of its own

decided by probe (ci-provisioning-action) · confidence 0.9

astral-sh/setup-uv@v5
not_a_check
.github/workflows/lint.yml:38 · ci_step

The step's whole effect is to put files or a toolchain on the runner so later steps have something to act on. It reads nothing about the repository's correctness and emits no pass/fail verdict about the artefact — a failure here means the runner could not fetch or install, not that the code is wrong. The check in such a job is whatever executes afterwards, so this edge must not enter the ratio in either direction.

producer a GitHub Actions provisioning action (repository checkout, toolchain setup, or dependency cache restore) — actor can write to it: not established

  • the step body is a bare `uses:` of a checkout/setup/cache action with no command of its own

decided by probe (ci-provisioning-action) · confidence 0.9

Crystallization curve

Keel crystallization curve Crystallization curve over 15 runs from <repo>/reports. falls — total fitted change -66.5% of the mean across 15 runs (R^2 0.28, so the line explains less than half the variance — read the raw squares) CRYSTALLIZATION CURVE 15 sequential runs · 345 nodes judged of 1838 gathered · 124 anchored Keel corpus — 15 repositories, 2026-07-24 — measured corpus (declared on the command line) estimated tokens per node estimated tokens / judged node 0 300 600 run 0 · keel · estimated tokens per node = 498 · judged 25 of 32 gathered run 1 · anthropic-sdk-python · estimated tokens per node = 369 · judged 25 of 41 gathered run 2 · openai-python · estimated tokens per node = 398 · judged 25 of 95 gathered run 3 · vercel-ai · estimated tokens per node = 410 · judged 25 of 1014 gathered run 4 · aider · estimated tokens per node = 204 · judged 25 of 62 gathered run 5 · browser-use · estimated tokens per node = 378 · judged 25 of 104 gathered run 6 · mcp-python-sdk · estimated tokens per node = 592 · judged 25 of 121 gathered run 7 · simonw-llm · estimated tokens per node = 189 · judged 25 of 38 gathered run 8 · tiktoken · estimated tokens per node = 240 run 9 · requests · estimated tokens per node = 290 · judged 25 of 106 gathered run 10 · flask · estimated tokens per node = 408 · judged 25 of 59 gathered run 11 · sinatra · estimated tokens per node = 265 · judged 25 of 59 gathered run 12 · commander-js · estimated tokens per node = 316 run 13 · anthropic-quickstarts · estimated tokens per node = 310 · judged 25 of 62 gathered run 14 · tiktoken · estimated tokens per node = 53 0* 1* 2* 3* 4* 5* 6* 7* 8 9* 10* 11* 12 13* 14 seconds per node measured s / judged node 0.0 60.0 120.0 run 0 · keel · seconds per node = 0.1 · judged 25 of 32 gathered run 1 · anthropic-sdk-python · seconds per node = 15.8 · judged 25 of 41 gathered run 2 · openai-python · seconds per node = 15.4 · judged 25 of 95 gathered run 3 · vercel-ai · seconds per node = 105.9 · judged 25 of 1014 gathered run 4 · aider · seconds per node = 21.3 · judged 25 of 62 gathered run 5 · browser-use · seconds per node = 21.7 · judged 25 of 104 gathered run 6 · mcp-python-sdk · seconds per node = 20.1 · judged 25 of 121 gathered run 7 · simonw-llm · seconds per node = 11.8 · judged 25 of 38 gathered run 8 · tiktoken · seconds per node = 23.7 run 9 · requests · seconds per node = 16.8 · judged 25 of 106 gathered run 10 · flask · seconds per node = 20.5 · judged 25 of 59 gathered run 11 · sinatra · seconds per node = 16.9 · judged 25 of 59 gathered run 12 · commander-js · seconds per node = 18.7 run 13 · anthropic-quickstarts · seconds per node = 20.4 · judged 25 of 62 gathered run 14 · tiktoken · seconds per node = 0.1 0* 1* 2* 3* 4* 5* 6* 7* 8 9* 10* 11* 12 13* 14 probe-decided share measured share of decided nodes 0.00 0.50 1.00 run 0 · keel · probe-decided share = 0.00 · judged 25 of 32 gathered run 1 · anthropic-sdk-python · probe-decided share = 0.00 · judged 25 of 41 gathered run 2 · openai-python · probe-decided share = 0.16 · judged 25 of 95 gathered run 3 · vercel-ai · probe-decided share = 0.08 · judged 25 of 1014 gathered run 4 · aider · probe-decided share = 0.56 · judged 25 of 62 gathered run 5 · browser-use · probe-decided share = 0.32 · judged 25 of 104 gathered run 6 · mcp-python-sdk · probe-decided share = 0.00 · judged 25 of 121 gathered run 7 · simonw-llm · probe-decided share = 0.56 · judged 25 of 38 gathered run 8 · tiktoken · probe-decided share = 0.58 run 9 · requests · probe-decided share = 0.28 · judged 25 of 106 gathered run 10 · flask · probe-decided share = 0.00 · judged 25 of 59 gathered run 11 · sinatra · probe-decided share = 0.28 · judged 25 of 59 gathered run 12 · commander-js · probe-decided share = 0.19 run 13 · anthropic-quickstarts · probe-decided share = 0.24 · judged 25 of 62 gathered run 14 · tiktoken · probe-decided share = 1.00 0* 1* 2* 3* 4* 5* 6* 7* 8 9* 10* 11* 12 13* 14 probe library size measured probes in library 0 15 30 run 0 · keel · probe library size = 0 · judged 25 of 32 gathered run 1 · anthropic-sdk-python · probe library size = 3 · judged 25 of 41 gathered run 2 · openai-python · probe library size = 5 · judged 25 of 95 gathered run 3 · vercel-ai · probe library size = 8 · judged 25 of 1014 gathered run 4 · aider · probe library size = 10 · judged 25 of 62 gathered run 5 · browser-use · probe library size = 12 · judged 25 of 104 gathered run 6 · mcp-python-sdk · probe library size = 13 · judged 25 of 121 gathered run 7 · simonw-llm · probe library size = 15 · judged 25 of 38 gathered run 8 · tiktoken · probe library size = 17 run 9 · requests · probe library size = 19 · judged 25 of 106 gathered run 10 · flask · probe library size = 22 · judged 25 of 59 gathered run 11 · sinatra · probe library size = 23 · judged 25 of 59 gathered run 12 · commander-js · probe library size = 26 run 13 · anthropic-quickstarts · probe library size = 29 · judged 25 of 62 gathered run 14 · tiktoken · probe library size = 29 0* 1* 2* 3* 4* 5* 6* 7* 8 9* 10* 11* 12 13* 14 x axis: run index in the recorded corpus order (listed below). Squares are the raw per-run values; the dashed line is an ordinary-least-squares fit and is never shown without them. The fit is clipped to the panel, and withheld entirely (with the panel saying so) where a straight line would predict values the points cannot take — R^2 for every fit is in the trend notes below. * on run 0, 1, 2, 3, 4, 5, 6, 7, 9, 10, 11, 13 = judged fewer nodes than gathered; per-node values are per JUDGED node. Coverage (judged) ci_step 224 · script 64 · test_target 46 · review_gate 10 · deploy_gate 1 Trend estimated tokens per node: falls — total fitted change -66.5% of the mean across 15 runs (R^2 0.28, so the line explains less than half the variance — read the raw squares). raw first 498 -> last 53. seconds per node: falls — total fitted change -72.9% of the mean across 15 runs (R^2 0.04, so the line explains less than half the variance — read the raw squares). raw first 0.1 -> last 0.1. probe-decided share: rises — total fitted change 144.9% of the mean across 15 runs (R^2 0.21, so the line explains less than half the variance — read the raw squares). raw first 0.00 -> last 1.00. probe library size: rises — total fitted change 187.7% of the mean across 15 runs (R^2 0.99). raw first 0 -> last 29. Run order run order derived from Report.generatedAt (ascending, filename as tiebreak) — no order.json manifest present 0 keel -> 1 anthropic-sdk-python -> 2 openai-python -> 3 vercel-ai -> 4 aider -> 5 browser-use -> 6 mcp-python-sdk -> 7 simonw-llm -> 8 tiktoken -> 9 requests -> 10 flask -> 11 sinatra -> 12 commander-js -> 13 anthropic-quickstarts -> 14 tiktoken Shuffle check shuffle check INCOMPLETE — the curve direction ran, the ratio direction did not Permutation (400 draws, seed 20260724): median |delta normalized slope| 0.633, 92% of draws move it by >= 0.15. A permutation reorders ALREADY-RECORDED runs; it cannot reproduce what a genuine re-run in a different order would have cost, because the probe library would have accumulated differently. It is a sanity signal on order-dependence, never a substitute for the empirical re-run. Ratio stability under shuffle was NOT checked: no re-run supplied (--shuffled) and none declared in corpus.meta.json. Permuting recorded runs cannot move a per-target ratio, so the permutation result below says nothing about it. Supply a re-run with --shuffled <dir>. Disclosures - Skipped 3 file(s) that are not usable run reports: corpus-summary.json (no string "target"); curve.json (no string "target"); keel.bindings.json (no "nodes" array). - Run 0 (keel) judged 25 of 32 gathered nodes — a cap. Per-node cost below is per JUDGED node. - Run 1 (anthropic-sdk-python) judged 25 of 41 gathered nodes — a cap. Per-node cost below is per JUDGED node. - Run 2 (openai-python) judged 25 of 95 gathered nodes — a cap. Per-node cost below is per JUDGED node. - Run 3 (vercel-ai) judged 25 of 1014 gathered nodes — a cap. Per-node cost below is per JUDGED node. - Run 4 (aider) judged 25 of 62 gathered nodes — a cap. Per-node cost below is per JUDGED node. - Run 5 (browser-use) judged 25 of 104 gathered nodes — a cap. Per-node cost below is per JUDGED node. - Run 6 (mcp-python-sdk) judged 25 of 121 gathered nodes — a cap. Per-node cost below is per JUDGED node. - Run 7 (simonw-llm) judged 25 of 38 gathered nodes — a cap. Per-node cost below is per JUDGED node. - Run 9 (requests) judged 25 of 106 gathered nodes — a cap. Per-node cost below is per JUDGED node. - Run 10 (flask) judged 25 of 59 gathered nodes — a cap. Per-node cost below is per JUDGED node. - Run 11 (sinatra) judged 25 of 59 gathered nodes — a cap. Per-node cost below is per JUDGED node. - Run 13 (anthropic-quickstarts) judged 25 of 62 gathered nodes — a cap. Per-node cost below is per JUDGED node. - Run 14 (tiktoken): decidedByProbe + decidedByAgent = 10 but nodesSampled = 12. Probe-decided share uses the decided total (10). - Provenance "measured" was declared on the COMMAND LINE (--provenance), not in <repo>/reports/corpus.meta.json. The declaration therefore lives in the invocation and is only as trustworthy as the run sheet that records it. - Token counts are ESTIMATES in 15 of 15 run(s) (RunEconomics.tokensEstimated). No API exposes session token usage to a skill, so the token axis reads "estimated tokens" and nothing here claims a measured token count. Scope. Keel measures the shape of verification, not its quality. A repo can be 100% anchored with terrible tests. Anchoring says the signal comes from outside; it does not say the signal is sufficient.

Run economics

8 decided by probe
17 decided by agent
32% probe-decided share
25 / 104 nodes judged / gathered
4,501 estimated tokens in
4,939 estimated tokens out
9m 04s wall clock
2 probes minted
12 probe library size

Token counts are estimated, and labelled so: a skill running inside an agent session has no API for its own usage, so the figure is ceil(chars/4) over the judgment payloads and responses. Wall clock and probe-decided share are measured directly. 25 of 104 gathered edges were judged — the cap is printed here and beside the ratio, never applied silently.