Keel

Grounding report.

Every verification edge in anthropic-sdk-python, classified by who produces the signal and whether the actor being verified can write to that producer.

targetanthropic-sdk-python
revision60c64fba5c2bf340567f627328e57cf0196b868f
generated2026-07-25T01:35:15.174Z
0.67

anchored / (anchored + self_referential + unknown) = 8 / 12

anchored 8 self_referential 2 unknown 2 not_a_check 13 excluded from the denominator

Coverage (judged)

node kindjudged
ci_step22
test_target2
review_gate1

Nodes by kind, over the 25 node(s) carried in this report. Surfaces the gatherer cannot read are absent here rather than counted as unknown, which makes non-coverage Keel's own shoppable class — so it is stated, not implied.

ε-audit — not run

No verdict in this report carries a readable audit block, so nothing was re-decided and there is no agreement rate to report. This is not agreement, and it is not full agreement either: a rate over zero comparisons would state a measurement that was never taken. (“Readable” is load-bearing: a block can be present and state no comparison, and this state covers that too — see below if so.)

This run carries 0 probe-decided verdict(s), none of them contributing a readable comparison. (“Readable” rather than “audited”, because a block can arrive and state nothing — if any did, they are counted below.) Audit coverage is itself shoppable — audit none of the library and no disagreement can ever be found — so the absence is printed here rather than left to be inferred from a missing section.

This run judged 25 of 41 gathered edges. Every number on this page describes that sample, not the whole surface — 16 gathered edges are absent from all of it. A cap is disclosed rather than smoothed: an undisclosed sample is how a ratio gets shopped.

Scope. Keel measures the shape of verification, not its quality. A repo can be 100% anchored with terrible tests. Anchoring says the signal comes from outside; it does not say the signal is sufficient.

Node graph — 25 node(s), in gather order

anchored 8 self_referential 2 unknown 2 not_a_check 13

2 marks are ringed in the unknown hue: an anchored verdict asserted below confidence 0.6. The ring says the claim is closer to unknown than its colour suggests. Hover a mark for its node.

anchored — 8

The producer sits outside the write boundary of the actor being verified.

pytest config ([tool.pytest.ini_options])
anchored
pyproject.toml:136 · test_target

This table parameterises the suite that CI actually runs: .github/workflows/ci.yml:95 runs ./scripts/test, which ends in `uv run --isolated --all-extras pytest` under `set -e`, so the interpreter executing the collected tests decides the job's exit status. `xfail_strict = true` and `filterwarnings = ["error"]` widen what that process treats as failure, so a committer cannot make a warning-emitting or unexpectedly-passing test report clean without changing code or this pinned config. DOCUMENTED LIMIT: only the EXECUTION axis is anchored. The oracle is not — scripts/test boots a Steady mock on 127.0.0.1:4010 from the OpenAPI spec URL in .stats.yml, and src/anthropic is generated by Stainless from that same spec, so the counterparty the assertions check against is authored inside the same write boundary as the code.

producer the pytest process exit code, under the settings in this table — actor can write to it: no

  • pyproject.toml:136
  • .github/workflows/ci.yml:94-95
  • scripts/test:69 (uv run --isolated --all-extras pytest)
  • scripts/mock:11 (grep 'openapi_spec_url' .stats.yml)

decided by agent · confidence 0.65

linter/type-checker config ([tool.pyright])
anchored
pyproject.toml:149 · ci_step

`typeCheckingMode = "strict"` is consumed by `uv run pyright` at scripts/lint:17, which runs under `set -e` inside the `Run lints` step at .github/workflows/ci.yml:34-35. The non-zero exit comes from pyright performing deterministic static analysis over the committed sources, so an author cannot make a type error report clean without editing the source, adding an explicit ignore comment, or widening the `exclude` list here — all of which are visible diffs.

producer the pyright type-checker process exit code — actor can write to it: no

  • pyproject.toml:149
  • scripts/lint:17
  • .github/workflows/ci.yml:34-35

decided by agent · confidence 0.75

linter/type-checker config ([tool.mypy])
anchored
pyproject.toml:170 · ci_step

These strictness flags are read by `uv run mypy .` at scripts/lint:20, invoked by the gating `Run lints` step (.github/workflows/ci.yml:34). The signal is mypy's own analysis of the committed tree; nothing the SDK code writes at runtime feeds it. LIMIT worth naming: the `exclude` list here removes tests/, several examples and two library modules from the analysed set, so the assertion covers less of the tree than the table's strictness suggests.

producer the mypy process exit code — actor can write to it: no

  • pyproject.toml:170
  • scripts/lint:20
  • .github/workflows/ci.yml:34-35

decided by agent · confidence 0.75

linter/type-checker config ([tool.ruff.lint])
anchored
pyproject.toml:237 · ci_step

This `select` list is the rule set `uv run ruff check .` enforces at scripts/lint:12, under `set -e`, inside the gating `Run lints` step. ruff exits non-zero when the committed source violates one of I / B / F401 / FA102 / E722 / ARG / T201 / T203 / TC004 / TID251; the verdict is produced by the linter binary reading the files, and the CI invocation omits `--fix`, so a violation blocks rather than being silently rewritten. LIMIT: this constrains lint-level style and import hygiene, not behavioural correctness.

producer the ruff process exit code over the selected rule families — actor can write to it: no

  • pyproject.toml:237
  • scripts/lint:12 (uv run ruff check .)
  • .github/workflows/ci.yml:34-35

decided by agent · confidence 0.7

linter/type-checker config ([tool.ruff.lint.flake8-tidy-imports.banned-api])
anchored
pyproject.toml:271 · ci_step

TID251 is present in the `select` list at pyproject.toml:237, so this banned-api entry is live: `ruff check .` in scripts/lint exits non-zero if any committed module imports `functools.lru_cache`. The signal is produced by the linter parsing the source, not by anything the library writes at runtime, and the CI step has no --fix and no `|| true`.

producer the ruff process exit code for rule TID251 — actor can write to it: no

  • pyproject.toml:271
  • pyproject.toml:237 (TID251 selected)
  • scripts/lint:12
  • .github/workflows/ci.yml:34-35

decided by agent · confidence 0.7

linter/type-checker config ([tool.ruff.lint.isort])
anchored
pyproject.toml:274 · ci_step

"I" is selected at pyproject.toml:237, so this table defines the ordering `ruff check .` enforces in the gating lint job; ruff exits non-zero on a mis-sorted import block and CI runs it without --fix. The producer is the linter reading committed files. LIMIT, stated plainly: this is a style constraint only — it can fail, which makes it a check by shape, but it says nothing about behaviour.

producer the ruff process exit code for the isort ("I") rule family — actor can write to it: no

  • pyproject.toml:274
  • pyproject.toml:237 (I selected)
  • scripts/lint:12
  • .github/workflows/ci.yml:34-35

decided by agent · confidence 0.55 · confidence below 0.6 on an anchored verdict

Publish to PyPI
anchored
.github/workflows/create-releases.yml:34 · ci_step

`bash ./bin/publish-pypi` runs under `set -eux` and does `uv build` then `uv publish`. Two signals come from outside the committers' write path: the packaging backend either produces a valid sdist/wheel from the committed tree or exits non-zero, and the PyPI upload endpoint accepts or rejects the artefact (a duplicate version, a malformed metadata block or a bad token yields an HTTP error that fails the job). Neither the SDK source nor the workflow can declare that upload successful. LIMIT, and it is a large one: the assertion is confined to 'this tree packages and the registry accepted it' — it says nothing about whether the released code behaves correctly.

producer the uv build backend's exit code and PyPI's upload response — actor can write to it: no

  • .github/workflows/create-releases.yml:34-37
  • bin/publish-pypi:3-9 (set -eux; uv build; uv publish)

decided by agent · confidence 0.55 · confidence below 0.6 on an anchored verdict

Run lints
anchored
.github/workflows/ci.yml:34 · ci_step

I descended into ./scripts/lint: it sets `set -e` and then runs `uv run ruff check .`, `uv run pyright`, `uv run mypy .` and `uv run python -c 'import anthropic'`. Every one of those exit codes is produced by a separate process performing deterministic analysis of — or in the last case actually importing — the committed source, and the first non-zero one terminates the script and fails the job. An author cannot make a type error, a lint violation or an import-time exception report clean without changing the source or the pinned tool config in pyproject.toml. The `import anthropic` line is the strongest of the four: the interpreter really executes module initialisation. Noted, not class-changing: the job's `if:` at ci.yml:22 skips it for same-repo pull_request events, but the same commits run it on push.

producer the exit codes of ruff, pyright, mypy and the CPython interpreter, aggregated by `set -e` — actor can write to it: no

  • .github/workflows/ci.yml:34-35
  • scripts/lint:3 (set -e)
  • scripts/lint:12,17,20,23
  • .github/workflows/ci.yml:22 (job if:)

decided by agent · confidence 0.85

self_referential — 2

The producer sits inside the write boundary of the actor being verified.

pytest config (conftest.py)
self_referential
tests/conftest.py · test_target

This conftest defines what the suite compares against. `base_url = os.environ.get("TEST_API_BASE_URL", "http://127.0.0.1:4010")` points every `client`/`async_client` fixture at the local Steady mock that scripts/mock boots from `openapi_spec_url` in .stats.yml — the same OpenAPI spec Stainless uses to generate src/anthropic, so the counterparty the assertions verify against is derived from the very artefact under test. The `snapshot_client` / `async_snapshot_client` fixtures go further: they wrap the client in `HttpxSyncSnapshotClient(http_snapshot, is_recording, ...)`, so when `is_recording` is on, the expected request and response payloads are recordings of the client's own traffic — the code authors its own expected output. `pytest_collection_modifyitems` additionally adds `pytest.mark.skip` markers, which can only remove assertions.

producer a Steady mock server generated from the OpenAPI spec, plus inline_snapshot recordings of the client's own HTTP traffic — actor can write to it: yes

  • tests/conftest.py:58-60 (base_url / api_key)
  • tests/conftest.py:78-92 (snapshot_client, is_recording)
  • tests/conftest.py:37-56 (pytest_collection_modifyitems adds skip markers)
  • scripts/mock:11
  • tests/conftest.py:1 ("File generated from our OpenAPI spec by Stainless")

decided by agent · confidence 0.6

Run Claude Code
self_referential
.github/workflows/claude.yml:38 · ci_step

The step hands the checked-out repository to an LLM and lets it answer @claude mentions on issues, PRs and reviews. Any assessment it renders is produced by the same class of system that produces the code under discussion, and here the coupling is literal rather than analogical: the job holds `contents: write` and `pull-requests: write`, and `allowed_tools` explicitly permits `git commit`, `git push`, `git merge` and `gh pr`, so the reviewer can write into the branch whose state it is reporting on. It is also mention-triggered (workflow `if:` at line 15-19), so it is not on the merge path at all.

producer a large language model invoked through anthropics/claude-code-action — actor can write to it: yes

  • .github/workflows/claude.yml:38-52
  • .github/workflows/claude.yml:21-26 (contents: write, pull-requests: write)
  • .github/workflows/claude.yml:15-19 (triggered by @claude mention, not by merge)

decided by agent · confidence 0.85

unknown — 2

The fork point could not be established. Fails closed — counts against the ratio exactly like self_referential.

code owners (CODEOWNERS)
unknown
.github/CODEOWNERS · review_gate

The file assigns @anthropics/sdk as owner of every path, and its own header comment says it is 'used to automatically assign reviewers to PRs'. Whether that assignment BLOCKS a merge depends on a 'Require review from Code Owners' branch-protection rule, which lives in GitHub repository settings and is not present anywhere in the clone — so I cannot establish whether this edge gates anything at all, let alone trace its fork point. Even if it does gate, the reviewing team is the same team that owns the generated source, so the direction of the boundary would still need argument rather than assumption.

producer GitHub's branch-protection engine, whose configuration is not in the repository — actor can write to it: not established

  • .github/CODEOWNERS:4
  • no branch-protection config present in the repository tree

decided by agent · confidence 0.75

stainless-api/trigger-release-please@bb6677c5a04578eec1ccfd9e1913b5b78ed64c61 # v1.4.0
unknown
.github/workflows/create-releases.yml:22 · ci_step

The step's exit code gates the whole production-release job and its `releases_created` output gates the PyPI publish at line 35, but the action's body lives in another repository — pinned by SHA, yet not vendored here, so I cannot descend into it to see whether it validates anything (version consistency, changelog, tag state) or merely calls an API. I will not guess a class for a wrapper I cannot open. Compounding it: the operator of that API, Stainless, is also the generator of the source it would be releasing (tests/conftest.py:1, .stats.yml), so even the direction of the write boundary is not clean.

producer the stainless-api/trigger-release-please action, whose implementation is not in this repository — actor can write to it: not established

  • .github/workflows/create-releases.yml:22-26
  • .github/workflows/create-releases.yml:29,35 (steps.release.outputs.releases_created gates later steps)
  • action implementation not present in the clone

decided by agent · confidence 0.6

not_a_check — 13

Asserts nothing about correctness, so it is excluded from the denominator. This is the one shoppable class: mis-filing a real check here shrinks the denominator and inflates the score, so each row carries the same burden of argument as any other verdict.

linter/type-checker config ([tool.ruff])
not_a_check
pyproject.toml:229 · ci_step

`line-length`, `output-format` and `target-version` emit no pass/fail signal. line-length would only gate through rule E501, which is absent from the `select` list at pyproject.toml:237, and `ruff format` (the only consumer of line-length here) is never invoked by the CI path: scripts/lint runs `ruff check`, `pyright`, `mypy` and an import, and scripts/format is not referenced by any workflow. `output-format = "grouped"` only changes how ruff prints. Nothing in this table can cause a job to fail.

producer nothing — this table is inert configuration — actor can write to it: not established

  • pyproject.toml:229
  • pyproject.toml:237 (select list has no E501)
  • scripts/lint:1-22
  • .github/workflows/ci.yml

decided by agent · confidence 0.65

linter/type-checker config ([tool.ruff.format])
not_a_check
pyproject.toml:234 · ci_step

`docstring-code-format = true` only alters how `ruff format` rewrites docstring code blocks, and `ruff format` is invoked nowhere in the pipeline path — scripts/lint runs `ruff check` (not `format`), and scripts/format is not called by any workflow in .github/workflows/. A formatter that only rewrites, and is not even run under a --check flag in CI, asserts nothing about correctness.

producer the ruff formatter, which rewrites files rather than judging them — actor can write to it: not established

  • pyproject.toml:234
  • scripts/lint:9-13
  • scripts/format (not referenced by .github/workflows/*)

decided by agent · confidence 0.8

linter/type-checker config ([tool.ruff.lint.per-file-ignores])
not_a_check
pyproject.toml:281 · ci_step

Every entry here removes rules T201/T203 (print statements) from bin/, scripts/, tests/ and examples/. A suppression list is monotonically failure-reducing: it can turn a ruff failure into a pass but can never produce a non-zero exit of its own. It asserts nothing about correctness and cannot fail.

producer nothing — this table only suppresses diagnostics — actor can write to it: not established

  • pyproject.toml:281
  • scripts/lint:12

decided by agent · confidence 0.8

actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
not_a_check
.github/workflows/create-releases.yml:20 · ci_step

Pure provisioning: it materialises the repository into the runner workspace so later steps have files to act on. It emits no verdict about the code — a failure here means the runner could not fetch, not that anything is wrong with the artefact.

producer the checkout action's git clone — actor can write to it: not established

  • .github/workflows/create-releases.yml:20

decided by agent · confidence 0.95

Install uv
not_a_check
.github/workflows/create-releases.yml:28 · ci_step

Provisioning of the uv runtime at a pinned version so later steps can build and publish. It installs; it evaluates nothing about the repository's contents, and it is additionally gated behind `if: steps.release.outputs.releases_created` so it often does not run at all.

producer the setup-uv action installing a toolchain binary — actor can write to it: not established

  • .github/workflows/create-releases.yml:28-32

decided by agent · confidence 0.95

Checkout repository
not_a_check
.github/workflows/claude.yml:28 · ci_step

Provisioning only — it fetches the repository (with fetch-depth: 0 so the agent step has history). No property of the code is evaluated and no verdict is emitted.

producer the checkout action's git clone — actor can write to it: not established

  • .github/workflows/claude.yml:28-31

decided by agent · confidence 0.95

Configure git
not_a_check
.github/workflows/claude.yml:33 · ci_step

It sets user.name and user.email so the agent's later commits are attributable. Setting an identity is environment setup; it reads nothing about the repository and can only fail if git itself is missing.

producer git config writing two local identity values — actor can write to it: not established

  • .github/workflows/claude.yml:33-36

decided by agent · confidence 0.95

actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
not_a_check
.github/workflows/ci.yml:24 · ci_step

First step of the lint job: it materialises the tree for the steps that follow. Provisioning emits no correctness signal — the check in this job is `Run lints` at line 34.

producer the checkout action's git clone — actor can write to it: not established

  • .github/workflows/ci.yml:24
  • .github/workflows/ci.yml:34-35

decided by agent · confidence 0.95

Install uv
not_a_check
.github/workflows/ci.yml:26 · ci_step

Installs uv 0.10.2 on the runner so `uv sync` and `uv run` work in later steps. It provisions a runtime and evaluates nothing about the repository.

producer the setup-uv action installing a toolchain binary — actor can write to it: not established

  • .github/workflows/ci.yml:26-29

decided by agent · confidence 0.95

Install dependencies
not_a_check
.github/workflows/ci.yml:31 · ci_step

`uv sync --all-extras` carries neither `--locked` nor `--frozen`, so uv will re-resolve and UPDATE uv.lock when it is stale rather than erroring — the step therefore does not assert lockfile consistency, and there is no later step that diffs the lockfile. What remains is environment provisioning for the `Run lints` step that follows.

producer uv's dependency resolver populating a virtualenv — actor can write to it: not established

  • .github/workflows/ci.yml:31-32
  • .github/workflows/ci.yml:34-35

decided by agent · confidence 0.7

actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
not_a_check
.github/workflows/ci.yml:46 · ci_step

First step of the build job — it fetches the tree so `uv build` at line 57 has something to package. Provisioning; it emits no verdict about the code.

producer the checkout action's git clone — actor can write to it: not established

  • .github/workflows/ci.yml:46
  • .github/workflows/ci.yml:56-57

decided by agent · confidence 0.95

Install uv
not_a_check
.github/workflows/ci.yml:48 · ci_step

Installs the uv runtime on the build job's runner. Toolchain provisioning; nothing about the repository's contents is evaluated here.

producer the setup-uv action installing a toolchain binary — actor can write to it: not established

  • .github/workflows/ci.yml:48-51

decided by agent · confidence 0.95

Install dependencies
not_a_check
.github/workflows/ci.yml:53 · ci_step

Same shape as the lint job's install: `uv sync --all-extras` with no `--locked`/`--frozen`, so it will silently refresh a stale uv.lock instead of failing, and no subsequent step compares the lockfile. It provisions the environment for `uv build` at line 57; the assertion in this job lives there, not here.

producer uv's dependency resolver populating a virtualenv — actor can write to it: not established

  • .github/workflows/ci.yml:53-54
  • .github/workflows/ci.yml:56-57

decided by agent · confidence 0.7

Crystallization curve

Keel crystallization curve Crystallization curve over 15 runs from <repo>/reports. falls — total fitted change -66.5% of the mean across 15 runs (R^2 0.28, so the line explains less than half the variance — read the raw squares) CRYSTALLIZATION CURVE 15 sequential runs · 345 nodes judged of 1838 gathered · 124 anchored Keel corpus — 15 repositories, 2026-07-24 — measured corpus (declared on the command line) estimated tokens per node estimated tokens / judged node 0 300 600 run 0 · keel · estimated tokens per node = 498 · judged 25 of 32 gathered run 1 · anthropic-sdk-python · estimated tokens per node = 369 · judged 25 of 41 gathered run 2 · openai-python · estimated tokens per node = 398 · judged 25 of 95 gathered run 3 · vercel-ai · estimated tokens per node = 410 · judged 25 of 1014 gathered run 4 · aider · estimated tokens per node = 204 · judged 25 of 62 gathered run 5 · browser-use · estimated tokens per node = 378 · judged 25 of 104 gathered run 6 · mcp-python-sdk · estimated tokens per node = 592 · judged 25 of 121 gathered run 7 · simonw-llm · estimated tokens per node = 189 · judged 25 of 38 gathered run 8 · tiktoken · estimated tokens per node = 240 run 9 · requests · estimated tokens per node = 290 · judged 25 of 106 gathered run 10 · flask · estimated tokens per node = 408 · judged 25 of 59 gathered run 11 · sinatra · estimated tokens per node = 265 · judged 25 of 59 gathered run 12 · commander-js · estimated tokens per node = 316 run 13 · anthropic-quickstarts · estimated tokens per node = 310 · judged 25 of 62 gathered run 14 · tiktoken · estimated tokens per node = 53 0* 1* 2* 3* 4* 5* 6* 7* 8 9* 10* 11* 12 13* 14 seconds per node measured s / judged node 0.0 60.0 120.0 run 0 · keel · seconds per node = 0.1 · judged 25 of 32 gathered run 1 · anthropic-sdk-python · seconds per node = 15.8 · judged 25 of 41 gathered run 2 · openai-python · seconds per node = 15.4 · judged 25 of 95 gathered run 3 · vercel-ai · seconds per node = 105.9 · judged 25 of 1014 gathered run 4 · aider · seconds per node = 21.3 · judged 25 of 62 gathered run 5 · browser-use · seconds per node = 21.7 · judged 25 of 104 gathered run 6 · mcp-python-sdk · seconds per node = 20.1 · judged 25 of 121 gathered run 7 · simonw-llm · seconds per node = 11.8 · judged 25 of 38 gathered run 8 · tiktoken · seconds per node = 23.7 run 9 · requests · seconds per node = 16.8 · judged 25 of 106 gathered run 10 · flask · seconds per node = 20.5 · judged 25 of 59 gathered run 11 · sinatra · seconds per node = 16.9 · judged 25 of 59 gathered run 12 · commander-js · seconds per node = 18.7 run 13 · anthropic-quickstarts · seconds per node = 20.4 · judged 25 of 62 gathered run 14 · tiktoken · seconds per node = 0.1 0* 1* 2* 3* 4* 5* 6* 7* 8 9* 10* 11* 12 13* 14 probe-decided share measured share of decided nodes 0.00 0.50 1.00 run 0 · keel · probe-decided share = 0.00 · judged 25 of 32 gathered run 1 · anthropic-sdk-python · probe-decided share = 0.00 · judged 25 of 41 gathered run 2 · openai-python · probe-decided share = 0.16 · judged 25 of 95 gathered run 3 · vercel-ai · probe-decided share = 0.08 · judged 25 of 1014 gathered run 4 · aider · probe-decided share = 0.56 · judged 25 of 62 gathered run 5 · browser-use · probe-decided share = 0.32 · judged 25 of 104 gathered run 6 · mcp-python-sdk · probe-decided share = 0.00 · judged 25 of 121 gathered run 7 · simonw-llm · probe-decided share = 0.56 · judged 25 of 38 gathered run 8 · tiktoken · probe-decided share = 0.58 run 9 · requests · probe-decided share = 0.28 · judged 25 of 106 gathered run 10 · flask · probe-decided share = 0.00 · judged 25 of 59 gathered run 11 · sinatra · probe-decided share = 0.28 · judged 25 of 59 gathered run 12 · commander-js · probe-decided share = 0.19 run 13 · anthropic-quickstarts · probe-decided share = 0.24 · judged 25 of 62 gathered run 14 · tiktoken · probe-decided share = 1.00 0* 1* 2* 3* 4* 5* 6* 7* 8 9* 10* 11* 12 13* 14 probe library size measured probes in library 0 15 30 run 0 · keel · probe library size = 0 · judged 25 of 32 gathered run 1 · anthropic-sdk-python · probe library size = 3 · judged 25 of 41 gathered run 2 · openai-python · probe library size = 5 · judged 25 of 95 gathered run 3 · vercel-ai · probe library size = 8 · judged 25 of 1014 gathered run 4 · aider · probe library size = 10 · judged 25 of 62 gathered run 5 · browser-use · probe library size = 12 · judged 25 of 104 gathered run 6 · mcp-python-sdk · probe library size = 13 · judged 25 of 121 gathered run 7 · simonw-llm · probe library size = 15 · judged 25 of 38 gathered run 8 · tiktoken · probe library size = 17 run 9 · requests · probe library size = 19 · judged 25 of 106 gathered run 10 · flask · probe library size = 22 · judged 25 of 59 gathered run 11 · sinatra · probe library size = 23 · judged 25 of 59 gathered run 12 · commander-js · probe library size = 26 run 13 · anthropic-quickstarts · probe library size = 29 · judged 25 of 62 gathered run 14 · tiktoken · probe library size = 29 0* 1* 2* 3* 4* 5* 6* 7* 8 9* 10* 11* 12 13* 14 x axis: run index in the recorded corpus order (listed below). Squares are the raw per-run values; the dashed line is an ordinary-least-squares fit and is never shown without them. The fit is clipped to the panel, and withheld entirely (with the panel saying so) where a straight line would predict values the points cannot take — R^2 for every fit is in the trend notes below. * on run 0, 1, 2, 3, 4, 5, 6, 7, 9, 10, 11, 13 = judged fewer nodes than gathered; per-node values are per JUDGED node. Coverage (judged) ci_step 224 · script 64 · test_target 46 · review_gate 10 · deploy_gate 1 Trend estimated tokens per node: falls — total fitted change -66.5% of the mean across 15 runs (R^2 0.28, so the line explains less than half the variance — read the raw squares). raw first 498 -> last 53. seconds per node: falls — total fitted change -72.9% of the mean across 15 runs (R^2 0.04, so the line explains less than half the variance — read the raw squares). raw first 0.1 -> last 0.1. probe-decided share: rises — total fitted change 144.9% of the mean across 15 runs (R^2 0.21, so the line explains less than half the variance — read the raw squares). raw first 0.00 -> last 1.00. probe library size: rises — total fitted change 187.7% of the mean across 15 runs (R^2 0.99). raw first 0 -> last 29. Run order run order derived from Report.generatedAt (ascending, filename as tiebreak) — no order.json manifest present 0 keel -> 1 anthropic-sdk-python -> 2 openai-python -> 3 vercel-ai -> 4 aider -> 5 browser-use -> 6 mcp-python-sdk -> 7 simonw-llm -> 8 tiktoken -> 9 requests -> 10 flask -> 11 sinatra -> 12 commander-js -> 13 anthropic-quickstarts -> 14 tiktoken Shuffle check shuffle check INCOMPLETE — the curve direction ran, the ratio direction did not Permutation (400 draws, seed 20260724): median |delta normalized slope| 0.633, 92% of draws move it by >= 0.15. A permutation reorders ALREADY-RECORDED runs; it cannot reproduce what a genuine re-run in a different order would have cost, because the probe library would have accumulated differently. It is a sanity signal on order-dependence, never a substitute for the empirical re-run. Ratio stability under shuffle was NOT checked: no re-run supplied (--shuffled) and none declared in corpus.meta.json. Permuting recorded runs cannot move a per-target ratio, so the permutation result below says nothing about it. Supply a re-run with --shuffled <dir>. Disclosures - Skipped 3 file(s) that are not usable run reports: corpus-summary.json (no string "target"); curve.json (no string "target"); keel.bindings.json (no "nodes" array). - Run 0 (keel) judged 25 of 32 gathered nodes — a cap. Per-node cost below is per JUDGED node. - Run 1 (anthropic-sdk-python) judged 25 of 41 gathered nodes — a cap. Per-node cost below is per JUDGED node. - Run 2 (openai-python) judged 25 of 95 gathered nodes — a cap. Per-node cost below is per JUDGED node. - Run 3 (vercel-ai) judged 25 of 1014 gathered nodes — a cap. Per-node cost below is per JUDGED node. - Run 4 (aider) judged 25 of 62 gathered nodes — a cap. Per-node cost below is per JUDGED node. - Run 5 (browser-use) judged 25 of 104 gathered nodes — a cap. Per-node cost below is per JUDGED node. - Run 6 (mcp-python-sdk) judged 25 of 121 gathered nodes — a cap. Per-node cost below is per JUDGED node. - Run 7 (simonw-llm) judged 25 of 38 gathered nodes — a cap. Per-node cost below is per JUDGED node. - Run 9 (requests) judged 25 of 106 gathered nodes — a cap. Per-node cost below is per JUDGED node. - Run 10 (flask) judged 25 of 59 gathered nodes — a cap. Per-node cost below is per JUDGED node. - Run 11 (sinatra) judged 25 of 59 gathered nodes — a cap. Per-node cost below is per JUDGED node. - Run 13 (anthropic-quickstarts) judged 25 of 62 gathered nodes — a cap. Per-node cost below is per JUDGED node. - Run 14 (tiktoken): decidedByProbe + decidedByAgent = 10 but nodesSampled = 12. Probe-decided share uses the decided total (10). - Provenance "measured" was declared on the COMMAND LINE (--provenance), not in <repo>/reports/corpus.meta.json. The declaration therefore lives in the invocation and is only as trustworthy as the run sheet that records it. - Token counts are ESTIMATES in 15 of 15 run(s) (RunEconomics.tokensEstimated). No API exposes session token usage to a skill, so the token axis reads "estimated tokens" and nothing here claims a measured token count. Scope. Keel measures the shape of verification, not its quality. A repo can be 100% anchored with terrible tests. Anchoring says the signal comes from outside; it does not say the signal is sufficient.

Run economics

0 decided by probe
25 decided by agent
0% probe-decided share
25 / 41 nodes judged / gathered
3,990 estimated tokens in
5,233 estimated tokens out
6m 35s wall clock
3 probes minted
3 probe library size

Token counts are estimated, and labelled so: a skill running inside an agent session has no API for its own usage, so the figure is ceil(chars/4) over the judgment payloads and responses. Wall clock and probe-decided share are measured directly. 25 of 41 gathered edges were judged — the cap is printed here and beside the ratio, never applied silently.