Keel

Grounding report.

Every verification edge in vercel-ai, classified by who produces the signal and whether the actor being verified can write to that producer.

targetvercel-ai
revisione29788dd545f8bf2300db0885658e639b4fd91bd
generated2026-07-25T02:26:49.454Z
0.80

anchored / (anchored + self_referential + unknown) = 8 / 10

anchored 8 self_referential 0 unknown 2 not_a_check 15 excluded from the denominator

Coverage (judged)

node kindjudged
ci_step8
script8
test_target8
deploy_gate1

Nodes by kind, over the 25 node(s) carried in this report. Surfaces the gatherer cannot read are absent here rather than counted as unknown, which makes non-coverage Keel's own shoppable class — so it is stated, not implied.

ε-audit — not run

No verdict in this report carries a readable audit block, so nothing was re-decided and there is no agreement rate to report. This is not agreement, and it is not full agreement either: a rate over zero comparisons would state a measurement that was never taken. (“Readable” is load-bearing: a block can be present and state no comparison, and this state covers that too — see below if so.)

This run carries 2 probe-decided verdict(s), none of them contributing a readable comparison. (“Readable” rather than “audited”, because a block can arrive and state nothing — if any did, they are counted below.) Audit coverage is itself shoppable — audit none of the library and no disagreement can ever be found — so the absence is printed here rather than left to be inferred from a missing section.

This run judged 25 of 1,014 gathered edges. Every number on this page describes that sample, not the whole surface — 989 gathered edges are absent from all of it. A cap is disclosed rather than smoothed: an undisclosed sample is how a ratio gets shopped.

Scope. Keel measures the shape of verification, not its quality. A repo can be 100% anchored with terrible tests. Anchoring says the signal comes from outside; it does not say the signal is sufficient.

Node graph — 25 node(s), in gather order

anchored 8 unknown 2 not_a_check 15

1 mark is ringed in the unknown hue: an anchored verdict asserted below confidence 0.6. The ring says the claim is closer to unknown than its colour suggests. Hover a mark for its node.

anchored — 8

The producer sits outside the write boundary of the actor being verified.

test runner config (vitest.config.js)
anchored
packages/vue/vitest.config.js · test_target

ci.yml:313 runs `pnpm test:ci` = `turbo test --filter=!@example/* --filter=!ai --filter=!@ai-sdk/codemod --only`, which reaches @ai-sdk/vue's `test` script = `vitest --config vitest.config.js --run`; this file is the config that script names, so its `include` glob and jsdom environment determine which assertions execute and the runner's exit code fails the 'Test' job, which the branch-protection aggregate job `test` (ci.yml:511-520) hard-depends on. DOCUMENTED LIMIT, on two axes: the include glob `src/**/*.ui.test.ts{,x}` is authored inside the write boundary, so narrowing it silences tests without any runtime noticing, and the assertions themselves are co-authored with the implementation — the execution axis is what is anchored here, not the oracle.

producer the vitest process exit code, decided by a jsdom-hosted runtime executing the committed Vue component tests — actor can write to it: no

  • .github/workflows/ci.yml:313 (run: pnpm test:ci)
  • package.json:17 (test:ci turbo filter chain; @ai-sdk/vue not excluded)
  • packages/vue/package.json scripts.test = 'vitest --config vitest.config.js --run'
  • .github/workflows/ci.yml:511-520 (required aggregate job 'test' needs test_matrix)

decided by agent · confidence 0.78

test runner config (vitest.config.ts)
anchored
packages/angular/vitest.config.ts · test_target

Same causal path as the Vue config: ci.yml:313 `pnpm test:ci` fans out through turbo to @ai-sdk/angular's `test` = `vitest --config vitest.config.ts --run`, and this file supplies the environment and the `src/lib/**/*.test.ts` include set the runner uses; a thrown assertion makes the interpreter exit non-zero and reddens the Test job feeding the required aggregate. DOCUMENTED LIMIT: execution is decided outside the authors' reach, but the include glob and the expectations are both committed artefacts, so this measures that the code does what the tests say, not that the tests say the right thing.

producer the vitest process exit code, decided by a jsdom-hosted runtime executing the committed Angular tests — actor can write to it: no

  • .github/workflows/ci.yml:313 (run: pnpm test:ci)
  • packages/angular/package.json scripts.test = 'vitest --config vitest.config.ts --run'
  • packages/angular/vitest.config.ts (environment jsdom, include src/lib/**/*.test.ts)

decided by agent · confidence 0.76

test runner config (vitest.config.js)
anchored
packages/ai/vitest.config.js · test_target

This file exports createVitestConfig(environment); vitest.node.config.js and vitest.edge.config.js are two-line files that call it, and ci.yml:479-480 runs `pnpm test:node --shard` and `pnpm test:edge --shard` inside packages/ai across a 3-Node x 4-shard matrix. So the include/exclude sets and `typecheck: { enabled: true }` declared here decide what the runtime executes and whether the tsc pass runs, and the resulting exit code reddens the 'Test AI' jobs. The typecheck flag is the strongest part: it puts a compiler, not an assertion, in the path. DOCUMENTED LIMIT: the exclusion list (`**/*.ui.test.ts{,x}`, `**/*.e2e.test.ts{,x}`) is author-controlled, and the assertions are co-authored with the implementation, so the anchoring is on execution, not on the specification.

producer the vitest process exit code for the core `ai` package, plus vitest's typecheck pass, both decided by execution over the committed sources — actor can write to it: no

  • packages/ai/vitest.node.config.js and vitest.edge.config.js (import { createVitestConfig } from './vitest.config.js')
  • .github/workflows/ci.yml:475-480 (test_ai_matrix: pnpm test:node --shard / pnpm test:edge --shard)
  • packages/ai/vitest.config.js typecheck.enabled = true

decided by agent · confidence 0.82

test runner config (playwright.config.ts)
anchored
packages/rsc/playwright.config.ts · test_target

ci.yml:445-448 runs `pnpm test:e2e` with `working-directory: packages/rsc`, which is `playwright test` reading this config; the config boots `pnpm run dev` in tests/e2e/next-server, waits on http://localhost:3000 with `reuseExistingServer: false`, and the specs then click through the page and assert on rendered text (tests/e2e/spec/streamable.e2e.test.ts). The verdict comes from a browser observing a server process, neither of which an author can persuade — a broken stream renders wrong text and the run goes red. TWO DOCUMENTED LIMITS: `retries: 2` means only consistently-failing behaviour reddens the job, and the whole step is guarded by `if: steps.rsc-e2e.outputs.run == 'true'`, a git-diff-derived flag, so on PRs that do not touch packages/rsc this edge does not execute at all.

producer Chromium driven by Playwright against a real Next.js dev server, with the browser's observed DOM deciding pass/fail — actor can write to it: no

  • .github/workflows/ci.yml:445-448 (Run RSC e2e tests, working-directory packages/rsc, pnpm test:e2e)
  • packages/rsc/package.json scripts.test:e2e = 'playwright test'
  • packages/rsc/playwright.config.ts (webServer command 'pnpm run dev', retries: 2, reuseExistingServer: false)
  • packages/rsc/tests/e2e/spec/streamable.e2e.test.ts:3-10 (page.goto + expect(logs).toHaveText)
  • .github/workflows/ci.yml:400-415 ('Detect RSC e2e changes' sets run=true/false from git diff)

decided by agent · confidence 0.8

test runner config (vitest.config.js)
anchored
packages/react/vitest.config.js · test_target

ci.yml:313 `pnpm test:ci` reaches @ai-sdk/react (it is not in the exclusion filters) and runs its `test` script `vitest --config vitest.config.js --run`, which loads this file; the jsdom environment, the `src/**/*.test.ts{,x}` include and the `./src/vitest-setup.ts` setup file it declares determine what executes, and the runner's exit code propagates into the Test job that the required aggregate `test` job depends on. DOCUMENTED LIMIT: the setup file and the assertions live inside the same repository as the implementation, so what is anchored is that the runtime honestly decided pass/fail, not that the expectations are correct.

producer the vitest process exit code, decided by a jsdom-hosted runtime executing the committed React hook tests — actor can write to it: no

  • .github/workflows/ci.yml:313 (run: pnpm test:ci)
  • package.json:17 (test:ci filters exclude only @example/*, ai, @ai-sdk/codemod)
  • packages/react/package.json scripts.test = 'vitest --config vitest.config.js --run'
  • packages/react/vitest.config.js (environment jsdom, setupFiles ./src/vitest-setup.ts)

decided by agent · confidence 0.78

test runner config (vitest.config.ts)
anchored
packages/codemod/vitest.config.ts · test_target

ci.yml:507 names this file directly — `pnpm exec vitest --config vitest.config.ts --run --shard ${{ matrix.shard }}` in packages/codemod across a 3-Node x 4-shard matrix, and that job is one of the `needs` of the required aggregate `test` job (ci.yml:511-520). The transform actually runs (test-utils.ts applyTransform) and the comparison is `expect(actualOutput).toBe(expectedOutput)`, so the interpreter decides. DOCUMENTED LIMIT, and it is the sharp one: the expected output is a committed golden file that `pnpm test:update` (`UPDATE_SNAPSHOT=true vitest -u --run`) rewrites from the transformer's own output — test-utils.ts:156-165 branches on that env var and writes the fixture instead of asserting. CI does not set UPDATE_SNAPSHOT, so the comparison genuinely executes, but the oracle was authored by the thing under test and can be re-authored by one documented command. Execution anchored; specification inside the boundary.

producer the vitest process exit code, decided by executing each codemod transformer over its input fixture and comparing the produced text to the committed output fixture — actor can write to it: no

  • .github/workflows/ci.yml:507 (pnpm exec vitest --config vitest.config.ts --run --shard)
  • .github/workflows/ci.yml:511-520 (aggregate 'test' job needs test_codemod_matrix)
  • packages/codemod/src/test/test-utils.ts:150-168 (applyTransform, UPDATE_SNAPSHOT branch, expect(actualOutput).toBe(expectedOutput))
  • packages/codemod/package.json scripts.test:update = 'UPDATE_SNAPSHOT=true vitest -u --run'

decided by agent · confidence 0.82

Install Dependencies
anchored
.github/workflows/update-model-settings.yml:64 · ci_step

`--frozen-lockfile` is what makes this different from a plain install: pnpm will not refresh a stale lock, it exits non-zero, so the step is a genuine falsifiable assertion about two committed files agreeing — an author who edits a dependency and forgets to regenerate the lock cannot make this pass without changing the tree. The producer is the resolver executing, not anyone's claim. The exit code gates the rest of the job: the 'Generate model settings' and PR-creation steps below only run if this succeeds. DOCUMENTED LIMIT: what is anchored is manifest/lock agreement and registry reachability, nothing about whether the code works; and this instance sits in a scheduled bot workflow, so it gates that workflow's PR creation rather than a merge (the same command in ci.yml is the merge-gating instance).

producer pnpm's resolver comparing the committed pnpm-lock.yaml against the committed package.json manifests, and refusing to proceed on a mismatch — actor can write to it: no

  • .github/workflows/update-model-settings.yml:64-65 (run: pnpm install --frozen-lockfile)
  • .github/workflows/update-model-settings.yml:67-71 ('Generate model settings' runs after it)
  • .github/workflows/ci.yml:31, :57, :100, :126 (same frozen install on the merge-gating path)

decided by agent · confidence 0.72

deploy config (vercel.json)
anchored
apps/docs/vercel.json · deploy_gate

The file's only substantive key sets the command Vercel runs to produce the deployment, and Vercel promotes a deployment only when that command exits zero — so the promotion condition is the Next.js compiler's verdict on the committed docs tree, produced on infrastructure the repository's authors do not write to. A broken MDX import or a type error surfaced by next build cannot be argued into a successful deploy. TWO LIMITS, and they are why confidence is low rather than the class being upgraded: the signal says only that the site builds, nothing about whether the docs are correct; and this tree contains no evidence about whether the resulting deployment is required for a merge, so the strength of the gate is asserted from Vercel's platform behaviour rather than from anything I could read here.

producer Vercel's build infrastructure executing `pnpm build:site` (sync-content -> fumadocs-mdx -> next build) on the deployment runner — actor can write to it: no

  • apps/docs/vercel.json (buildCommand: pnpm build:site)
  • apps/docs/package.json scripts.build:site = 'pnpm sync-content && fumadocs-mdx && next build'
  • .github/workflows/ci.yml:179 (separate merge-gating docs job: pnpm --filter ai-sdk-docs validate:site)

decided by agent · confidence 0.45 · confidence below 0.6 on an anchored verdict

unknown — 2

The fork point could not be established. Fails closed — counts against the ratio exactly like self_referential.

build
unknown
tools/konsistent-provider/package.json · script

This step IS on the merge-gating path: ci.yml:62 runs `pnpm konsistent`, which is `turbo run konsistent:validate konsistent:check`, and both of those tasks declare `dependsOn: ["konsistent-provider#build"]` (turbo.json:102-108), so a non-zero exit here reddens the required 'Code Consistency' job. What I cannot establish is what the exit code means: `konsistent-convention emit` is a binary from @konsistent/convention ^1.0.0-beta, which is not vendored in the tree and which I cannot descend into from this clone, so I cannot tell whether it validates the conventions declared in src/index.ts (a real property of committed source) or merely serialises them to dist/conventions.json (pure generation). Guessing either way would be inventing the fork point.

producer the `konsistent-convention` CLI from the third-party @konsistent/convention package, executing on the runner — actor can write to it: not established

  • .github/workflows/ci.yml:62 (run: pnpm konsistent)
  • package.json:31 (konsistent: turbo run konsistent:validate konsistent:check)
  • turbo.json:102-108 (//#konsistent:check and //#konsistent:validate dependsOn konsistent-provider#build)
  • tools/konsistent-provider/package.json dependencies: @konsistent/convention ^1.0.0-beta (implementation not in tree)

decided by agent · confidence 0.62

build
unknown
package.json · script

This is an aggregate wrapper, and the rule for aggregates is to descend rather than classify the wrapper. Descending does not resolve it: the children are heterogeneous, several are third-party generators I already could not trace (konsistent-convention emit), and the dominant child, tsup, drives esbuild, which strips types without checking them — so a green `turbo build` does not even mean the tree type-checks. Its one pipeline caller is release.yml:122 (`pnpm clean && pnpm build`) in the snapshot-publish branch, not the merge gate. I can name the producer but not establish, uniformly across the children, whether the signal reports a property of the committed source or only that artifacts were emitted.

producer turbo, whose exit code is the maximum over ~70 per-package build commands (tsup/esbuild, vite build, next build, konsistent-convention emit, ...) — actor can write to it: not established

  • package.json:4 (build: turbo build --concurrency 16)
  • .github/workflows/release.yml:122 (pnpm clean && pnpm build, snapshot path only)
  • turbo.json tasks.build (dependsOn ^build; fans out over all workspace packages)
  • packages/*/package.json build scripts = 'tsup --tsconfig tsconfig.build.json' (esbuild-based, no type checking)

decided by agent · confidence 0.55

not_a_check — 15

Asserts nothing about correctness, so it is excluded from the denominator. This is the one shoppable class: mis-filing a real check here shrinks the denominator and inflates the score, so each row carries the same burden of argument as any other verdict.

analyze-versions
not_a_check
tools/analyze-downloads/package.json · script

src/analyze-versions.ts fetches https://api.npmjs.org/versions/ai/last-week, buckets the counts by major.minor and prints a table; there is no comparison, threshold or expectation anywhere in the file, so its exit status varies with network reachability and never with any property of the committed tree. No workflow, turbo task or root script invokes it (grep of .github/workflows finds no 'analyze-' reference), so nothing consumes whatever it prints.

producer a tsx-run reporting script that GETs npm registry download counts and prints a console.table — actor can write to it: not established

  • tools/analyze-downloads/src/analyze-versions.ts:44-60 (fetch + console.table, no assertion)
  • tools/analyze-downloads/package.json (no test/check script; package name 'analyze-downloads' absent from turbo.json tasks)
  • grep -rn 'analyze-' .github/workflows/ -> no match

decided by agent · confidence 0.88

analyze-providers
not_a_check
tools/analyze-downloads/package.json · script

src/analyze-providers.ts pulls weekly download counts per provider package and tabulates week-over-week deltas; every failure path is swallowed by a `catch (err) { console.error('Error:', err) }` at line 131, so even a total network outage leaves the process exiting zero. It asserts nothing about the repository and no workflow or turbo task invokes it.

producer a tsx-run reporting script that GETs npm registry download counts for provider packages and prints a console.table — actor can write to it: not established

  • tools/analyze-downloads/src/analyze-providers.ts:129-131 (console.table then catch/console.error)
  • grep -rn 'analyze-' .github/workflows/ -> no match

decided by agent · confidence 0.9

analyze-market
not_a_check
tools/analyze-downloads/package.json · script

src/analyze-market.ts fetches last-week and previous-week download counts for a hard-coded package list and prints market-share rows; the whole body is wrapped in a try/catch that logs and returns (line 141), so nothing it observes can turn into a non-zero exit. It states a market fact, not a claim about the code, and no workflow or turbo task runs it.

producer a tsx-run reporting script that GETs npm registry download counts for competing SDKs and prints a console.table — actor can write to it: not established

  • tools/analyze-downloads/src/analyze-market.ts:139-141 (console.table then catch/console.error)
  • grep -rn 'analyze-' .github/workflows/ -> no match

decided by agent · confidence 0.9

analyze-ui-frameworks
not_a_check
tools/analyze-downloads/package.json · script

src/analyze-ui-frameworks.ts fetches download stats for react/vue/svelte/angular adapter packages and tabulates them, catching every error at line 107. Nothing in the file compares an observed value to an expected one, so the step can neither pass nor fail in any informative sense; it is a reporting tool that no pipeline consumes.

producer a tsx-run reporting script that GETs npm registry download counts for UI framework packages and prints a console.table — actor can write to it: not established

  • tools/analyze-downloads/src/analyze-ui-frameworks.ts:102-107 (console.table then catch/console.error)
  • grep -rn 'analyze-' .github/workflows/ -> no match

decided by agent · confidence 0.9

generate-llms-txt
not_a_check
tools/generate-llms-txt/package.json · script

src/generate-llms-txt.ts walks the docs directory with readdir/readFile, concatenates the markdown and calls writeFile('llms.txt'); the only non-zero exit is process.exit(1) inside the IO catch at line 89, which reports that a file could not be read or written, not that the documentation is wrong. It produces an artifact rather than a verdict, and no workflow invokes it.

producer a tsx generator that reads the content/ docs tree and writes llms.txt — actor can write to it: not established

  • tools/generate-llms-txt/src/generate-llms-txt.ts:3 (readdir/readFile/writeFile), :86 (writeFile('llms.txt')), :88-89 (catch -> process.exit(1))
  • grep -rn 'generate-llms-txt' .github/workflows/ -> no match

decided by agent · confidence 0.88

type-check
not_a_check
tools/konsistent-provider/package.json · script

`tsc --noEmit` would be an anchored producer if it ran: the compiler decides the exit code from the committed sources and no assertion can talk it out of a type error. It does not run. The only type-checking job is ci.yml:129 `pnpm run type-check:full` = `tsc --build tsconfig.with-examples.json`, whose reference graph resolves through tsconfig.json's 69 project references, and none of them points at tools/konsistent-provider; no workflow runs `turbo type-check` either. An edge no pipeline path consumes carries no information into any gate, which is why it is filed here rather than as an anchored check — the same reasoning as a step guarded by `|| true`.

producer the TypeScript compiler — but nothing in the repository's pipelines invokes it for this package — actor can write to it: not established

  • tools/konsistent-provider/package.json scripts.type-check = 'tsc --noEmit'
  • .github/workflows/ci.yml:129 (pnpm run type-check:full -> tsc --build tsconfig.with-examples.json)
  • tsconfig.json references (69 entries, none under tools/)
  • grep -rn 'type-check' .github/workflows/ -> only ci.yml:129

decided by agent · confidence 0.6

test runner config (vitest.config.js)
not_a_check
examples/ai-functions/vitest.config.js · test_target

The config selects `**/*.test.ts{,x}` under the node environment, but nothing runs vitest for this package: examples/ai-functions/package.json declares no `test` script at all (only `test:e2e:all`, `test:file`, `type-check`), and both root aggregates that CI does run — `test` and `test:ci` (package.json:16-17) — carry `--filter=!@example/*`. The one CI job that enters this directory, load-time_matrix, runs `pnpm tsx src/benchmark/load-time.ts`, not vitest. So no exit code produced under this config ever reaches a gate.

producer a vitest config that no pipeline path loads — actor can write to it: not established

  • examples/ai-functions/package.json scripts (no 'test' key)
  • package.json:16-17 (test and test:ci both --filter=!@example/*)
  • .github/workflows/ci.yml load-time_matrix step 'Measure and check load time' (working-directory: examples/ai-functions, runs tsx src/benchmark/load-time.ts)

decided by agent · confidence 0.78

test runner config (playwright.config.ts)
not_a_check
packages/devtools/playwright.config.ts · test_target

The config is real — it boots `pnpm tsx src/viewer/server.ts` on port 14983 and drives Desktop Chrome against it — but the only script that loads it is @ai-sdk/devtools' `test:e2e`, and `test:e2e` is not a turbo task (turbo.json declares `test`, `test:e2e:ci`, not `test:e2e`) and appears in exactly one workflow step, ci.yml:448, which is pinned to `working-directory: packages/rsc`. The package's own `test` script resolves to `test:unit` (vitest.node.config.js), so the turbo test fan-out never touches this file. Nothing it could report reaches a gate.

producer a Playwright config that no pipeline path loads — actor can write to it: not established

  • packages/devtools/package.json scripts.test = 'pnpm test:unit'; scripts.test:e2e = 'pnpm build:client && playwright test'
  • .github/workflows/ci.yml:445-448 ('Run RSC e2e tests', working-directory: packages/rsc, run: pnpm test:e2e)
  • turbo.json tasks (test, test:e2e:ci — no test:e2e)

decided by agent · confidence 0.72

actions/create-github-app-token@bcd2ba49218906704ab6c1aa796996da409d3eb1 # v3.2.0
not_a_check
.github/workflows/ai-provider-api-changes.yml:15 · ci_step

The step exchanges `vars.VERCEL_AI_SDK_GITHUB_APP_CLIENT_ID` and a PKCS8 private key for a short-lived token published as `steps.app-token.outputs.token`. It exists so the next step can authenticate; nothing about the repository is read or evaluated, and a failure here means the credential is wrong or GitHub is down, not that the code is. Whatever this workflow does is done by the step that consumes the token.

producer GitHub's app-token endpoint, minting an installation token from an app id and a private key — actor can write to it: not established

  • .github/workflows/ai-provider-api-changes.yml:15-19 (uses actions/create-github-app-token, id: app-token)
  • .github/workflows/ai-provider-api-changes.yml:35 (GITHUB_TOKEN: ${{ steps.app-token.outputs.token }})

decided by agent · confidence 0.88

Create issues
not_a_check
.github/workflows/ai-provider-api-changes.yml:21 · ci_step

On a `repository_dispatch` of type ai-provider-api-change the step POSTs /repos/vercel/ai/issues with a title and body copied verbatim out of `github.event.client_payload.release`. It files a maintenance to-do; it evaluates nothing, compares nothing, and its failure would mean the API call was rejected. The workflow is not triggered by a pull request and gates no merge — it is an inbox, not a verdict.

producer the GitHub Issues REST endpoint, called via octokit/request-action — actor can write to it: not established

  • .github/workflows/ai-provider-api-changes.yml:5-7 (on: repository_dispatch)
  • .github/workflows/ai-provider-api-changes.yml:21-34 (route: POST /repos/{owner}/{repo}/issues, body from client_payload)

decided by agent · confidence 0.85

Send Slack notification
not_a_check
.github/workflows/slack-workflow-failure-notification.yml:31 · ci_step

The step is a transport for a verdict already reached elsewhere: the job only runs at all under `if: github.event.workflow_run.conclusion == 'failure'`, and the payload is just the failed workflow's name and html_url. It re-states someone else's result and makes no claim of its own. It also cannot report a problem — `curl` is invoked without `-f`, so an HTTP 4xx/5xx from Slack still exits zero, meaning even the delivery is unverified.

producer curl POSTing a jq-built JSON payload to a Slack webhook URL held in secrets — actor can write to it: not established

  • .github/workflows/slack-workflow-failure-notification.yml:20-29 (job-level if: conclusion == 'failure')
  • .github/workflows/slack-workflow-failure-notification.yml:31-45 (curl "$SLACK_WORKFLOW_FAILURE_URL" -X POST, no -f/--fail)

decided by agent · confidence 0.85

Configure Git
not_a_check
.github/workflows/update-model-settings.yml:44 · ci_step

`git config --global user.email/user.name` sets the identity later commits in this job will carry. It reads nothing from the repository and evaluates nothing about it; the only way it exits non-zero is if the runner's HOME is unwritable. The assertions in this workflow, such as they are, live in the generate and diff steps that follow.

producer git writing two keys into the runner's global config file — actor can write to it: not established

  • .github/workflows/update-model-settings.yml:44-48 (git config --global user.email / user.name)
  • .github/workflows/update-model-settings.yml:114-125 ('Create branch and push as signed commit' — the consumer of that identity)

decided by agent · confidence 0.93

Checkout Repository
not_a_check
.github/workflows/update-model-settings.yml:49 · ci_step

The step's whole effect is to put files or a toolchain on the runner so later steps have something to act on. It reads nothing about the repository's correctness and emits no pass/fail verdict about the artefact — a failure here means the runner could not fetch or install, not that the code is wrong. The check in such a job is whatever executes afterwards, so this edge must not enter the ratio in either direction.

producer a GitHub Actions provisioning action (repository checkout, toolchain setup, or dependency cache restore) — actor can write to it: not established

  • the step body is a bare `uses:` of a checkout/setup/cache action with no command of its own

decided by probe (ci-provisioning-action) · confidence 0.9

Setup pnpm
not_a_check
.github/workflows/update-model-settings.yml:55 · ci_step

The step body is a bare `uses:` with no `with:` block and no command; its whole effect is to put a package manager on PATH so subsequent steps have one. Nothing about the committed tree is evaluated and a failure means the toolchain could not be installed. Every assertion in this job comes later, from the commands pnpm goes on to run.

producer the pnpm/action-setup action, installing the pnpm binary named by package.json's packageManager field onto the runner — actor can write to it: not established

  • .github/workflows/update-model-settings.yml:55-56 (uses: pnpm/action-setup@fc06bc1... with no inputs)
  • package.json packageManager = 'pnpm@10.33.4'

decided by agent · confidence 0.9

Setup Node.js 22
not_a_check
.github/workflows/update-model-settings.yml:58 · ci_step

The step's whole effect is to put files or a toolchain on the runner so later steps have something to act on. It reads nothing about the repository's correctness and emits no pass/fail verdict about the artefact — a failure here means the runner could not fetch or install, not that the code is wrong. The check in such a job is whatever executes afterwards, so this edge must not enter the ratio in either direction.

producer a GitHub Actions provisioning action (repository checkout, toolchain setup, or dependency cache restore) — actor can write to it: not established

  • the step body is a bare `uses:` of a checkout/setup/cache action with no command of its own

decided by probe (ci-provisioning-action) · confidence 0.9

Crystallization curve

Keel crystallization curve Crystallization curve over 15 runs from <repo>/reports. falls — total fitted change -66.5% of the mean across 15 runs (R^2 0.28, so the line explains less than half the variance — read the raw squares) CRYSTALLIZATION CURVE 15 sequential runs · 345 nodes judged of 1838 gathered · 124 anchored Keel corpus — 15 repositories, 2026-07-24 — measured corpus (declared on the command line) estimated tokens per node estimated tokens / judged node 0 300 600 run 0 · keel · estimated tokens per node = 498 · judged 25 of 32 gathered run 1 · anthropic-sdk-python · estimated tokens per node = 369 · judged 25 of 41 gathered run 2 · openai-python · estimated tokens per node = 398 · judged 25 of 95 gathered run 3 · vercel-ai · estimated tokens per node = 410 · judged 25 of 1014 gathered run 4 · aider · estimated tokens per node = 204 · judged 25 of 62 gathered run 5 · browser-use · estimated tokens per node = 378 · judged 25 of 104 gathered run 6 · mcp-python-sdk · estimated tokens per node = 592 · judged 25 of 121 gathered run 7 · simonw-llm · estimated tokens per node = 189 · judged 25 of 38 gathered run 8 · tiktoken · estimated tokens per node = 240 run 9 · requests · estimated tokens per node = 290 · judged 25 of 106 gathered run 10 · flask · estimated tokens per node = 408 · judged 25 of 59 gathered run 11 · sinatra · estimated tokens per node = 265 · judged 25 of 59 gathered run 12 · commander-js · estimated tokens per node = 316 run 13 · anthropic-quickstarts · estimated tokens per node = 310 · judged 25 of 62 gathered run 14 · tiktoken · estimated tokens per node = 53 0* 1* 2* 3* 4* 5* 6* 7* 8 9* 10* 11* 12 13* 14 seconds per node measured s / judged node 0.0 60.0 120.0 run 0 · keel · seconds per node = 0.1 · judged 25 of 32 gathered run 1 · anthropic-sdk-python · seconds per node = 15.8 · judged 25 of 41 gathered run 2 · openai-python · seconds per node = 15.4 · judged 25 of 95 gathered run 3 · vercel-ai · seconds per node = 105.9 · judged 25 of 1014 gathered run 4 · aider · seconds per node = 21.3 · judged 25 of 62 gathered run 5 · browser-use · seconds per node = 21.7 · judged 25 of 104 gathered run 6 · mcp-python-sdk · seconds per node = 20.1 · judged 25 of 121 gathered run 7 · simonw-llm · seconds per node = 11.8 · judged 25 of 38 gathered run 8 · tiktoken · seconds per node = 23.7 run 9 · requests · seconds per node = 16.8 · judged 25 of 106 gathered run 10 · flask · seconds per node = 20.5 · judged 25 of 59 gathered run 11 · sinatra · seconds per node = 16.9 · judged 25 of 59 gathered run 12 · commander-js · seconds per node = 18.7 run 13 · anthropic-quickstarts · seconds per node = 20.4 · judged 25 of 62 gathered run 14 · tiktoken · seconds per node = 0.1 0* 1* 2* 3* 4* 5* 6* 7* 8 9* 10* 11* 12 13* 14 probe-decided share measured share of decided nodes 0.00 0.50 1.00 run 0 · keel · probe-decided share = 0.00 · judged 25 of 32 gathered run 1 · anthropic-sdk-python · probe-decided share = 0.00 · judged 25 of 41 gathered run 2 · openai-python · probe-decided share = 0.16 · judged 25 of 95 gathered run 3 · vercel-ai · probe-decided share = 0.08 · judged 25 of 1014 gathered run 4 · aider · probe-decided share = 0.56 · judged 25 of 62 gathered run 5 · browser-use · probe-decided share = 0.32 · judged 25 of 104 gathered run 6 · mcp-python-sdk · probe-decided share = 0.00 · judged 25 of 121 gathered run 7 · simonw-llm · probe-decided share = 0.56 · judged 25 of 38 gathered run 8 · tiktoken · probe-decided share = 0.58 run 9 · requests · probe-decided share = 0.28 · judged 25 of 106 gathered run 10 · flask · probe-decided share = 0.00 · judged 25 of 59 gathered run 11 · sinatra · probe-decided share = 0.28 · judged 25 of 59 gathered run 12 · commander-js · probe-decided share = 0.19 run 13 · anthropic-quickstarts · probe-decided share = 0.24 · judged 25 of 62 gathered run 14 · tiktoken · probe-decided share = 1.00 0* 1* 2* 3* 4* 5* 6* 7* 8 9* 10* 11* 12 13* 14 probe library size measured probes in library 0 15 30 run 0 · keel · probe library size = 0 · judged 25 of 32 gathered run 1 · anthropic-sdk-python · probe library size = 3 · judged 25 of 41 gathered run 2 · openai-python · probe library size = 5 · judged 25 of 95 gathered run 3 · vercel-ai · probe library size = 8 · judged 25 of 1014 gathered run 4 · aider · probe library size = 10 · judged 25 of 62 gathered run 5 · browser-use · probe library size = 12 · judged 25 of 104 gathered run 6 · mcp-python-sdk · probe library size = 13 · judged 25 of 121 gathered run 7 · simonw-llm · probe library size = 15 · judged 25 of 38 gathered run 8 · tiktoken · probe library size = 17 run 9 · requests · probe library size = 19 · judged 25 of 106 gathered run 10 · flask · probe library size = 22 · judged 25 of 59 gathered run 11 · sinatra · probe library size = 23 · judged 25 of 59 gathered run 12 · commander-js · probe library size = 26 run 13 · anthropic-quickstarts · probe library size = 29 · judged 25 of 62 gathered run 14 · tiktoken · probe library size = 29 0* 1* 2* 3* 4* 5* 6* 7* 8 9* 10* 11* 12 13* 14 x axis: run index in the recorded corpus order (listed below). Squares are the raw per-run values; the dashed line is an ordinary-least-squares fit and is never shown without them. The fit is clipped to the panel, and withheld entirely (with the panel saying so) where a straight line would predict values the points cannot take — R^2 for every fit is in the trend notes below. * on run 0, 1, 2, 3, 4, 5, 6, 7, 9, 10, 11, 13 = judged fewer nodes than gathered; per-node values are per JUDGED node. Coverage (judged) ci_step 224 · script 64 · test_target 46 · review_gate 10 · deploy_gate 1 Trend estimated tokens per node: falls — total fitted change -66.5% of the mean across 15 runs (R^2 0.28, so the line explains less than half the variance — read the raw squares). raw first 498 -> last 53. seconds per node: falls — total fitted change -72.9% of the mean across 15 runs (R^2 0.04, so the line explains less than half the variance — read the raw squares). raw first 0.1 -> last 0.1. probe-decided share: rises — total fitted change 144.9% of the mean across 15 runs (R^2 0.21, so the line explains less than half the variance — read the raw squares). raw first 0.00 -> last 1.00. probe library size: rises — total fitted change 187.7% of the mean across 15 runs (R^2 0.99). raw first 0 -> last 29. Run order run order derived from Report.generatedAt (ascending, filename as tiebreak) — no order.json manifest present 0 keel -> 1 anthropic-sdk-python -> 2 openai-python -> 3 vercel-ai -> 4 aider -> 5 browser-use -> 6 mcp-python-sdk -> 7 simonw-llm -> 8 tiktoken -> 9 requests -> 10 flask -> 11 sinatra -> 12 commander-js -> 13 anthropic-quickstarts -> 14 tiktoken Shuffle check shuffle check INCOMPLETE — the curve direction ran, the ratio direction did not Permutation (400 draws, seed 20260724): median |delta normalized slope| 0.633, 92% of draws move it by >= 0.15. A permutation reorders ALREADY-RECORDED runs; it cannot reproduce what a genuine re-run in a different order would have cost, because the probe library would have accumulated differently. It is a sanity signal on order-dependence, never a substitute for the empirical re-run. Ratio stability under shuffle was NOT checked: no re-run supplied (--shuffled) and none declared in corpus.meta.json. Permuting recorded runs cannot move a per-target ratio, so the permutation result below says nothing about it. Supply a re-run with --shuffled <dir>. Disclosures - Skipped 3 file(s) that are not usable run reports: corpus-summary.json (no string "target"); curve.json (no string "target"); keel.bindings.json (no "nodes" array). - Run 0 (keel) judged 25 of 32 gathered nodes — a cap. Per-node cost below is per JUDGED node. - Run 1 (anthropic-sdk-python) judged 25 of 41 gathered nodes — a cap. Per-node cost below is per JUDGED node. - Run 2 (openai-python) judged 25 of 95 gathered nodes — a cap. Per-node cost below is per JUDGED node. - Run 3 (vercel-ai) judged 25 of 1014 gathered nodes — a cap. Per-node cost below is per JUDGED node. - Run 4 (aider) judged 25 of 62 gathered nodes — a cap. Per-node cost below is per JUDGED node. - Run 5 (browser-use) judged 25 of 104 gathered nodes — a cap. Per-node cost below is per JUDGED node. - Run 6 (mcp-python-sdk) judged 25 of 121 gathered nodes — a cap. Per-node cost below is per JUDGED node. - Run 7 (simonw-llm) judged 25 of 38 gathered nodes — a cap. Per-node cost below is per JUDGED node. - Run 9 (requests) judged 25 of 106 gathered nodes — a cap. Per-node cost below is per JUDGED node. - Run 10 (flask) judged 25 of 59 gathered nodes — a cap. Per-node cost below is per JUDGED node. - Run 11 (sinatra) judged 25 of 59 gathered nodes — a cap. Per-node cost below is per JUDGED node. - Run 13 (anthropic-quickstarts) judged 25 of 62 gathered nodes — a cap. Per-node cost below is per JUDGED node. - Run 14 (tiktoken): decidedByProbe + decidedByAgent = 10 but nodesSampled = 12. Probe-decided share uses the decided total (10). - Provenance "measured" was declared on the COMMAND LINE (--provenance), not in <repo>/reports/corpus.meta.json. The declaration therefore lives in the invocation and is only as trustworthy as the run sheet that records it. - Token counts are ESTIMATES in 15 of 15 run(s) (RunEconomics.tokensEstimated). No API exposes session token usage to a skill, so the token axis reads "estimated tokens" and nothing here claims a measured token count. Scope. Keel measures the shape of verification, not its quality. A repo can be 100% anchored with terrible tests. Anchoring says the signal comes from outside; it does not say the signal is sufficient.

Run economics

2 decided by probe
23 decided by agent
8% probe-decided share
25 / 1,014 nodes judged / gathered
3,243 estimated tokens in
7,000 estimated tokens out
44m 06s wall clock
3 probes minted
8 probe library size

Token counts are estimated, and labelled so: a skill running inside an agent session has no API for its own usage, so the figure is ceil(chars/4) over the judgment payloads and responses. Wall clock and probe-decided share are measured directly. 25 of 1,014 gathered edges were judged — the cap is printed here and beside the ratio, never applied silently.