Keel
Grounding report.
Every verification edge in vercel-ai, classified by who produces the signal and whether the actor being verified can write to that producer.
| target | vercel-ai |
|---|---|
| revision | e29788dd545f8bf2300db0885658e639b4fd91bd |
| generated | 2026-07-25T02:26:49.454Z |
anchored / (anchored + self_referential + unknown) = 8 / 10
Coverage (judged)
| node kind | judged |
|---|---|
| ci_step | 8 |
| script | 8 |
| test_target | 8 |
| deploy_gate | 1 |
ε-audit — not run
No verdict in this report carries a readable audit block, so nothing was re-decided and there is no agreement rate to report. This is not agreement, and it is not full agreement either: a rate over zero comparisons would state a measurement that was never taken. (“Readable” is load-bearing: a block can be present and state no comparison, and this state covers that too — see below if so.)
This run judged 25 of 1,014 gathered edges. Every number on this page describes that sample, not the whole surface — 989 gathered edges are absent from all of it. A cap is disclosed rather than smoothed: an undisclosed sample is how a ratio gets shopped.
Scope. Keel measures the shape of verification, not its quality. A repo can be 100% anchored with terrible tests. Anchoring says the signal comes from outside; it does not say the signal is sufficient.
Node graph — 25 node(s), in gather order
anchored — 8
ci.yml:313 runs `pnpm test:ci` = `turbo test --filter=!@example/* --filter=!ai --filter=!@ai-sdk/codemod --only`, which reaches @ai-sdk/vue's `test` script = `vitest --config vitest.config.js --run`; this file is the config that script names, so its `include` glob and jsdom environment determine which assertions execute and the runner's exit code fails the 'Test' job, which the branch-protection aggregate job `test` (ci.yml:511-520) hard-depends on. DOCUMENTED LIMIT, on two axes: the include glob `src/**/*.ui.test.ts{,x}` is authored inside the write boundary, so narrowing it silences tests without any runtime noticing, and the assertions themselves are co-authored with the implementation — the execution axis is what is anchored here, not the oracle.
producer the vitest process exit code, decided by a jsdom-hosted runtime executing the committed Vue component tests — actor can write to it: no
- .github/workflows/ci.yml:313 (run: pnpm test:ci)
- package.json:17 (test:ci turbo filter chain; @ai-sdk/vue not excluded)
- packages/vue/package.json scripts.test = 'vitest --config vitest.config.js --run'
- .github/workflows/ci.yml:511-520 (required aggregate job 'test' needs test_matrix)
decided by agent · confidence 0.78
Same causal path as the Vue config: ci.yml:313 `pnpm test:ci` fans out through turbo to @ai-sdk/angular's `test` = `vitest --config vitest.config.ts --run`, and this file supplies the environment and the `src/lib/**/*.test.ts` include set the runner uses; a thrown assertion makes the interpreter exit non-zero and reddens the Test job feeding the required aggregate. DOCUMENTED LIMIT: execution is decided outside the authors' reach, but the include glob and the expectations are both committed artefacts, so this measures that the code does what the tests say, not that the tests say the right thing.
producer the vitest process exit code, decided by a jsdom-hosted runtime executing the committed Angular tests — actor can write to it: no
- .github/workflows/ci.yml:313 (run: pnpm test:ci)
- packages/angular/package.json scripts.test = 'vitest --config vitest.config.ts --run'
- packages/angular/vitest.config.ts (environment jsdom, include src/lib/**/*.test.ts)
decided by agent · confidence 0.76
This file exports createVitestConfig(environment); vitest.node.config.js and vitest.edge.config.js are two-line files that call it, and ci.yml:479-480 runs `pnpm test:node --shard` and `pnpm test:edge --shard` inside packages/ai across a 3-Node x 4-shard matrix. So the include/exclude sets and `typecheck: { enabled: true }` declared here decide what the runtime executes and whether the tsc pass runs, and the resulting exit code reddens the 'Test AI' jobs. The typecheck flag is the strongest part: it puts a compiler, not an assertion, in the path. DOCUMENTED LIMIT: the exclusion list (`**/*.ui.test.ts{,x}`, `**/*.e2e.test.ts{,x}`) is author-controlled, and the assertions are co-authored with the implementation, so the anchoring is on execution, not on the specification.
producer the vitest process exit code for the core `ai` package, plus vitest's typecheck pass, both decided by execution over the committed sources — actor can write to it: no
- packages/ai/vitest.node.config.js and vitest.edge.config.js (import { createVitestConfig } from './vitest.config.js')
- .github/workflows/ci.yml:475-480 (test_ai_matrix: pnpm test:node --shard / pnpm test:edge --shard)
- packages/ai/vitest.config.js typecheck.enabled = true
decided by agent · confidence 0.82
ci.yml:445-448 runs `pnpm test:e2e` with `working-directory: packages/rsc`, which is `playwright test` reading this config; the config boots `pnpm run dev` in tests/e2e/next-server, waits on http://localhost:3000 with `reuseExistingServer: false`, and the specs then click through the page and assert on rendered text (tests/e2e/spec/streamable.e2e.test.ts). The verdict comes from a browser observing a server process, neither of which an author can persuade — a broken stream renders wrong text and the run goes red. TWO DOCUMENTED LIMITS: `retries: 2` means only consistently-failing behaviour reddens the job, and the whole step is guarded by `if: steps.rsc-e2e.outputs.run == 'true'`, a git-diff-derived flag, so on PRs that do not touch packages/rsc this edge does not execute at all.
producer Chromium driven by Playwright against a real Next.js dev server, with the browser's observed DOM deciding pass/fail — actor can write to it: no
- .github/workflows/ci.yml:445-448 (Run RSC e2e tests, working-directory packages/rsc, pnpm test:e2e)
- packages/rsc/package.json scripts.test:e2e = 'playwright test'
- packages/rsc/playwright.config.ts (webServer command 'pnpm run dev', retries: 2, reuseExistingServer: false)
- packages/rsc/tests/e2e/spec/streamable.e2e.test.ts:3-10 (page.goto + expect(logs).toHaveText)
- .github/workflows/ci.yml:400-415 ('Detect RSC e2e changes' sets run=true/false from git diff)
decided by agent · confidence 0.8
ci.yml:313 `pnpm test:ci` reaches @ai-sdk/react (it is not in the exclusion filters) and runs its `test` script `vitest --config vitest.config.js --run`, which loads this file; the jsdom environment, the `src/**/*.test.ts{,x}` include and the `./src/vitest-setup.ts` setup file it declares determine what executes, and the runner's exit code propagates into the Test job that the required aggregate `test` job depends on. DOCUMENTED LIMIT: the setup file and the assertions live inside the same repository as the implementation, so what is anchored is that the runtime honestly decided pass/fail, not that the expectations are correct.
producer the vitest process exit code, decided by a jsdom-hosted runtime executing the committed React hook tests — actor can write to it: no
- .github/workflows/ci.yml:313 (run: pnpm test:ci)
- package.json:17 (test:ci filters exclude only @example/*, ai, @ai-sdk/codemod)
- packages/react/package.json scripts.test = 'vitest --config vitest.config.js --run'
- packages/react/vitest.config.js (environment jsdom, setupFiles ./src/vitest-setup.ts)
decided by agent · confidence 0.78
ci.yml:507 names this file directly — `pnpm exec vitest --config vitest.config.ts --run --shard ${{ matrix.shard }}` in packages/codemod across a 3-Node x 4-shard matrix, and that job is one of the `needs` of the required aggregate `test` job (ci.yml:511-520). The transform actually runs (test-utils.ts applyTransform) and the comparison is `expect(actualOutput).toBe(expectedOutput)`, so the interpreter decides. DOCUMENTED LIMIT, and it is the sharp one: the expected output is a committed golden file that `pnpm test:update` (`UPDATE_SNAPSHOT=true vitest -u --run`) rewrites from the transformer's own output — test-utils.ts:156-165 branches on that env var and writes the fixture instead of asserting. CI does not set UPDATE_SNAPSHOT, so the comparison genuinely executes, but the oracle was authored by the thing under test and can be re-authored by one documented command. Execution anchored; specification inside the boundary.
producer the vitest process exit code, decided by executing each codemod transformer over its input fixture and comparing the produced text to the committed output fixture — actor can write to it: no
- .github/workflows/ci.yml:507 (pnpm exec vitest --config vitest.config.ts --run --shard)
- .github/workflows/ci.yml:511-520 (aggregate 'test' job needs test_codemod_matrix)
- packages/codemod/src/test/test-utils.ts:150-168 (applyTransform, UPDATE_SNAPSHOT branch, expect(actualOutput).toBe(expectedOutput))
- packages/codemod/package.json scripts.test:update = 'UPDATE_SNAPSHOT=true vitest -u --run'
decided by agent · confidence 0.82
`--frozen-lockfile` is what makes this different from a plain install: pnpm will not refresh a stale lock, it exits non-zero, so the step is a genuine falsifiable assertion about two committed files agreeing — an author who edits a dependency and forgets to regenerate the lock cannot make this pass without changing the tree. The producer is the resolver executing, not anyone's claim. The exit code gates the rest of the job: the 'Generate model settings' and PR-creation steps below only run if this succeeds. DOCUMENTED LIMIT: what is anchored is manifest/lock agreement and registry reachability, nothing about whether the code works; and this instance sits in a scheduled bot workflow, so it gates that workflow's PR creation rather than a merge (the same command in ci.yml is the merge-gating instance).
producer pnpm's resolver comparing the committed pnpm-lock.yaml against the committed package.json manifests, and refusing to proceed on a mismatch — actor can write to it: no
- .github/workflows/update-model-settings.yml:64-65 (run: pnpm install --frozen-lockfile)
- .github/workflows/update-model-settings.yml:67-71 ('Generate model settings' runs after it)
- .github/workflows/ci.yml:31, :57, :100, :126 (same frozen install on the merge-gating path)
decided by agent · confidence 0.72
The file's only substantive key sets the command Vercel runs to produce the deployment, and Vercel promotes a deployment only when that command exits zero — so the promotion condition is the Next.js compiler's verdict on the committed docs tree, produced on infrastructure the repository's authors do not write to. A broken MDX import or a type error surfaced by next build cannot be argued into a successful deploy. TWO LIMITS, and they are why confidence is low rather than the class being upgraded: the signal says only that the site builds, nothing about whether the docs are correct; and this tree contains no evidence about whether the resulting deployment is required for a merge, so the strength of the gate is asserted from Vercel's platform behaviour rather than from anything I could read here.
producer Vercel's build infrastructure executing `pnpm build:site` (sync-content -> fumadocs-mdx -> next build) on the deployment runner — actor can write to it: no
- apps/docs/vercel.json (buildCommand: pnpm build:site)
- apps/docs/package.json scripts.build:site = 'pnpm sync-content && fumadocs-mdx && next build'
- .github/workflows/ci.yml:179 (separate merge-gating docs job: pnpm --filter ai-sdk-docs validate:site)
decided by agent · confidence 0.45 · confidence below 0.6 on an anchored verdict
unknown — 2
This step IS on the merge-gating path: ci.yml:62 runs `pnpm konsistent`, which is `turbo run konsistent:validate konsistent:check`, and both of those tasks declare `dependsOn: ["konsistent-provider#build"]` (turbo.json:102-108), so a non-zero exit here reddens the required 'Code Consistency' job. What I cannot establish is what the exit code means: `konsistent-convention emit` is a binary from @konsistent/convention ^1.0.0-beta, which is not vendored in the tree and which I cannot descend into from this clone, so I cannot tell whether it validates the conventions declared in src/index.ts (a real property of committed source) or merely serialises them to dist/conventions.json (pure generation). Guessing either way would be inventing the fork point.
producer the `konsistent-convention` CLI from the third-party @konsistent/convention package, executing on the runner — actor can write to it: not established
- .github/workflows/ci.yml:62 (run: pnpm konsistent)
- package.json:31 (konsistent: turbo run konsistent:validate konsistent:check)
- turbo.json:102-108 (//#konsistent:check and //#konsistent:validate dependsOn konsistent-provider#build)
- tools/konsistent-provider/package.json dependencies: @konsistent/convention ^1.0.0-beta (implementation not in tree)
decided by agent · confidence 0.62
This is an aggregate wrapper, and the rule for aggregates is to descend rather than classify the wrapper. Descending does not resolve it: the children are heterogeneous, several are third-party generators I already could not trace (konsistent-convention emit), and the dominant child, tsup, drives esbuild, which strips types without checking them — so a green `turbo build` does not even mean the tree type-checks. Its one pipeline caller is release.yml:122 (`pnpm clean && pnpm build`) in the snapshot-publish branch, not the merge gate. I can name the producer but not establish, uniformly across the children, whether the signal reports a property of the committed source or only that artifacts were emitted.
producer turbo, whose exit code is the maximum over ~70 per-package build commands (tsup/esbuild, vite build, next build, konsistent-convention emit, ...) — actor can write to it: not established
- package.json:4 (build: turbo build --concurrency 16)
- .github/workflows/release.yml:122 (pnpm clean && pnpm build, snapshot path only)
- turbo.json tasks.build (dependsOn ^build; fans out over all workspace packages)
- packages/*/package.json build scripts = 'tsup --tsconfig tsconfig.build.json' (esbuild-based, no type checking)
decided by agent · confidence 0.55
not_a_check — 15
src/analyze-versions.ts fetches https://api.npmjs.org/versions/ai/last-week, buckets the counts by major.minor and prints a table; there is no comparison, threshold or expectation anywhere in the file, so its exit status varies with network reachability and never with any property of the committed tree. No workflow, turbo task or root script invokes it (grep of .github/workflows finds no 'analyze-' reference), so nothing consumes whatever it prints.
producer a tsx-run reporting script that GETs npm registry download counts and prints a console.table — actor can write to it: not established
- tools/analyze-downloads/src/analyze-versions.ts:44-60 (fetch + console.table, no assertion)
- tools/analyze-downloads/package.json (no test/check script; package name 'analyze-downloads' absent from turbo.json tasks)
- grep -rn 'analyze-' .github/workflows/ -> no match
decided by agent · confidence 0.88
src/analyze-providers.ts pulls weekly download counts per provider package and tabulates week-over-week deltas; every failure path is swallowed by a `catch (err) { console.error('Error:', err) }` at line 131, so even a total network outage leaves the process exiting zero. It asserts nothing about the repository and no workflow or turbo task invokes it.
producer a tsx-run reporting script that GETs npm registry download counts for provider packages and prints a console.table — actor can write to it: not established
- tools/analyze-downloads/src/analyze-providers.ts:129-131 (console.table then catch/console.error)
- grep -rn 'analyze-' .github/workflows/ -> no match
decided by agent · confidence 0.9
src/analyze-market.ts fetches last-week and previous-week download counts for a hard-coded package list and prints market-share rows; the whole body is wrapped in a try/catch that logs and returns (line 141), so nothing it observes can turn into a non-zero exit. It states a market fact, not a claim about the code, and no workflow or turbo task runs it.
producer a tsx-run reporting script that GETs npm registry download counts for competing SDKs and prints a console.table — actor can write to it: not established
- tools/analyze-downloads/src/analyze-market.ts:139-141 (console.table then catch/console.error)
- grep -rn 'analyze-' .github/workflows/ -> no match
decided by agent · confidence 0.9
src/analyze-ui-frameworks.ts fetches download stats for react/vue/svelte/angular adapter packages and tabulates them, catching every error at line 107. Nothing in the file compares an observed value to an expected one, so the step can neither pass nor fail in any informative sense; it is a reporting tool that no pipeline consumes.
producer a tsx-run reporting script that GETs npm registry download counts for UI framework packages and prints a console.table — actor can write to it: not established
- tools/analyze-downloads/src/analyze-ui-frameworks.ts:102-107 (console.table then catch/console.error)
- grep -rn 'analyze-' .github/workflows/ -> no match
decided by agent · confidence 0.9
src/generate-llms-txt.ts walks the docs directory with readdir/readFile, concatenates the markdown and calls writeFile('llms.txt'); the only non-zero exit is process.exit(1) inside the IO catch at line 89, which reports that a file could not be read or written, not that the documentation is wrong. It produces an artifact rather than a verdict, and no workflow invokes it.
producer a tsx generator that reads the content/ docs tree and writes llms.txt — actor can write to it: not established
- tools/generate-llms-txt/src/generate-llms-txt.ts:3 (readdir/readFile/writeFile), :86 (writeFile('llms.txt')), :88-89 (catch -> process.exit(1))
- grep -rn 'generate-llms-txt' .github/workflows/ -> no match
decided by agent · confidence 0.88
`tsc --noEmit` would be an anchored producer if it ran: the compiler decides the exit code from the committed sources and no assertion can talk it out of a type error. It does not run. The only type-checking job is ci.yml:129 `pnpm run type-check:full` = `tsc --build tsconfig.with-examples.json`, whose reference graph resolves through tsconfig.json's 69 project references, and none of them points at tools/konsistent-provider; no workflow runs `turbo type-check` either. An edge no pipeline path consumes carries no information into any gate, which is why it is filed here rather than as an anchored check — the same reasoning as a step guarded by `|| true`.
producer the TypeScript compiler — but nothing in the repository's pipelines invokes it for this package — actor can write to it: not established
- tools/konsistent-provider/package.json scripts.type-check = 'tsc --noEmit'
- .github/workflows/ci.yml:129 (pnpm run type-check:full -> tsc --build tsconfig.with-examples.json)
- tsconfig.json references (69 entries, none under tools/)
- grep -rn 'type-check' .github/workflows/ -> only ci.yml:129
decided by agent · confidence 0.6
The config selects `**/*.test.ts{,x}` under the node environment, but nothing runs vitest for this package: examples/ai-functions/package.json declares no `test` script at all (only `test:e2e:all`, `test:file`, `type-check`), and both root aggregates that CI does run — `test` and `test:ci` (package.json:16-17) — carry `--filter=!@example/*`. The one CI job that enters this directory, load-time_matrix, runs `pnpm tsx src/benchmark/load-time.ts`, not vitest. So no exit code produced under this config ever reaches a gate.
producer a vitest config that no pipeline path loads — actor can write to it: not established
- examples/ai-functions/package.json scripts (no 'test' key)
- package.json:16-17 (test and test:ci both --filter=!@example/*)
- .github/workflows/ci.yml load-time_matrix step 'Measure and check load time' (working-directory: examples/ai-functions, runs tsx src/benchmark/load-time.ts)
decided by agent · confidence 0.78
The config is real — it boots `pnpm tsx src/viewer/server.ts` on port 14983 and drives Desktop Chrome against it — but the only script that loads it is @ai-sdk/devtools' `test:e2e`, and `test:e2e` is not a turbo task (turbo.json declares `test`, `test:e2e:ci`, not `test:e2e`) and appears in exactly one workflow step, ci.yml:448, which is pinned to `working-directory: packages/rsc`. The package's own `test` script resolves to `test:unit` (vitest.node.config.js), so the turbo test fan-out never touches this file. Nothing it could report reaches a gate.
producer a Playwright config that no pipeline path loads — actor can write to it: not established
- packages/devtools/package.json scripts.test = 'pnpm test:unit'; scripts.test:e2e = 'pnpm build:client && playwright test'
- .github/workflows/ci.yml:445-448 ('Run RSC e2e tests', working-directory: packages/rsc, run: pnpm test:e2e)
- turbo.json tasks (test, test:e2e:ci — no test:e2e)
decided by agent · confidence 0.72
The step exchanges `vars.VERCEL_AI_SDK_GITHUB_APP_CLIENT_ID` and a PKCS8 private key for a short-lived token published as `steps.app-token.outputs.token`. It exists so the next step can authenticate; nothing about the repository is read or evaluated, and a failure here means the credential is wrong or GitHub is down, not that the code is. Whatever this workflow does is done by the step that consumes the token.
producer GitHub's app-token endpoint, minting an installation token from an app id and a private key — actor can write to it: not established
- .github/workflows/ai-provider-api-changes.yml:15-19 (uses actions/create-github-app-token, id: app-token)
- .github/workflows/ai-provider-api-changes.yml:35 (GITHUB_TOKEN: ${{ steps.app-token.outputs.token }})
decided by agent · confidence 0.88
On a `repository_dispatch` of type ai-provider-api-change the step POSTs /repos/vercel/ai/issues with a title and body copied verbatim out of `github.event.client_payload.release`. It files a maintenance to-do; it evaluates nothing, compares nothing, and its failure would mean the API call was rejected. The workflow is not triggered by a pull request and gates no merge — it is an inbox, not a verdict.
producer the GitHub Issues REST endpoint, called via octokit/request-action — actor can write to it: not established
- .github/workflows/ai-provider-api-changes.yml:5-7 (on: repository_dispatch)
- .github/workflows/ai-provider-api-changes.yml:21-34 (route: POST /repos/{owner}/{repo}/issues, body from client_payload)
decided by agent · confidence 0.85
The step is a transport for a verdict already reached elsewhere: the job only runs at all under `if: github.event.workflow_run.conclusion == 'failure'`, and the payload is just the failed workflow's name and html_url. It re-states someone else's result and makes no claim of its own. It also cannot report a problem — `curl` is invoked without `-f`, so an HTTP 4xx/5xx from Slack still exits zero, meaning even the delivery is unverified.
producer curl POSTing a jq-built JSON payload to a Slack webhook URL held in secrets — actor can write to it: not established
- .github/workflows/slack-workflow-failure-notification.yml:20-29 (job-level if: conclusion == 'failure')
- .github/workflows/slack-workflow-failure-notification.yml:31-45 (curl "$SLACK_WORKFLOW_FAILURE_URL" -X POST, no -f/--fail)
decided by agent · confidence 0.85
`git config --global user.email/user.name` sets the identity later commits in this job will carry. It reads nothing from the repository and evaluates nothing about it; the only way it exits non-zero is if the runner's HOME is unwritable. The assertions in this workflow, such as they are, live in the generate and diff steps that follow.
producer git writing two keys into the runner's global config file — actor can write to it: not established
- .github/workflows/update-model-settings.yml:44-48 (git config --global user.email / user.name)
- .github/workflows/update-model-settings.yml:114-125 ('Create branch and push as signed commit' — the consumer of that identity)
decided by agent · confidence 0.93
The step's whole effect is to put files or a toolchain on the runner so later steps have something to act on. It reads nothing about the repository's correctness and emits no pass/fail verdict about the artefact — a failure here means the runner could not fetch or install, not that the code is wrong. The check in such a job is whatever executes afterwards, so this edge must not enter the ratio in either direction.
producer a GitHub Actions provisioning action (repository checkout, toolchain setup, or dependency cache restore) — actor can write to it: not established
- the step body is a bare `uses:` of a checkout/setup/cache action with no command of its own
decided by probe (ci-provisioning-action) · confidence 0.9
The step body is a bare `uses:` with no `with:` block and no command; its whole effect is to put a package manager on PATH so subsequent steps have one. Nothing about the committed tree is evaluated and a failure means the toolchain could not be installed. Every assertion in this job comes later, from the commands pnpm goes on to run.
producer the pnpm/action-setup action, installing the pnpm binary named by package.json's packageManager field onto the runner — actor can write to it: not established
- .github/workflows/update-model-settings.yml:55-56 (uses: pnpm/action-setup@fc06bc1... with no inputs)
- package.json packageManager = 'pnpm@10.33.4'
decided by agent · confidence 0.9
The step's whole effect is to put files or a toolchain on the runner so later steps have something to act on. It reads nothing about the repository's correctness and emits no pass/fail verdict about the artefact — a failure here means the runner could not fetch or install, not that the code is wrong. The check in such a job is whatever executes afterwards, so this edge must not enter the ratio in either direction.
producer a GitHub Actions provisioning action (repository checkout, toolchain setup, or dependency cache restore) — actor can write to it: not established
- the step body is a bare `uses:` of a checkout/setup/cache action with no command of its own
decided by probe (ci-provisioning-action) · confidence 0.9
Crystallization curve
Run economics