Keel
Grounding report.
Every verification edge in browser-use, classified by who produces the signal and whether the actor being verified can write to that producer.
| target | browser-use |
|---|---|
| revision | b909fbfba0ae42790f8402dd3daecc8f265be246 |
| generated | 2026-07-25T02:46:11.681Z |
anchored / (anchored + self_referential + unknown) = 7 / 7
Coverage (judged)
| node kind | judged |
|---|---|
| ci_step | 22 |
| test_target | 2 |
| review_gate | 1 |
ε-audit — not run
No verdict in this report carries a readable audit block, so nothing was re-decided and there is no agreement rate to report. This is not agreement, and it is not full agreement either: a rate over zero comparisons would state a measurement that was never taken. (“Readable” is load-bearing: a block can be present and state no comparison, and this state covers that too — see below if so.)
This run judged 25 of 104 gathered edges. Every number on this page describes that sample, not the whole surface — 79 gathered edges are absent from all of it. A cap is disclosed rather than smoothed: an undisclosed sample is how a ratio gets shopped.
Scope. Keel measures the shape of verification, not its quality. A repo can be 100% anchored with terrible tests. Anchoring says the signal comes from outside; it does not say the signal is sufficient.
Node graph — 25 node(s), in gather order
anchored — 7
This hook set is not decorative: .github/workflows/lint.yml:43 runs `uv run --no-sync pre-commit run --all-files --show-diff-on-failure` on every pull_request, so the gate is on the merge path. Each hook is a separate process fetched from an external repository at a declared rev and run over the files as committed; a violation makes that process exit non-zero and pre-commit propagates the failure, and a rewriting hook (ruff-format, end-of-file-fixer, fix-byte-order-marker) also fails the run because pre-commit treats a modified file as a failed hook. An author cannot make check-ast accept a syntax error, or detect-private-key ignore a key, without changing the file or changing the pinned hook config. Documented limits: the revs are mutable tags (v0.12.10, v1.1.404, v6.0.0) rather than SHAs, so the upstream owner — not the PR author — could move what runs; and the hook LIST itself is author-controlled, so several checks are commented out (add-trailing-comma, check-docstring-first, name-tests-test) and simply do not run.
producer exit codes of pinned third-party programs (ruff, pyright, codespell, pyupgrade, yesqa, pre-commit-hooks) executed by the pre-commit runner against the committed working tree — actor can write to it: no
- .pre-commit-config.yaml
- .github/workflows/lint.yml:43
- .github/workflows/lint.yml:16
decided by agent · confidence 0.8
Unlike the linter tables above, this node is the declaration of the test suite itself (`testpaths = ["tests"]`, `python_files`), and that suite is executed on the merge path: .github/workflows/test.yaml:182 runs `pytest "tests/ci/<file>.py"` as a matrix job over every discovered test file, with `max_attempts: 1` and no continue-on-error, so a non-zero pytest exit fails the required job. Nothing an author writes in the module under test can turn a raised AssertionError or a segfaulting browser process into a pass without changing the test or the code. Documented limits, on the axis this does NOT cover: the oracle is co-authored — the assertions live in the same repository as the implementation, so the suite proves the code does what the tests say and not that the tests say the right thing; `addopts = "-svx"` stops at the first failure; `timeout = 300` and the job's 4-minute cap mean a slow suite is truncated by the clock; and tests/ci/conftest.py replaces the LLM with an AsyncMock returning canned JSON, so agent-behaviour tests exercise a scripted oracle rather than a real model.
producer the pytest process exit code — the CPython runtime executing the declared suite and the assertions in it — actor can write to it: no
- pyproject.toml:190
- .github/workflows/test.yaml:182
- .github/workflows/test.yaml:100
- tests/ci/conftest.py
decided by agent · confidence 0.6
The step asks GitHub for the release environment's protection_rules and exits 1 unless a required_reviewers rule exists AND prevent_self_review is true; `set -euo pipefail` plus the explicit `exit 1` on a failed API call make it fail closed if the environment is missing or unreadable. The value being read is not in the repository and is not writable by this job: GITHUB_TOKEN here carries contents/id-token/actions only, and changing environment protection needs repository-admin access through the settings UI. Nothing a PR author commits, and nothing an earlier step in this pipeline writes, can make an unprotected environment report as protected. Documented limit: a repository admin can turn the protection off — but then this step fails rather than silently passing, which is the property that makes it a real gate on the publish job at publish.yml:76.
producer GitHub's environments API (`GET /repos/{repo}/environments/release`) — repository settings stored in GitHub's control plane, outside the git tree — actor can write to it: no
- .github/workflows/publish.yml:88-109
- .github/workflows/publish.yml:79
- .github/workflows/publish.yml:80-83
decided by agent · confidence 0.8
`--no-fix` means ruff reports instead of rewriting, so a PLE-class error (Python syntax / pyflakes-error rules) yields a non-zero exit, and with no `continue-on-error` and no `|| true` that failure stops the publish job before `uv build` at line 118 and before `uv publish` at line 137. The author cannot make a file with a syntax error report clean without editing the file or editing the pinned ruff configuration. Documented limit: `--select PLE` is a deliberately tiny rule set — this is a fast syntax tripwire, not the repository's lint gate, which lives in lint.yml.
producer the ruff process exit code — a linter binary parsing the committed source and returning non-zero on a violation — actor can write to it: no
- .github/workflows/publish.yml:117
- .github/workflows/publish.yml:118
- .github/workflows/publish.yml:137
decided by agent · confidence 0.9
The build backend resolves the project metadata and assembles the sdist and wheel; malformed pyproject metadata, an unresolvable version, or a missing declared file makes the process exit non-zero, and with no failure suppression that stops the job before `uv publish` at line 137. The verdict is produced by the packaging toolchain executing over the committed tree, not by anything the authors assert about it. Documented limit — narrow scope, and it should never be read as more: a Python wheel build neither imports nor executes browser_use, so this says the package can be assembled, not that the code in it works.
producer the uv/PEP 517 build-backend process exit code, running against pyproject.toml and the source tree — actor can write to it: no
- .github/workflows/publish.yml:118
- .github/workflows/publish.yml:137
decided by agent · confidence 0.6
`--trusted-publishing always` forces the OIDC path: the job exchanges a GitHub-signed id-token (permissions: id-token: write, publish.yml:82) for a short-lived PyPI credential, and PyPI decides whether the token's repository/workflow/environment claims match a configured trusted publisher. Neither the repository contents nor any earlier step in this pipeline can forge that token or overrule PyPI's decision, and PyPI also rejects a version that already exists or metadata it will not accept — a non-zero exit here stops the stable-branch push at line 138. Documented limit, and it is a real one: what the third party validates is publisher identity, version uniqueness and package metadata. It says nothing about whether the code in the wheel behaves. Confidence is held low deliberately because this step's primary role is the release action itself; the assertion is a by-product of it.
producer PyPI's upload API accepting or rejecting the distribution, with the uploader identity minted as an OIDC token by GitHub's identity provider — actor can write to it: no
- .github/workflows/publish.yml:137
- .github/workflows/publish.yml:82
- .github/workflows/publish.yml:138
decided by agent · confidence 0.55 · confidence below 0.6 on an anchored verdict
This is the `lint-syntax` job's only real step and lint.yml triggers on `pull_request` (lint.yml:16), so its exit code is on the merge path. `--no-fix` keeps it reporting rather than rewriting, so committing a file with a Python syntax error or a pyflakes-error-class defect makes ruff exit non-zero and fails the job; no `|| true`, no continue-on-error, nothing to suppress it. The author would have to change the file or the pinned ruff configuration to get a clean result. Documented limit: `--select PLE` restricts this to the error-class subset, so it is a syntax tripwire; the broader rule set only runs through the pre-commit job at lint.yml:43.
producer the ruff process exit code — the linter binary analysing the checked-out source and returning non-zero on a violation — actor can write to it: no
- .github/workflows/lint.yml:31
- .github/workflows/lint.yml:16
- .github/workflows/lint.yml:23
decided by agent · confidence 0.9
not_a_check — 18
`line-length = 130` and `fix = true` are parameters handed to a linter; the table is never executed and no exit code is derived from it. The signal-producing edges that read it are classified separately (.github/workflows/lint.yml:31 `ruff check --no-fix --select PLE`, and the ruff-check/ruff-format pre-commit hooks run at lint.yml:43). Filing it here is deliberate rather than convenient: counting a config table as a check would either add a phantom anchored edge or invent a producer that does not exist. Worth recording as shape, not as verdict: `fix = true` makes autofix the default, which is why the CI invocations must pass `--no-fix` to keep the run assertive rather than rewriting.
producer nothing — this is a TOML table read by ruff, not a process that emits a pass/fail — actor can write to it: no
- pyproject.toml:119
- .github/workflows/lint.yml:31
decided by agent · confidence 0.7
`select`/`ignore`/`unfixable` set which rules ruff enforces; the exit code that can block a merge is produced by the ruff process at .github/workflows/lint.yml:31 and by the ruff-check hook under lint.yml:43, both classified on their own. The table asserts nothing and cannot fail. It is, however, the actor-writable lever on the checks that do fire, and the narrowing here is real and should be read next to those verdicts: E501, F841 (unused variable), E731, E402, W291 and ASYNC109 are all ignored, and the standalone lint job at lint.yml:31 overrides `select` down to PLE alone, so the full rule set only gates via the pre-commit job.
producer nothing — a TOML rule-selection table consumed by ruff; it emits no verdict of its own — actor can write to it: no
- pyproject.toml:123
- .github/workflows/lint.yml:31
- .github/workflows/lint.yml:43
decided by agent · confidence 0.7
This block chooses single quotes, tab indentation and LF line endings. It emits no signal, and what it configures is a rewriter: the ruff-format hook's job is to change files, not to make a claim about whether the program is correct. It does become failure-capable inside `pre-commit run --all-files --show-diff-on-failure` at .github/workflows/lint.yml:43 — a file the formatter rewrites fails the hook — but that gate is a statement about typography, not about correctness, and it is already carried by the .pre-commit-config.yaml verdict rather than by this table.
producer nothing — a TOML style table (quote-style, indent-style, line-ending) consumed by the ruff formatter — actor can write to it: no
- pyproject.toml:136
- .pre-commit-config.yaml
- .github/workflows/lint.yml:43
decided by agent · confidence 0.65
`typeCheckingMode`, `include`, `exclude`, `venv` are inputs to pyright, not a verdict. The edges that actually gate are `uv run --no-sync pyright` at .github/workflows/lint.yml:54 and the pyright pre-commit hook at lint.yml:43, and those are where the anchored/self-referential question is decided. Recorded as shape because it directly bounds those edges: mode is `basic` (not strict), `reportMissingTypeStubs` is off, and the exclude list removes real product code from analysis — notably `browser_use/beta/service.py` together with `tests/ci/test_beta_agent.py`, plus browser_use/llm/oci_raw/* and two llm test modules. That narrowing is written by the same authors the checker is supposed to constrain.
producer nothing — a TOML scope/strictness table read by pyright; the type checker's exit code is produced by the invocations elsewhere — actor can write to it: no
- pyproject.toml:144
- .github/workflows/lint.yml:54
- .github/workflows/lint.yml:43
decided by agent · confidence 0.7
This module sets env vars (SKIP_LLM_API_KEY_VERIFICATION, ANONYMIZED_TELEMETRY=false, placeholder cloud URLs), makes a temp config dir, disables blocking socket shutdown, and builds fixtures. It is harness provisioning: nothing here can fail in a way that says anything about the correctness of browser_use. The assertions it supports run under the pytest edge classified at pyproject.toml#tool-pytest-ini-options. What it does deserve recording is where it puts the write boundary for those assertions: `create_mock_llm` returns an AsyncMock whose responses are hard-coded JSON agent outputs authored in this file, so every test that drives the agent loop through `mock_llm` is scoring the code against a script the test author wrote, not against a model's behaviour — execution stays anchored, the oracle does not.
producer nothing — pytest fixtures and environment setup; the file contains no test and emits no pass/fail — actor can write to it: no
- tests/ci/conftest.py
- .github/workflows/test.yaml:182
decided by agent · confidence 0.7
Every input to this step is an inactivity timer (days-before-stale 60/45, days-before-close 14) and every output is a label, a comment, or a close. It makes no claim about whether any code, artifact or deployment is correct, and there is no pass/fail it could contribute to a merge decision — `debug-only: false` just means it acts for real. It is repository housekeeping, so it is excluded from the ratio rather than counted as a weak check. The pin is a full commit SHA, so the implementation is at least identified even though it lives outside this repo — but that is moot here, since the step asserts nothing regardless of what the action does internally.
producer actions/stale reading issue/PR timestamps from the GitHub API and writing labels and comments back — actor can write to it: no
- .github/workflows/stale-bot.yml:15
decided by agent · confidence 0.9
The step's whole effect is to put files or a toolchain on the runner so later steps have something to act on. It reads nothing about the repository's correctness and emits no pass/fail verdict about the artefact — a failure here means the runner could not fetch or install, not that the code is wrong. The check in such a job is whatever executes afterwards, so this edge must not enter the ratio in either direction.
producer a GitHub Actions provisioning action (repository checkout, toolchain setup, or dependency cache restore) — actor can write to it: not established
- the step body is a bare `uses:` of a checkout/setup/cache action with no command of its own
decided by probe (ci-provisioning-action) · confidence 0.9
The step's purpose is mutation: compute the next rc version and `git tag` / `git push origin $new_tag`. It does contain `exit 1` branches, but every one of them fires on a property of the tag namespace the same maintainers push (no tag matched the version regex, or the rc counter hit 99) — never on a property of the code being released. So it asserts nothing about correctness and is excluded from the ratio rather than counted; the reason `actorCanWrite` is true here is recorded because the only thing it could be said to validate is an input the actor authored, which is why treating it as a check would have been the wrong call in the other direction.
producer bash + git, reading the repository's own tag list and writing a new tag back to origin — actor can write to it: yes
- .github/workflows/publish.yml:39-74
- .github/workflows/publish.yml:35
decided by agent · confidence 0.65
The step's whole effect is to put files or a toolchain on the runner so later steps have something to act on. It reads nothing about the repository's correctness and emits no pass/fail verdict about the artefact — a failure here means the runner could not fetch or install, not that the code is wrong. The check in such a job is whatever executes afterwards, so this edge must not enter the ratio in either direction.
producer a GitHub Actions provisioning action (repository checkout, toolchain setup, or dependency cache restore) — actor can write to it: not established
- the step body is a bare `uses:` of a checkout/setup/cache action with no command of its own
decided by probe (ci-provisioning-action) · confidence 0.9
The step's whole effect is to put files or a toolchain on the runner so later steps have something to act on. It reads nothing about the repository's correctness and emits no pass/fail verdict about the artefact — a failure here means the runner could not fetch or install, not that the code is wrong. The check in such a job is whatever executes afterwards, so this edge must not enter the ratio in either direction.
producer a GitHub Actions provisioning action (repository checkout, toolchain setup, or dependency cache restore) — actor can write to it: not established
- the step body is a bare `uses:` of a checkout/setup/cache action with no command of its own
decided by probe (ci-provisioning-action) · confidence 0.9
The command installs declared dependencies so that later steps can run. Without a lockfile-freezing flag the resolver will refresh a stale lock rather than error, so the step cannot report a discrepancy between the manifest and the lock; its non-zero exits mean the network or the index failed, not that the repository is wrong. The assertion in such a job lives in whatever executes against the installed environment afterwards.
producer a package manager's dependency resolver populating an environment — actor can write to it: not established
- the run command is a bare dependency install with no --locked/--frozen equivalent
decided by probe (unfrozen-dependency-install) · confidence 0.8
Pure machine preparation — it fetches a browser binary so later steps have one. Its only failure modes are network or CDN availability, which are properties of the runner's environment rather than of the repository, and no assertion about browser-use's correctness is derived from it. Note it is doing nothing for this job in particular: the test invocation it would have supported (`uv run pytest tests/ci/test_tools.py`) is commented out at publish.yml:134, so the download feeds no check at all in this workflow.
producer the playwright CLI downloading a Chrome build into the runner's cache — actor can write to it: no
- .github/workflows/publish.yml:130
- .github/workflows/publish.yml:134
decided by agent · confidence 0.9
Same shape as the chrome install one line above: a binary fetch that provisions the runner. It emits no verdict about the artifact being published, its failure modes are environmental, and the only step in this job that would have consumed the browser is commented out at publish.yml:134, so nothing downstream reads it.
producer the playwright CLI downloading a Chromium build into the runner's cache — actor can write to it: no
- .github/workflows/publish.yml:131
- .github/workflows/publish.yml:134
decided by agent · confidence 0.9
`git checkout -b stable` followed by `git push origin -f stable` is state mutation with no predicate: it force-moves a branch pointer to whatever commit the release was cut from. There is nothing here that can fail on a property of the code, and the `if:` guard is a condition on the triggering event, not a verification. It is excluded from the ratio; recorded separately is that this is the step which makes the `stable` branch a claim the release process writes about itself, so anything downstream that trusts `stable` is trusting an actor-set pointer rather than an independent signal.
producer git, writing a ref to origin — actor can write to it: yes
- .github/workflows/publish.yml:138-142
decided by agent · confidence 0.85
The step's whole effect is to put files or a toolchain on the runner so later steps have something to act on. It reads nothing about the repository's correctness and emits no pass/fail verdict about the artefact — a failure here means the runner could not fetch or install, not that the code is wrong. The check in such a job is whatever executes afterwards, so this edge must not enter the ratio in either direction.
producer a GitHub Actions provisioning action (repository checkout, toolchain setup, or dependency cache restore) — actor can write to it: not established
- the step body is a bare `uses:` of a checkout/setup/cache action with no command of its own
decided by probe (ci-provisioning-action) · confidence 0.9
The step's whole effect is to put files or a toolchain on the runner so later steps have something to act on. It reads nothing about the repository's correctness and emits no pass/fail verdict about the artefact — a failure here means the runner could not fetch or install, not that the code is wrong. The check in such a job is whatever executes afterwards, so this edge must not enter the ratio in either direction.
producer a GitHub Actions provisioning action (repository checkout, toolchain setup, or dependency cache restore) — actor can write to it: not established
- the step body is a bare `uses:` of a checkout/setup/cache action with no command of its own
decided by probe (ci-provisioning-action) · confidence 0.9
The step's whole effect is to put files or a toolchain on the runner so later steps have something to act on. It reads nothing about the repository's correctness and emits no pass/fail verdict about the artefact — a failure here means the runner could not fetch or install, not that the code is wrong. The check in such a job is whatever executes afterwards, so this edge must not enter the ratio in either direction.
producer a GitHub Actions provisioning action (repository checkout, toolchain setup, or dependency cache restore) — actor can write to it: not established
- the step body is a bare `uses:` of a checkout/setup/cache action with no command of its own
decided by probe (ci-provisioning-action) · confidence 0.9
The step's whole effect is to put files or a toolchain on the runner so later steps have something to act on. It reads nothing about the repository's correctness and emits no pass/fail verdict about the artefact — a failure here means the runner could not fetch or install, not that the code is wrong. The check in such a job is whatever executes afterwards, so this edge must not enter the ratio in either direction.
producer a GitHub Actions provisioning action (repository checkout, toolchain setup, or dependency cache restore) — actor can write to it: not established
- the step body is a bare `uses:` of a checkout/setup/cache action with no command of its own
decided by probe (ci-provisioning-action) · confidence 0.9
Crystallization curve
Run economics