Keel
Grounding report.
Every verification edge in openai-python, classified by who produces the signal and whether the actor being verified can write to that producer.
| target | openai-python |
|---|---|
| revision | d4c151d92ba713e56518a4483d3b7918dd1be838 |
| generated | 2026-07-25T01:42:13.604Z |
anchored / (anchored + self_referential + unknown) = 10 / 13
Coverage (judged)
| node kind | judged |
|---|---|
| ci_step | 22 |
| test_target | 2 |
| review_gate | 1 |
ε-audit — not run
No verdict in this report carries a readable audit block, so nothing was re-decided and there is no agreement rate to report. This is not agreement, and it is not full agreement either: a rate over zero comparisons would state a measurement that was never taken. (“Readable” is load-bearing: a block can be present and state no comparison, and this state covers that too — see below if so.)
This run judged 25 of 95 gathered edges. Every number on this page describes that sample, not the whole surface — 70 gathered edges are absent from all of it. A cap is disclosed rather than smoothed: an undisclosed sample is how a ratio gets shopped.
Scope. Keel measures the shape of verification, not its quality. A repo can be 100% anchored with terrible tests. Anchoring says the signal comes from outside; it does not say the signal is sufficient.
Node graph — 25 node(s), in gather order
anchored — 10
ci.yml:144/175/180 runs ./scripts/test, which ends in `rye run pytest` (scripts/test:62) under `set -e`, so a failing assertion or an unraised-warning (filterwarnings=["error"]) or a passing xfail (xfail_strict=true) makes the interpreter exit non-zero and the job red — an author cannot make a failing test report green without changing the committed source or this pinned config. Documented limit on the ORACLE axis: the assertions in tests/ are Stainless-codegenned from the same OpenAPI spec that generates src/openai, and scripts/test:33 boots a local mock server built from that spec, so the expected behaviour was authored inside the write boundary; only the execution is independent.
producer the pytest process exit code, executed as `rye run pytest` by ./scripts/test in the ci.yml `test` and `test-httpx2` jobs — actor can write to it: no
- pyproject.toml:155
- .github/workflows/ci.yml:143-144
- scripts/test:62
- scripts/test:28-34
decided by agent · confidence 0.8
This table is the pinned configuration pyright reads when it statically analyses the committed sources in the lint job; typeCheckingMode="strict" makes an unannotated or ill-typed symbol produce a non-zero exit that fails the job under `set -e` (scripts/lint:2). The verdict is computed by the type checker over the source tree, not asserted by anyone. Limit worth naming: the `exclude` list and the `report*=false` flags narrow what is analysed, and both are author-editable in-repo — so the scope of the check is inside the boundary even though the analysis is not.
producer the pyright process exit code, reached via ci.yml `Run lints` -> ./scripts/lint -> `rye run lint` -> `typecheck:pyright` -> scripts/run-pyright -> `pyright -p pyproject.toml` — actor can write to it: no
- pyproject.toml:168
- .github/workflows/ci.yml:40-41
- scripts/lint:12
- scripts/run-pyright:8
- pyproject.toml:110-114
decided by agent · confidence 0.85
ci.yml:41 runs ./scripts/lint, whose `rye run lint` chain includes `typecheck` -> `typecheck:mypy` = `mypy .` (pyproject.toml:115); mypy's static analysis of the committed tree decides the exit code and `set -e` propagates it to the job. disallow_untyped_defs/warn_return_any etc. are enforced by the checker, not by a declaration. Same scope limit as pyright: `exclude` drops src/openai/_files.py and all of tests/ from the analysis, so what is inside the check is author-selected while the analysis itself is not.
producer the mypy process exit code from `mypy .` (rye script `typecheck:mypy`), run in the ci.yml lint job — actor can write to it: no
- pyproject.toml:193
- pyproject.toml:110-115
- .github/workflows/ci.yml:40-41
- scripts/lint:12
decided by agent · confidence 0.85
Of the keys here, only target-version="py310" reaches the gating signal: it determines which Python version ruff assumes when deciding rules such as FA102 (missing future annotations), which IS selected, so it changes whether `ruff check .` exits non-zero. line-length=120 does not gate anything because E501 is not in the select list (pyproject.toml:262-292) and therefore only feeds the rewrite-only formatter, and output-format="grouped" is presentation. Low confidence is deliberate: this is a thin slice of an anchored edge, not an edge of its own.
producer the ruff process exit code from `ruff check .` (rye script `check:ruff`) in the ci.yml lint job — but only via one of this table's three keys — actor can write to it: no
- pyproject.toml:254
- pyproject.toml:104
- pyproject.toml:262-292
- .github/workflows/ci.yml:40-41
decided by agent · confidence 0.5 · confidence below 0.6 on an anchored verdict
This select list is what `ruff check .` enforces: a bare `except:` (E722), an unused import (F401), a stray `print` (T201, additionally listed as unfixable so it cannot be auto-silenced), or an unsorted import (I) makes ruff exit non-zero, `set -e` in scripts/lint aborts, and the lint job fails. The signal is produced by the linter binary reading the committed source; an author cannot make a violating file report clean without editing the file or this pinned rule set in a reviewed commit.
producer the ruff process exit code from `ruff check .` (rye script `check:ruff`), run by ./scripts/lint in the ci.yml lint job — actor can write to it: no
- pyproject.toml:262
- pyproject.toml:99-104
- scripts/lint:2
- scripts/lint:12
- .github/workflows/ci.yml:40-41
decided by agent · confidence 0.85
TID251 is in the select list (pyproject.toml:262-292), so this entry makes any `functools.lru_cache` import in the tree a hard ruff diagnostic and a non-zero exit that fails the lint job. The ban is decided by the linter scanning imports in committed source, not by anyone asserting the rule is respected.
producer the ruff process exit code — rule TID251 evaluated by `ruff check .` in the ci.yml lint job — actor can write to it: no
- pyproject.toml:296
- pyproject.toml:262-292
- .github/workflows/ci.yml:40-41
- scripts/lint:12
decided by agent · confidence 0.85
`I` (isort) is selected at pyproject.toml:264-265, so these ordering settings define a machine-decidable property of every source file that `ruff check .` verifies; a wrongly-ordered import block exits non-zero and reddens the lint job. Note the limit on what is being asserted: this constrains import layout only — it is a style property produced by executing a checker, not a correctness property.
producer the ruff process exit code — rule family `I` evaluated by `ruff check .` in the ci.yml lint job — actor can write to it: no
- pyproject.toml:299
- pyproject.toml:263-265
- .github/workflows/ci.yml:40-41
decided by agent · confidence 0.75
The producer is still the linter binary executing over the tree in the ci.yml lint job, and this table participates in that exit code by exempting bin/, scripts/, tests/ and examples/ from T201/T203. Confidence is deliberately low and the direction of the effect is worth stating plainly: an exemption table can only WEAKEN the gate, never make it fail, so it inherits anchoring from the check it configures while contributing no assertion of its own — a smell, surfaced rather than smoothed away.
producer the ruff process exit code from `ruff check .`; this table only narrows the file set the rules apply to — actor can write to it: no
- pyproject.toml:306
- pyproject.toml:270-275
- .github/workflows/ci.yml:40-41
decided by agent · confidence 0.5 · confidence below 0.6 on an anchored verdict
The fixtures here construct real OpenAI/AsyncOpenAI clients and hand them to tests whose pass/fail the interpreter decides during the CI pytest run, so the execution axis is outside the author's write path. Two limits belong on the record. First, base_url defaults to http://127.0.0.1:4010 — a mock Steady server that scripts/test:33 boots from the repo's own OpenAPI spec, the same spec the Stainless codegen used to write src/openai, so the responses the tests observe are a replica authored inside the boundary. Second, pytest_collection_modifyitems adds skip markers for aiohttp+respx combinations, i.e. the harness removes cases from the denominator; a skipped test produces no signal at all.
producer the pytest process exit code — conftest.py is the fixture harness for the suite ./scripts/test runs in the ci.yml `test` / `test-httpx2` jobs — actor can write to it: no
- tests/conftest.py:1
- tests/conftest.py:27-47
- tests/conftest.py:53-54
- scripts/test:28-34
- .github/workflows/ci.yml:143-144
decided by agent · confidence 0.55 · confidence below 0.6 on an anchored verdict
curl --fail turns any non-2xx from the two third-party endpoints into a non-zero exit, and jq --exit-status then evaluates the fetched payload's shape ('length > 5 and all(.[]; has("status") and has("end_of_life"))', and 'has("data")' for the download stats) — failing the step when the remote data is missing or malformed. Both signals originate at services outside the repository: the openai-python maintainers cannot write the Python release-cycle API or PyPI download statistics, so this step's outcome is decided by external reality. Limit: it validates the SHAPE of the fetched data, not that the numbers are right.
producer HTTP responses from peps.python.org and pypistats.org, surfaced as exit codes by `curl --fail` and then `jq --exit-status` — actor can write to it: no
- .github/workflows/python-version-review.yml:58-92
- .github/workflows/python-version-review.yml:72-74
- .github/workflows/python-version-review.yml:88-90
decided by agent · confidence 0.8
self_referential — 2
The step hands the repository to a language model and takes back prose, a review, or commits. Whatever verdict it renders is produced by the same class of system that produces the code being judged, so the two share correlated blind spots and agreement between them is not evidence. In the common configuration the model is also granted repository write tools, which means the reviewer can edit the very state it is reporting on.
producer a large language model invoked by a CI step — actor can write to it: yes
- the step's `uses:` reference resolves to an LLM coding/review agent action
decided by probe (llm-agent-ci-step) · confidence 0.75
The consequential output of this step — action_required, which is the sole condition gating the open_issue job at line 174 — is `head -n 1 $CODEX_OUTPUT_FILE` matched against two marker strings, i.e. the LLM decides whether its own finding is actionable by emitting the marker that says so; the envelope checks (non-empty, <60000 bytes, valid marker) constrain the shape of that artifact, not its truth. One genuinely independent sub-assertion is present and should be credited: `git diff --quiet HEAD` plus the untracked-file check is computed by git over a .git directory the codex user was explicitly denied write access to (line 101-102), so the agent cannot conceal a modification it made. That anchored fragment does not change the character of the step, whose product is an LLM's self-labelled verdict.
producer a markdown file written by the Codex LLM in the immediately preceding step (openai/codex-action, line 104); this step reads its first line and its byte count — actor can write to it: yes
- .github/workflows/python-version-review.yml:125-160
- .github/workflows/python-version-review.yml:144-158
- .github/workflows/python-version-review.yml:104-116
- .github/workflows/python-version-review.yml:171-174
- .github/workflows/python-version-review.yml:100-102
decided by agent · confidence 0.75
unknown — 1
The file's own header says it is 'used to automatically assign reviewers to PRs'. Assignment is routing, not a gate; the gate would be a 'Require review from Code Owners' branch-protection rule, and no such configuration is present anywhere in the clone (no rulesets, no settings-as-code) and it is not pinned to a revision I can read. I therefore cannot establish whether this edge blocks a merge at all, nor — since @openai/sdks-team both authors and reviews the codegenned changes — where the reviewer sits relative to the author's write boundary. Fails closed.
producer GitHub's pull-request review machinery — but whether an approving review is REQUIRED is decided by branch-protection / ruleset configuration that lives in GitHub's settings, not in this repository — actor can write to it: not established
- .github/CODEOWNERS:1-4
- no branch-protection or ruleset config in the clone
decided by agent · confidence 0.7
not_a_check — 12
docstring-code-format only changes how `ruff format` REWRITES docstring code blocks; it produces no pass/fail signal. Its only caller is the rye `format` chain via scripts/format, and `grep -rn 'format\|ruff' .github/workflows/` returns nothing — no CI job runs the formatter or a `--check`/`--diff` variant of it, so there is no exit code anywhere in the merge path that this table can influence.
producer `ruff format`, which rewrites files in place and is never invoked by any workflow in .github/workflows/ — actor can write to it: not established
- pyproject.toml:259
- pyproject.toml:89-97
- scripts/format:8
- .github/workflows/ (no ruff format invocation)
decided by agent · confidence 0.8
The step creates a system user and group so the Codex agent runs unprivileged. It asserts nothing about openai-python's correctness; its only failure mode is the runner being unable to create an account. Real safety value (credential isolation), zero verification content — it is setup for a later step, not a check of the artifact.
producer adduser/usermod exit codes — machine provisioning on the runner — actor can write to it: not established
- .github/workflows/python-version-review.yml:32-42
decided by agent · confidence 0.85
The step's whole effect is to put files or a toolchain on the runner so later steps have something to act on. It reads nothing about the repository's correctness and emits no pass/fail verdict about the artefact — a failure here means the runner could not fetch or install, not that the code is wrong. The check in such a job is whatever executes afterwards, so this edge must not enter the ratio in either direction.
producer a GitHub Actions provisioning action (repository checkout, toolchain setup, or dependency cache restore) — actor can write to it: not established
- the step body is a bare `uses:` of a checkout/setup/cache action with no command of its own
decided by probe (ci-provisioning-action) · confidence 0.9
Pure provisioning: creates the agent home directory, copies a config file into it, and appends a path to .git/info/exclude. No property of the repository is evaluated and no comparison is made, so there is no signal here that could come out either way on a correct-vs-incorrect codebase.
producer mkdir/cp/echo exit codes — actor can write to it: not established
- .github/workflows/python-version-review.yml:50-56
decided by agent · confidence 0.9
Filesystem permission provisioning that sets up the agent sandbox (workspace group-readable, .git non-writable). It creates the conditions under which a LATER step's git-cleanliness assertion is meaningful, but on its own it evaluates no property of the code and cannot report a defect.
producer chown/chmod exit codes — actor can write to it: not established
- .github/workflows/python-version-review.yml:94-102
decided by agent · confidence 0.85
All three commands end in `|| true` and the step carries `if: always()`, so its exit status is 0 unconditionally. A signal that cannot vary carries no information — this is teardown, and structurally incapable of being a check regardless of the state of the repository.
producer pkill/chown exit codes, every one of them swallowed — actor can write to it: not established
- .github/workflows/python-version-review.yml:118-123
decided by agent · confidence 0.95
Artifact transport between the assess and open_issue jobs. `if-no-files-found: error` gives it a failure mode, but the only thing that failure would report is that the preceding step in the same job did not leave the file it had just declared present — an operational guard on the pipeline's own plumbing, asserting nothing about openai-python's behaviour or types. Filed here rather than in the ratio because the class of thing it could ever detect is 'the upload path is broken', not 'the code is wrong'.
producer actions/upload-artifact's own exit code — actor can write to it: not established
- .github/workflows/python-version-review.yml:162-169
- .github/workflows/python-version-review.yml:146-149
decided by agent · confidence 0.6
Provisioning for the publish step: it fetches the artifact the previous job uploaded into runner.temp. No property of the repository or of the assessment is evaluated — it is the inbound half of the same transport.
producer actions/download-artifact's own exit code — actor can write to it: not established
- .github/workflows/python-version-review.yml:181-185
decided by agent · confidence 0.9
A side-effecting publication step: it creates a label and opens or refreshes an issue whose body is the LLM assessment verbatim. Nothing is compared and nothing can be found wanting — the step's own text calls the result 'advisory', and changing the Python floor is explicitly deferred to a separate reviewed pull request. Its failure modes are GitHub API errors, not findings.
producer the GitHub Issues API via `gh issue create/edit` — actor can write to it: not established
- .github/workflows/python-version-review.yml:187-233
- .github/workflows/python-version-review.yml:218
decided by agent · confidence 0.8
Computes a clone depth from the PR's commit count so the later checkout can reach the base SHA. It sets a variable; it evaluates no property of the diff and has no failing branch that would correspond to a defect in the code.
producer `expr` arithmetic written into $GITHUB_ENV — actor can write to it: not established
- .github/workflows/detect-breaking-changes.yml:14-21
decided by agent · confidence 0.9
The step's whole effect is to put files or a toolchain on the runner so later steps have something to act on. It reads nothing about the repository's correctness and emits no pass/fail verdict about the artefact — a failure here means the runner could not fetch or install, not that the code is wrong. The check in such a job is whatever executes afterwards, so this edge must not enter the ratio in either direction.
producer a GitHub Actions provisioning action (repository checkout, toolchain setup, or dependency cache restore) — actor can write to it: not established
- the step body is a bare `uses:` of a checkout/setup/cache action with no command of its own
decided by probe (ci-provisioning-action) · confidence 0.9
The step's whole effect is to put files or a toolchain on the runner so later steps have something to act on. It reads nothing about the repository's correctness and emits no pass/fail verdict about the artefact — a failure here means the runner could not fetch or install, not that the code is wrong. The check in such a job is whatever executes afterwards, so this edge must not enter the ratio in either direction.
producer a GitHub Actions provisioning action (repository checkout, toolchain setup, or dependency cache restore) — actor can write to it: not established
- the step body is a bare `uses:` of a checkout/setup/cache action with no command of its own
decided by probe (ci-provisioning-action) · confidence 0.9
Crystallization curve
Run economics