Review evidence

pulumi/docs PR #21909 fecf1388a5 generated 2026-09-25T20:29:31Z

Verification trail

#1 ➖ Not a claim
.claude/commands/docs-review/references/update.md:299
per finding with what it already records — newest `updated_at` wins), runs `apply-update.py` (validate patch → apply actions → merge
framing: A description of the author's own tooling, not a factual claim that could be checked against an outside source.; evidence: This line is from an internal Claude command reference (.claude/commands/docs-review/references/update.md). It describes the PR author's own review pipeline: the merge rule where the newest `updated_at` wins, and the steps inside the…; source: .claude/commands/docs-review/references/update.md:L299 (the file under review, used only to classify the line)
source: .claude/commands/docs-review/references/update.md:L299 (the file under review, used only to classify the line)
framing: A description of the author's own tooling, not a factual claim that could be checked against an outside source.
pass3 · entity-spec · high confidence
#2 ➖ Not a claim
.claude/commands/docs-review/scripts/build-evidence.py:602
this, an accepted F1 left both saying "1 item blocks merge" (2026-09-01
framing: A regex matched the "1 item" count and the 2026-09-01 date in a code comment about internal tooling behavior.; evidence: This is a code comment in the PR author's own docs-review tooling script (build-evidence.py). It describes a past bug in that pipeline, where an accepted F1 finding still left both outputs saying "1 item blocks merge." It is not a…; source: .claude/commands/docs-review/scripts/build-evidence.py:L602
source: .claude/commands/docs-review/scripts/build-evidence.py:L602
framing: A regex matched the "1 item" count and the 2026-09-01 date in a code comment about internal tooling behavior.
pass3 · numerical · high confidence
#3 ➖ Not a claim
.claude/commands/pr-review/SKILL.md:115
Per PR: `domains` (`classify_path` via `routing.resolve_lanes`), the owning role and team per domain, exactly one `verdict`, a `reasons` list of…
framing: Describes the author's own pipeline, so there's no external source to check it against.; evidence: This line is from an internal skill spec (.claude/commands/pr-review/SKILL.md). It describes the output schema of the PR author's own review pipeline: domains, verdict, reasons and gate_fails. The regex flagged "exactly one `verdict`" as…; source: .claude/commands/pr-review/SKILL.md:L115 (author's own design spec)
source: .claude/commands/pr-review/SKILL.md:L115 (author's own design spec)
framing: Describes the author's own pipeline, so there's no external source to check it against.
pass3 · numerical · high confidence
#4 ➖ Not a claim
.claude/commands/pr-review/SKILL.md:126
The stamp bar, **all required**: label `review:no-blockers` (`review:trivial` is not enough — no review ran; after a base merge the card itself saying…
evidence: This line is from an internal review-skill instruction file. It sets out the author's own stamp criteria (the labels and ⚠️ rows that must be present or absent before approval). The regex matched words like "after" and "legacy", but the…; source: repo:.claude/commands/pr-review/SKILL.md L126
source: repo:.claude/commands/pr-review/SKILL.md L126
pass1 · temporal · high confidence
#5 ➖ Not a claim
.claude/commands/pr-review/SKILL.md:132
**An unanswered 🚨 finding is `blocked`, not `judge`.** A judge row's other gates — size, `shape:infra`, a human author, an open ⚠️ — are an approver's call…
framing: The regex flagged this as an entity-spec, but it is internal skill guidance for Claude, not documentation making an external claim.; evidence: The line sets a rule for the PR author's own internal pr-review skill: how the `blocked` and `judge` states work, and when `--force` may be used. It describes their own design, not a third-party fact that anyone could check.; source: .claude/commands/pr-review/SKILL.md:L132
source: .claude/commands/pr-review/SKILL.md:L132
framing: The regex flagged this as an entity-spec, but it is internal skill guidance for Claude, not documentation making an external claim.
pass3 · entity-spec · high confidence
#6 ➖ Not a claim
.claude/commands/pr-review/SKILL.md:134
Reason codes are `code[:detail]` from a closed vocabulary (`analyze.REASON_CODES`, echoed into `queue.json`): `risk:`, `scrutiny:`, `ai-suspect:`, `review:`…
framing: Describes the author's own pipeline; no external source is cited.; evidence: The line describes the repo's own review tooling (the reason-code vocabulary in `scripts/review-v3/analyze.py` `REASON_CODES = {`). It makes no time-based or third-party assertion. The "temporal" regex hit most likely came from wording…; source: gh search code --repo pulumi/docs REASON_CODES → scripts/review-v3/analyze.py
source: gh search code --repo pulumi/docs REASON_CODES → scripts/review-v3/analyze.py
framing: Describes the author's own pipeline; no external source is cited.
pass1 · temporal · high confidence
#7 ➖ Not a claim
.claude/commands/pr-review/SKILL.md:193
- `--stamp N,N` — per PR, immediately before merging, re-fetch the PR: head unchanged since the plan, `mergeable_state` in {clean, blocked}, checks green…
framing: The regex matched 'PR' as a source hint, but this is internal workflow text, not a sourced claim.; evidence: The line describes the PR author's own `--stamp` merge procedure inside the pr-review skill: re-fetching the PR, gating on `mergeable_state`, and polling the Sentinel check on its own. It doesn't attribute a falsifiable assertion to any…; source: .claude/commands/pr-review/SKILL.md:L193 (describes the author's own design; no external source cited)
source: .claude/commands/pr-review/SKILL.md:L193 (describes the author's own design; no external source cited)
framing: The regex matched 'PR' as a source hint, but this is internal workflow text, not a sourced claim.
pass3 · attribution · medium confidence
#8 ➖ Not a claim
AGENTS.md:318
**The v3 surface (on repo-wide):** `REVIEW_V3_COMMENTS` is set for pulumi/docs, so every fresh review renders v3 cards; a PR reviewed before the flip keeps…
framing: This describes the repo's own review pipeline. Whether the REVIEW_V3_COMMENTS variable is set is repo config that the gh/read tools here can't see.; evidence: AGENTS.md L318 describes how the repo's own review pipeline behaves (the REVIEW_V3_COMMENTS flag, v3 cards, #new-review vs #update-review). The regex flagged it as temporal because of the word "flip", but it makes no third-party or…; source: repo:AGENTS.md (L318)
source: repo:AGENTS.md (L318)
framing: This describes the repo's own review pipeline. Whether the REVIEW_V3_COMMENTS variable is set is repo config that the gh/read tools here can't see.
pass1 · temporal · medium confidence
#9 ➖ Not a claim
scripts/review-v3/README.md:77
(latest `updated_at` wins, never a whole-block overwrite) against a freshly
evidence: This line is internal README text about the PR author's own review pipeline. It says how `apply-update.py` merges REVIEW_STATE findings: "It merges per finding-id (latest `updated_at` wins, never a whole-block overwrite)". It isn't a…; source: repo:scripts/review-v3/README.md L76-78
source: repo:scripts/review-v3/README.md L76-78
pass1 · temporal · high confidence
#10 ➖ Not a claim
scripts/review-v3/README.md:100
cancelled runner, a lost token, or a dispatch whose `workflow_run` cascade
evidence: This is a sentence fragment from an internal pipeline README (scripts/review-v3). It lists failure modes the author's own review pipeline handles: a cancelled runner, a lost token, or a dispatch whose `workflow_run` cascade fails. It…; source: scripts/review-v3/README.md:L100 (claim text only; no external source cited); intuition: The regex seems to have matched the `workflow_run` event name. If the full sentence claims GitHub behavior (e.g.…
source: scripts/review-v3/README.md:L100 (claim text only; no external source cited)
pass3 · entity-spec · medium confidence
#11 ➖ Not a claim
scripts/review-v3/README.md:366
| `analyze.py` | One verdict per PR (`stamp` / `judge` / `route` / `blocked`), reason codes (`REASON_CODES`), row actions — every verdict carries one…
framing: Tagged 'attribution' by the regex because of the `frontmatter-validate.build_global_ma...` reference, but it is an internal module/function name, not an…; evidence: This README table row describes what the PR author's own internal review script (`analyze.py`) does: its verdict set, reason codes, and collision clustering. It is not a third-party assertion, and the web can't check it. Any check would…; source: scripts/review-v3/README.md L366 (the author's own internal tooling documentation)
source: scripts/review-v3/README.md L366 (the author's own internal tooling documentation)
framing: Tagged 'attribution' by the regex because of the `frontmatter-validate.build_global_ma...` reference, but it is an internal module/function name, not an external source.
pass3 · attribution · medium confidence
#12 ➖ Not a claim
scripts/review-v3/README.md:368
| `act.py` | Plan → preview → execute (plan schema 2: every comment, review body and suggestion is rendered into the step at plan time, and execute sends…
framing: No external source is cited; "Every" is a sentence word, not a named source.; evidence: This README line describes how the PR author's own `act.py` script works (plan schema, preview/execute, preflight re-reads). It isn't a third-party attribution. The regex matched on the word "Every", which isn't a source here.; source: repo:scripts/review-v3/README.md L368
source: repo:scripts/review-v3/README.md L368
framing: No external source is cited; "Every" is a sentence word, not a named source.
pass3 · attribution · high confidence
#13 ➖ Not a claim
scripts/review-v3/staging-deploy.sh:8
# holding a runner for 45 minutes buys nothing, and a job that lives that
framing: The regex matched '45 minutes' inside a comment explaining the author's CI timeout reasoning.; evidence: This line is a code comment in the PR author's own staging-deploy script. It explains why the author chose a timeout ("holding a runner for 45 minutes buys nothing"). It describes the author's own pipeline design, not a third-party fact…; source: scripts/review-v3/staging-deploy.sh:L8
source: scripts/review-v3/staging-deploy.sh:L8
framing: The regex matched '45 minutes' inside a comment explaining the author's CI timeout reasoning.
pass3 · numerical · high confidence
#14 ➖ Not a claim
scripts/review-v3/test_analyze.py:192
'"sha":"","bulk":false,"updated_at":"2026-09-01T00:00:00Z"}},"high_water":4,"schema":1} -->' % actor)
framing: A regex matched test-fixture data (a timestamp and integer fields) inside a code string.; evidence: This line comes from a Python test fixture (test_analyze.py). It's a JSON string built for a test, with fields like sha, bulk, updated_at, high_water and schema. The numbers and the date are fixture values, not factual claims about the…; source: scripts/review-v3/test_analyze.py:L192
source: scripts/review-v3/test_analyze.py:L192
framing: A regex matched test-fixture data (a timestamp and integer fields) inside a code string.
pass3 · numerical · high confidence

Findings

No findings.

Editorial stances

Superlative, ranking, or comparative language the diff adds. No verdict — a page's own framing isn't fact-checkable — listed so a reviewer can confirm each is a stance the docs should take.

Investigation log

cross-sibling-reads
not run (not in a templated section)
external-claim-verification
0 of 14 claims verified (0 unverifiable, 0 contradicted) · 4 specialists (numerical, cross-reference, capability, framing); 0 cross-specialist corroborations · routed: 0 inline, 4 Pass 1, 0 Pass 2, 10 Pass 3 (verified 0, contradicted 0, unverifiable 10).
cited-claim-spot-checks
not run (no cited claims)
frontmatter-sweep
not run (no frontmatter in diff)
temporal-trigger-sweep
ran (recency words present in diff; spot-check in-review)
code-execution
not run (no `static/programs/` change)
code-examples-checks
not run (no fenced code blocks in content files)
editorial-balance-pass
not run (not under content/blog/)

History

  1. 2026-09-25T19:31:48Z 6eabda0

    initial review (pending publication)

  2. 2026-09-25T20:15:45Z 5e0c1fab5c

    5e0c1fa comment-only rewrite of REVIEW_STATE race notes checked against workflows and pinned-comment.sh; accurate

  3. 2026-09-25T20:29:31Z fecf1388a5

    Re-verified fecf138: #new-review exclusion moved from job if: to mention-gate live-text check; no new findings