Review evidence — pulumi/docs #21952

Verification trail

#1 ✅ Verified

scripts/weekly-digest/rank.py:29

API_URL = "https://api.anthropic.com/v1/messages"

framing: A URL constant in a script, not prose. The /v1/messages path is confirmed by secondary sources. None of the fetched results showed Anthropic's own docs…; evidence: Several third-party docs describe the Anthropic Messages API as "POST /v1/messages - Create messages" (Vercel AI Gateway), and LM Studio's compat page links Anthropic's own messages/create reference. That matches the script constant…; source: https://vercel.com/docs/ai-gateway/sdks-and-apis/anthropic-messages-api; https://lmstudio.ai/docs/developer/anthropic-compat/messages

source: https://vercel.com/docs/ai-gateway/sdks-and-apis/anthropic-messages-api; https://lmstudio.ai/docs/developer/anthropic-compat/messages

framing: exact-match — A URL constant in a script, not prose. The /v1/messages path is confirmed by secondary sources. None of the fetched results showed Anthropic's own docs quoting the api.anthropic.com host.

pass3 · url · medium confidence

#2 ✅ Verified

scripts/weekly-digest/rank.py:48

"anthropic-version": "2023-06-01",

framing: The claim is an HTTP header value in script code; it matches the version string Anthropic documents.; evidence: Anthropic's official versioning docs say the anthropic-version header is required and give "For example, `anthropic-version: 2023-06-01`." Anthropic's current (2026) beta-headers docs use the same value, so it is still the supported API…; source: https://platform.claude.com/docs/en/api/versioning

source: https://platform.claude.com/docs/en/api/versioning

framing: exact-match — The claim is an HTTP header value in script code; it matches the version string Anthropic documents.

pass3 · numerical · high confidence

#3 ✅ Verified

scripts/weekly-digest/render.py:27

REPO_URL = "https://github.com/pulumi/docs"

evidence: The GitHub API resolves the repo as "pulumi/docs https://github.com/pulumi/docs", so the REPO_URL constant points to a live repository with the correct name.; source: gh api repos/pulumi/docs --jq '.full_name + " " + .html_url'

source: gh api repos/pulumi/docs --jq '.full_name + " " + .html_url'

pass1 · url · high confidence

#4 ➖ Not a claim

scripts/weekly-digest/render.py:121

# Most overdue relative to its own SLA; a 1bd tools SLA blown by 3 days

framing: Code comment with an example ("1bd tools SLA blown by 3 days"). The regex matched the numbers, but nothing here can be checked against an outside source.; evidence: This is a code comment in the PR author's own script (scripts/weekly-digest/render.py). It gives a made-up example of the script's own sorting logic: rank items by how overdue they are against their own SLA. It is not a factual claim…; source: scripts/weekly-digest/render.py:L121 (code comment describing the author's own design)

source: scripts/weekly-digest/render.py:L121 (code comment describing the author's own design)

framing: Code comment with an example ("1bd tools SLA blown by 3 days"). The regex matched the numbers, but nothing here can be checked against an outside source.

pass3 · numerical · high confidence

#5 ➖ Not a claim

scripts/weekly-digest/test_digest.py:60

"generated_at": "2026-09-28T14:00:00+00:00",

framing: The regex flagged an ISO-8601 datetime literal inside a test fixture dict. It is sample data, not a claim.; evidence: This line is a hardcoded fixture timestamp in a unit-test file (test_digest.py) for the weekly-digest script. It is sample input data for the test, not a factual assertion that anyone could check.; source: scripts/weekly-digest/test_digest.py:L60

source: scripts/weekly-digest/test_digest.py:L60

framing: The regex flagged an ISO-8601 datetime literal inside a test fixture dict. It is sample data, not a claim.

pass3 · numerical · high confidence

#6 ➖ Not a claim

scripts/weekly-digest/test_digest.py:74

{"workflow": "Audit Logs", "runs": 4, "failures": 4, "streak": 4, "last_failure_url": "https://x/run/1"},

framing: Regex picked up a placeholder URL in synthetic test data; there's nothing to check.; evidence: This line is a test fixture dict in a Python unit test (test_digest.py). "https://x/run/1" is a placeholder URL in mock data, not a real citation or factual assertion.; source: scripts/weekly-digest/test_digest.py:L74-75

source: scripts/weekly-digest/test_digest.py:L74-75

framing: Regex picked up a placeholder URL in synthetic test data; there's nothing to check.

pass3 · url · high confidence

#7 ➖ Not a claim

scripts/weekly-digest/test_digest.py:83

"merged_with_outstanding": [{"pr": 30, "title": "Reorg", "url": "https://github.com/pulumi/docs/pull/30",

framing: Synthetic test-fixture URL; the regex extractor flagged it as a citation.; evidence: The line is from a unit-test fixture dict in scripts/weekly-digest/test_digest.py. The PR number 30, the title "Reorg", and the URL are placeholder test data for the digest script, not a factual reference to real PR content.; source: repo:scripts/weekly-digest/test_digest.py L83

source: repo:scripts/weekly-digest/test_digest.py L83

framing: Synthetic test-fixture URL; the regex extractor flagged it as a citation.

pass1 · url · high confidence

#8 ➖ Not a claim

scripts/weekly-digest/test_digest.py:125

{"workflowName": "A", "status": "completed", "conclusion": "success", "createdAt": "2026-09-20", "url": "u0"},

framing: The regex extractor flagged a date string inside synthetic test data.; evidence: This line is a test fixture in a Python unit test (scripts/weekly-digest/test_digest.py). It holds a mock GitHub workflow run record with a made-up date ("2026-09-20") and placeholder values ("A", "u0"). It doesn't assert any fact, so…; source: scripts/weekly-digest/test_digest.py:L125-131

source: scripts/weekly-digest/test_digest.py:L125-131

framing: The regex extractor flagged a date string inside synthetic test data.

pass3 · numerical · high confidence

#9 ➖ Not a claim

scripts/weekly-digest/test_digest.py:198

assert line.split(": ", 2)[2].startswith("<https://github.com/pulumi/docs/pull/45|#45> red CI")

framing: Test fixture string in the PR author's own test code; the regex matched a URL inside a Python assert.; evidence: The line is a unit-test assertion in scripts/weekly-digest/test_digest.py. It checks that the digest formatter renders a fixture PR (#45) as a Slack-style link. The URL is synthetic test data, not an assertion about the real pulumi/docs…; source: repo:scripts/weekly-digest/test_digest.py L198

source: repo:scripts/weekly-digest/test_digest.py L198

framing: Test fixture string in the PR author's own test code; the regex matched a URL inside a Python assert.

pass1 · url · high confidence

Findings

No findings.

Editorial stances

Superlative, ranking, or comparative language the diff adds. No verdict — a page's own framing isn't fact-checkable — listed so a reviewer can confirm each is a stance the docs should take.

scripts/weekly-digest/rank-prompt.md L12 1. Size of the breach relative to the SLA (5 business days on a 1-day SLA beats 5 on a 3-day SLA). comparison · found by regex

Investigation log

cross-sibling-reads: not run (not in a templated section)
external-claim-verification: 3 of 9 claims verified (0 unverifiable, 0 contradicted) · 4 specialists (numerical, cross-reference, capability, framing); 0 cross-specialist corroborations · routed: 0 inline, 3 Pass 1, 0 Pass 2, 6 Pass 3 (verified 2, contradicted 0, unverifiable 4).
cited-claim-spot-checks: not run (no cited claims)
frontmatter-sweep: not run (no frontmatter in diff)
temporal-trigger-sweep: ran (recency words present in diff; spot-check in-review)
code-execution: not run (no `static/programs/` change)
code-examples-checks: not run (no fenced code blocks in content files)
editorial-balance-pass: not run (not under content/blog/)

History

2026-09-28T19:29:43Z 33ef65f
initial review (pending publication)
2026-09-28T20:43:09Z 0251e0fdeb
Re-verified 0251e0fd (C&C fixes in render/rank/digest + 7 new tests); no open findings, nothing new raised
2026-09-28T21:07:29Z 6b62478ec1
Re-checked 6b62478: SLA-unavailable footer now says "other open"; rank.py never-fatal test added. No new findings.
2026-09-28T22:29:01Z c8d4497764
Re-verified c8d4497 (Sentinel ready list, team queue links, blog line removed); no new issues
2026-09-28T23:11:12Z dde18633b5
Re-verified dde1863 (Slack thread reply for capped lists); no new issues, no open findings