Review evidence

pulumi/docs PR #21440 7916fb43d3 generated 2026-09-17T00:45:55Z

Verification trail

#2 ✅ Verified
content/docs/best-practices/_index.md:22
The 'Organizing projects and stacks' guide at /docs/iac/guides/basics/organizing-projects-stacks/ exists and covers deciding how to split infrastructure…
evidence: The file content/docs/iac/guides/basics/organizing-projects-stacks/_index.md exists at that URL path and its content matches the claim: "No single structure is correct for every team. The right choice depends on tradeoffs, so this guide…; source: repo:content/docs/iac/guides/basics/organizing-projects-stacks/_index.md
source: repo:content/docs/iac/guides/basics/organizing-projects-stacks/_index.md
pass1
#4 ✅ Verified
content/docs/best-practices/_index.md:25
The Components documentation should be used when a group of resources is provisioned together repeatedly and deserves a single, reusable abstraction.
evidence: The Components page defines: "A component is a logical grouping of Pulumi resources that is exposed as a single Pulumi resource. Components encapsulate related resources and their configuration, letting consumers create complex…; source: repo:content/docs/iac/concepts/components/_index.md
source: repo:content/docs/iac/concepts/components/_index.md
pass1
#5 ✅ Verified
content/docs/best-practices/_index.md:28
- [Building and extending Pulumi](/docs/iac/guides/building-extending/) — reach for
evidence: The page /docs/iac/guides/building-extending/_index.md exists and covers building/packaging components, providers, packages, and templates for reuse across teams or the community, matching the link text and description "reach for this…; source: repo:content/docs/iac/guides/building-extending/_index.md
source: repo:content/docs/iac/guides/building-extending/_index.md
pass1
#6 ✅ Verified
content/docs/best-practices/_index.md:34
The 'Configuration' guide at /docs/iac/concepts/config/ exists and covers structuring per-stack settings so the same program can run safely against dev…
evidence: The page pulumi/docs:content/docs/iac/concepts/config.md exists, serves at /docs/iac/concepts/config/, and states: "Different stacks for a single project often need different values. You might want a different size for your AWS EC2…; source: gh api repos/pulumi/docs/contents/content/docs/iac/concepts/config.md
source: gh api repos/pulumi/docs/contents/content/docs/iac/concepts/config.md
pass1
#7 ✅ Verified
content/docs/best-practices/_index.md:36
The Configuration documentation should be used when structuring per-stack settings so the same program can run safely against dev, staging, and production.
evidence: The Configuration concepts page (content/docs/iac/concepts/config.md) opens: "Different stacks for a single project often need different values. You might want a different size for your AWS EC2 instance, or a different number of servers…; source: gh api repos/pulumi/docs/contents/content/docs/iac/concepts/config.md
source: gh api repos/pulumi/docs/contents/content/docs/iac/concepts/config.md
pass1
#8 ✅ Verified
content/docs/best-practices/_index.md:39
The 'ESC secrets-rotation best practices' guide at /docs/esc/operations/rotation/best-practices/ exists and covers who rotates a credential, how often, and…
framing: Source covers least privilege/role permissions (implies "who"), rotation-schedule grouping (implies "how often"), and composition/import propagation — the…; evidence: The page exists at content/docs/esc/operations/rotation/best-practices.md (linked from sibling docs) and covers least privilege for the rotating user/role ("who rotates"), organizing environments by rotation schedule ("It is recommended…; source: gh api repos/pulumi/docs/git/blobs/8a20a37fda6606a89aef2e38d5946f74e8c6c387 (content/docs/esc/operations/rotation/best-practices.md)
source: gh api repos/pulumi/docs/git/blobs/8a20a37fda6606a89aef2e38d5946f74e8c6c387 (content/docs/esc/operations/rotation/best-practices.md)
pass1
#9 ✅ Verified
content/docs/best-practices/_index.md:42
- [ESC environment composition patterns](/docs/esc/guides/environment-composition-patterns/) —
evidence: The linked page exists in the pulumi/docs repo at content/docs/esc/guides/environment-composition-patterns.md, which resolves to the URL path /docs/esc/guides/environment-composition-patterns/ referenced in the claim.; source: gh api repos/pulumi/docs/contents/content/docs/esc/guides/environment-composition-patterns.md
source: gh api repos/pulumi/docs/contents/content/docs/esc/guides/environment-composition-patterns.md
pass1
#10 ✅ Verified
content/docs/best-practices/_index.md:48
The Testing guide should be used to decide which layer to test at: unit tests against a program's logic, or integration tests against real provisioned…
framing: Source describes three testing styles (unit, property, integration); claim's summary of "unit vs integration" is a narrower but accurate restatement of the…; evidence: The linked Testing guide (/docs/iac/guides/testing/) states unit tests are "fast in-memory tests that mock all external calls" while integration tests "deploy ephemeral infrastructure and run external tests against it" / "the tests…; source: content/docs/iac/guides/testing/_index.md
source: content/docs/iac/guides/testing/_index.md
pass1
#11 ✅ Verified
content/docs/best-practices/_index.md:48
The 'Automation API' documentation at /docs/iac/concepts/automation-api/ exists and covers driving Pulumi programmatically instead of through the CLI, such…
evidence: The page content/docs/iac/concepts/automation-api.md (renders to /docs/iac/concepts/automation-api/) exists and states: "The Pulumi Automation API is a programmatic interface for running Pulumi programs without the Pulumi CLI... so you…; source: gh api repos/pulumi/docs/contents/content/docs/iac/concepts/automation-api.md
source: gh api repos/pulumi/docs/contents/content/docs/iac/concepts/automation-api.md
pass1
#12 ✅ Verified
content/docs/best-practices/_index.md:51
The Automation API should be used when a workflow needs to drive Pulumi programmatically instead of through the CLI, such as a self-service provisioning tool…
framing: Source lists several use cases (CI/CD, testing, custom CLIs, REST/gRPC APIs); claim cites two of them as examples, which is a valid narrower restatement.; evidence: The automation-api doc states Automation API "encapsulates the functionality of the CLI... as a strongly typed SDK, so you can drive the Pulumi engine from within your own application instead of invoking the pulumi command from a shell,"…; source: gh api repos/pulumi/docs/contents/content/docs/iac/concepts/automation-api.md
source: gh api repos/pulumi/docs/contents/content/docs/iac/concepts/automation-api.md
pass1
#13 ✅ Verified
content/docs/best-practices/_index.md:57
The policy CI/CD integration guide should be used when policy checks need to run automatically in a pull request or a deployment pipeline, before…
evidence: The linked page content/docs/insights/policy/ci-cd.md confirms this exactly: "Pulumi policies integrate with CI/CD pipelines to automatically enforce compliance and security rules on every deployment... Non-compliant changes are blocked…; source: repo:content/docs/insights/policy/ci-cd.md
source: repo:content/docs/insights/policy/ci-cd.md
pass1
#14 ✅ Verified
content/docs/best-practices/_index.md:57
The 'Discovery & governance: policy CI/CD integration' guide at /docs/insights/policy/ci-cd/ exists and covers running policy checks automatically in a pull…
evidence: The page content/docs/discovery-governance/policy/ci-cd.md exists, is aliased to /docs/insights/policy/ci-cd/, and its content matches the claim: "Pulumi policies integrate with CI/CD pipelines to automatically enforce compliance and…; source: gh api repos/pulumi/docs/contents/content/docs/discovery-governance/policy/ci-cd.md (aliases: /docs/insights/policy/ci-cd/)
source: gh api repos/pulumi/docs/contents/content/docs/discovery-governance/policy/ci-cd.md (aliases: /docs/insights/policy/ci-cd/)
pass1
#16 ✅ Verified
content/docs/best-practices/_index.md:66
The 'IDP patterns' section at /docs/idp/guides/best-practices/#patterns exists and covers named, proven shapes for common platform problems, such as one ESC…
evidence: The page content/docs/idp/guides/best-practices/_index.md contains a "## Patterns" section (anchor #patterns) listing "IDP Pattern: One ESC environment per service", "...per team", and "IDP Pattern: Cost control using Components…; source: repo:content/docs/idp/guides/best-practices/_index.md
source: repo:content/docs/idp/guides/best-practices/_index.md
pass1
#17 ✅ Verified
content/docs/best-practices/_index.md:69
- [IDP patterns](/docs/idp/guides/best-practices/#patterns) — reach for this when you
evidence: The target page content/docs/idp/guides/best-practices/_index.md contains a heading `## Patterns` (line 24), which renders as an anchor `#patterns`, matching the linked URL /docs/idp/guides/best-practices/#patterns exactly.; source: repo:content/docs/idp/guides/best-practices/_index.md
source: repo:content/docs/idp/guides/best-practices/_index.md
pass1
#19 ✅ Verified
content/docs/best-practices/_index.md:79
- [Security hardening for self-hosted deployments](/docs/administration/self-hosting/operations/security-hardening/) —
evidence: The target page exists at pulumi/docs:content/docs/administration/self-hosting/operations/security-hardening.md (identifier: administration-security-compliance-self-hosted-operations-security-hardening), and is also linked from…; source: gh search code --owner pulumi security-hardening (pulumi/docs:content/docs/administration/self-hosting/operations/security-hardening.md)
source: gh search code --owner pulumi security-hardening (pulumi/docs:content/docs/administration/self-hosting/operations/security-hardening.md)
pass1
#1 ➖ Not a claim
content/docs/best-practices/_index.md:9
Pulumi publishes best-practices guidance throughout its documentation, located close to the capability each pattern applies to, rather than centralized in…
evidence: The claim restates the page's own stated rationale for its existence: "Pulumi's best-practices guidance lives close to the capability it applies to. This page indexes it in one place..." This is the author describing their own…; source: repo:content/docs/best-practices/_index.md
source: repo:content/docs/best-practices/_index.md
pass1
#3 ➖ Not a claim
content/docs/best-practices/_index.md:23
reach for this when deciding how to split infrastructure across Pulumi projects and
evidence: The text is a fragment of introductory/navigational prose describing when to use the best-practices doc section, not a falsifiable factual assertion.; source: content/docs/best-practices/_index.md
source: content/docs/best-practices/_index.md
pass3
#20 🤷 Unverifiable
content/docs/best-practices/_index.md:88
There is a tracking issue at https://github.com/pulumi/docs/issues/16455 discussing the open question of this best-practices index page's placement in the…
framing: Cannot confirm the issue's actual content/topic from the truncated pre-fetched body; HTTP 200 only confirms the issue page loads, not its subject matter.; evidence: The pre-fetched body only contains a stylesheet link tag fragment from GitHub's SPA shell; no issue title, body, or content confirming the described discussion topic could be retrieved from this truncated payload.; source: https://github.com/pulumi/docs/issues/16455
source: https://github.com/pulumi/docs/issues/16455
pass2
#21 🤷 Unverifiable
content/docs/best-practices/_index.md:88
There is a tracking issue at https://github.com/pulumi/docs/issues/16455 discussing the placement of this best-practices index page in the docs navigation.
evidence: The pre-fetched page returned HTTP 200 but the body is only a truncated fragment of GitHub's asset/stylesheet header; no issue title, description, or discussion content is present to confirm the issue's subject matter (placement of the…; source: https://github.com/pulumi/docs/issues/16455; intuition: The truncated body gives no evidence of the issue's actual content, so the specific claim about what it discusses…
source: https://github.com/pulumi/docs/issues/16455
pass2
#15 ❌ Contradicted
content/docs/best-practices/_index.md:60
- [Discovery & governance: policy CI/CD integration](/docs/insights/policy/ci-cd/) —
framing: The old URL likely still 301-redirects to the new path, but it's a stale link to a superseded slug rather than the current canonical doc location the other…; evidence: The content has been moved: the canonical page is now content/docs/discovery-governance/policy/ci-cd.md, and /docs/insights/policy/ci-cd/ is only retained as a legacy redirect alias listed in that file's frontmatter ("…; source: gh search code --repo pulumi/docs "policy/ci-cd" (content/docs/discovery-governance/policy/ci-cd.md redirect list; content/docs/discovery-governance/policy/_index.md link)
source: gh search code --repo pulumi/docs "policy/ci-cd" (content/docs/discovery-governance/policy/ci-cd.md redirect list; content/docs/discovery-governance/policy/_index.md link)
pass1
#18 🌀 Framing drift
content/docs/best-practices/_index.md:76
The 'Setting up for success' guide at /docs/administration/get-started/setting-up-for-success/ exists and covers standing up Pulumi for an organization for…
framing: shifted — source page's actual scope is security/testing/code-reuse decisions; claim describes it as covering team/project/access structuring decisions…; evidence: The page /docs/administration/get-started/setting-up-for-success/ exists and is titled "Setting up for success" with meta_desc "Make key decisions about security, testing strategies, and code reusability that will set your team up for…; source: gh api repos/pulumi/docs/contents/content/docs/administration/get-started/setting-up-for-success.md
source: gh api repos/pulumi/docs/contents/content/docs/administration/get-started/setting-up-for-success.md
pass1

Findings

IDBucketFile:linesStatusDisposition
F2 ❓ Author answer content/docs/best-practices/_index.md:88-89 open —
F4 ⚠️ Reviewer check content/docs/best-practices/_index.md:76-81 open —

Editorial stances

Superlative, ranking, or comparative language the diff adds. No verdict — a page's own framing isn't fact-checkable — listed so a reviewer can confirm each is a stance the docs should take.

Investigation log

cross-sibling-reads
not run (not in a templated section)
external-claim-verification
15 of 21 claims verified (2 unverifiable, 1 contradicted, 1 framing-drift) · 4 specialists (numerical, cross-reference, capability, framing); 0 cross-specialist corroborations · routed: 0 inline, 18 Pass 1, 2 Pass 2 (verified 0, contradicted 0, unverifiable 2), 1 Pass 3 (verified 0, contradicted 0, unverifiable 1).
cited-claim-spot-checks
2 of 2 cited claims fetched and compared
frontmatter-sweep
ran on body + meta_desc
temporal-trigger-sweep
ran (recency words present in diff; spot-check in-review)
code-execution
not run (no `static/programs/` change)
code-examples-checks
not run (no fenced code blocks in content files)
editorial-balance-pass
not run (not under content/blog/)

Triaged

History

  1. 2026-09-17T00:45:55Z 7916fb4

    initial review (pending publication)