# FORGE evidence research — operational gate #292

> **Cut:** 2026-07-17T23:24:33Z
>
> **Executor:** `forge.public-evidence-fetch/v1`, merged in
> [AI Env #63](https://github.com/needyuai/trustyu-ai-env/pull/63) at
> `fb43ea3d722399de7c2f3285d070f2e14b8c7674`
>
> **Scope:** framework-only; no CRM, Hub Agents or Needyu 2.0 change
>
> **Issue:** [Docs #292](https://github.com/needyuai/trustyu-docs/issues/292)

## Outcome

The public evidence-intake gate was exercised rather than closed from a fixture or an offline schema:

| Measure | Result |
|---|---:|
| approved public sources | **20/20** |
| independent publisher groups | **11** |
| reproducible multi-source researches | **3/3** |
| structurally grounded claims | **9/9** |
| source-bound receipts | **20/20** |
| bytes received | **423,265** |
| HTTP calls | **21** |
| measured fetch time, summed | **37,675 ms** |
| model calls / tokens | **0 / 0** |
| direct model cost | **US$ 0** |
| raw external content persisted | **none** |

The source mix is 3 normative, 1 empirical-primary and 16 official-primary records. Each claim has
support from at least two sources in at least two independent publisher groups. The three researches
span reliable harness engineering, secure/auditable delivery, and evals + observability + unit
economics.

This result proves the **intake and research mechanism** for the selected snapshots. It does not prove
that FORGE is operational or attested across products, and it does not turn a self-digested receipt
into independent attestation.

## Evidence layout

| Artifact | Role |
|---|---|
| [`source-catalog-v1.json`](source-catalog-v1.json) | reviewed source, quality tier, recency, publisher group, topics, immutable snapshot/hash and usage basis |
| [`manifests/`](manifests/) | 20 closed `forge.evidence-intake/v1` requests with deny-by-default capabilities and zero-model budget |
| [`receipts/`](receipts/) | 20 source-bound trusted-fetch receipts; no retrieved or derived text |
| [`research-pack-v1.json`](research-pack-v1.json) | three questions, nine claims, source/location mappings, confidence, limitations and official billing boundaries |
| [`scripts/canon/evidence_research.py`](../../../../../../scripts/canon/evidence_research.py) | deterministic catalog, manifest, receipt, corroboration, billing and cost validator |

The receipts bind request URL, final URL, publisher, snapshot, content SHA-256, derived-text SHA-256,
DNS answer-set digest, policy checks, retention and measured cost. External bytes existed only in the
trusted fetcher's memory and were never committed.

## The three conclusions

### R1 — reliable engineering harness

Public evidence converges on a system, not a better prompt: explicit task/workspace lifecycle,
isolated execution, small reviewable changes, observable feedback, explicit team orchestration and
durable architectural decisions. FORGE should preserve spec-driven worktrees and ADRs while enforcing
the missing isolation and feedback boundaries mechanically.

### R2 — secure and auditable delivery

Measurement/verification, least privilege and provenance are complementary. Reproducible AI tests do
not replace a sandbox; a sandbox does not replace source-bound provenance; an attestation does not
prove semantic correctness. FORGE therefore keeps these as separate gates.

### R3 — evals, observability and unit economics

Quality requires explicit eval criteria plus observable run evidence. Cost becomes useful only when
related to a value unit. Nonzero provider spend must come from official billing interfaces rather than
token estimates or an invented zero.

## Official provider billing boundary

| Provider | Official source received | State in this run |
|---|---|---|
| OpenAI | organization Admin Usage Costs API in the official SDK | documented, **not configured**; no model call or admin credential |
| Anthropic | organization Usage and Cost API in the official cookbook | documented, **not configured**; no model call or admin credential |
| Google | standard/detailed usage-cost export in the official Google Cloud billing-dashboard blueprint | documented, **not configured**; no Gemini call or billing export |

`documented-not-configured` is intentional. This framework-only intake used no model, so its model cost
is measured at zero. Future product/model cost remains **unknown** until an authorized provider export
exists and is reconciled by project/SKU. The `oracle` pattern remains `defer`; this gate does not grant
credentials or authorize multi-provider model spend.

## Fail-closed observations

The positive corpus was not obtained by weakening policy:

- OpenAI marketing pages returned HTTP 403 to automated retrieval; equivalent official repository
  documents were pinned by full commit instead of using cookies or a browser profile.
- public Anthropic and security pages that matched DLP or prompt-injection patterns were excluded from
  the positive receipt set; no exception or text persistence was introduced.
- live NIST, Gemini and OWASP pages changed bytes between review and execution and were denied with
  `content-hash-mismatch`; they were replaced by official, immutable NIST Dioptra, Google Cloud Billing
  export and OWASP AISVS snapshots.

These observations are not counted as successful sources. Their value is demonstrating that expected
denials remain denials.

## Reproduce the validation

From the `trustyu-docs` repository:

```bash
python3 scripts/canon/evidence_research.py validate
python3 -m unittest scripts/canon/tests/test_evidence_research.py -v
```

To acquire a new snapshot, review and update the catalog hash first, generate a new dated evidence
directory, and run the AI Env trusted fetcher against each generated manifest. Never overwrite this
receipt set: changed external bytes are a new observation, not a correction to history.

## Claim boundary

Allowed conclusion: **the FORGE public evidence-intake and multi-source research gate was exercised
successfully for 20 reviewed sources at this cut.**

Not allowed from this pack: `Operational`, `Attested`, “100%”, certified, complete provider FinOps,
product adoption, customer outcome, or world-leading performance. Those remain dependent on the
product, time, economic and independent-assurance gates in the canonical benchmark.