Simplify WrightKit verification to a tests-first model

[Teakowa]

Goal

Adopt a conventional tests-first verification model across WrightKit and remove evidence / provenance as generic WrightKit engineering abstractions.

Compatibility remains a product/behavior requirement, but its verification is expressed through ordinary tests, expected outputs, reference comparisons, runtime/workflow validation, and focused integration coverage rather than a parallel evidence subsystem.

Context

WrightKit accumulated durable concepts such as evidence manifests, evidence classes, evidence lifecycle/admission rules, support/evidence matrices, recorded observations, generic provenance records, evidence-integrity tests, and evidence-first agent procedures.

Some underlying mechanisms remain valid:

• unit/integration/regression/compatibility/differential/real-project tests;
• test inputs and fixtures;
• reviewed expected outputs/goldens/snapshots;
• upstream/reference implementation comparisons;
• runtime or workflow validation;
• source mapping for diagnostics/editing;
• source attribution and licensing for third-party material;
• pinned versions/revisions, lockfiles, checksums, or commands where reproducibility requires them.

These concrete mechanisms should not be grouped into a separate WrightKit verification domain model.

The intended contributor model remains conventional:

cargo test
├─ unit tests
├─ integration tests
├─ regression tests
├─ compatibility tests
└─ real-project tests

Scope

• Make tests the primary durable repository verification mechanism.
• Treat compatibility as a behavior/contract concern verified by tests.
• Keep fixtures, corpora, goldens, snapshots, and reference implementations only as subordinate test-support mechanisms where useful.
• Remove generic evidence manifests, schemas, classes, result databases, lifecycle/admission systems, evidence-integrity frameworks, and equivalent machinery unless a concrete surviving product contract requires them.
• Remove provenance as an umbrella abstraction. Name the actual concern directly:
  • source mapping / source location;
  • source attribution;
  • licensing metadata;
  • pinned revision/version;
  • checksum/hash;
  • reproducibility;
  • related decision history.
• Keep current Git trees focused on current implementation, tests, test data, expected results, and durable contracts.
• Keep point-in-time run output, debugging observations, historical corrections, and completed-task history in CI, Issues/PRs, releases, or Git history.
• Audit organization-level policy and shared skills so they do not reintroduce the old model.
• Audit each core Rust repository and simplify owner-local verification structures without weakening meaningful behavioral coverage.
• Reconcile current/Accepted ADRs that still establish the old concepts.

Non-goals

• Removing meaningful compatibility, regression, real-project, differential, integration, or runtime tests.
• Removing expected-output files when they are the simplest assertion.
• Preventing use of upstream implementations as reference implementations.
• Removing source mapping required for diagnostics or editing.
• Removing third-party source attribution, licensing, or reproducibility metadata that a concrete workflow requires.
• Replacing the old model with another WrightKit-specific verification taxonomy.
• Changing semantic ownership boundaries.

Organization-level contract cleanup

The shared organization layer is part of this decision, not only the four Rust repositories.

Current org-level policy/docs cleanup is tracked in #61 and includes:

• AGENTS.md;
• GOAL.md;
• docs/agent-guidance.md;
• docs/testing-policy.md;
• docs/entropy-policy.md;
• docs/engineering-quality.md;
• docs/issue-readiness-and-pr-audit.md;
• docs/ci-platform.md;
• docs/rust-ci.md;
• docs/release-engineering.md.

The shared agent skills are separately owned by wrightkit/.agents and tracked in wrightkit/.agents#11.

Shared policy and skills must name concrete mechanisms such as tests, reference comparison, runtime/workflow validation, ablation, consumer inspection, source mapping, attribution, licensing, or reproducibility instead of generic evidence/provenance authority.

ADR handling

ADRs remain decision history.

If an ADR materially established evidence/provenance architecture:

1. preserve its historical body;
2. mark it Superseded or Partially Superseded;
3. point to the current tests-first replacement decision;
4. ensure current architecture/docs no longer route agents to the superseded model.

If a still-valid current/Accepted ADR merely uses old umbrella terminology, replace it with the concrete concept intended.

No current/Accepted ADR should declare a generic evidence/provenance model, hierarchy, lifecycle, authority, admission rule, evidence-backed decision criterion, or equivalent abstraction.

Historical Issue/PR references should be described as related/decision history, not as current verification metadata.

Ownership

Existing semantic ownership does not change:

• workshop-rs: canonical Workshop/WIR/catalog/settings/localization/validation/emission tests;
• opy-rs: OverPy syntax/semantic/compiler compatibility tests;
• deltin-rs: DEL/OSTW syntax/semantic/runtime/compiler compatibility tests;
• wright: Wright-owned product/provider/tooling/integration tests;
• wrightkit/.github: organization policy and CI guidance;
• wrightkit/.agents: shared agent skills.

Consumer repositories should not mirror an owner's full compatibility suite merely to possess a second copy of the same checks.

Acceptance Criteria

• Tests are the primary durable verification mechanism across current organization policy and owner repositories.
• Compatibility is described as a behavior/contract concern whose checks live in ordinary tests.
• Current policy, skills, docs, and Accepted/current ADRs do not establish generic evidence/provenance architecture.
• docs/testing-policy.md no longer defines evidence classes, lifecycle, admission, or canonical evidence corpora as a parallel system.
• Shared skills no longer describe procedures as evidence-first or define evidence lifecycle/admission/result authority.
• Verification procedures use concrete terms: tests, expected output, reference comparison, runtime/workflow validation, ablation, consumer inspection, source mapping, source attribution, licensing, pinned revision/version, checksum, or reproducibility.
• Point-in-time task/run/history records are not committed as current test metadata unless an executable current contract directly consumes them.
• Meaningful behavioral coverage survives simplification.
• Source mapping, attribution/licensing, and reproducibility information remain where concretely required.
• No replacement generic verification taxonomy is introduced.
• Normal contributor entry points remain conventional repository test commands such as cargo test, with specialized runners only where behavior genuinely requires them.

Follow-up Issues

Organization layer:

• Replace org-level evidence and provenance policy with tests-first guidance #61 — org-level policy/docs cleanup
• Remove generic evidence and provenance concepts from shared skills .agents#11 — shared agent skill cleanup

Owner repositories:

• Finish tests-first conformance cleanup and retire evidence contracts workshop-rs#241
• Collapse OPY compatibility evidence metadata into ordinary differential tests opy-rs#337
• Replace the DEL/OSTW evidence model with ordinary compatibility tests deltin-rs#112
• Remove duplicated compatibility evidence storage and keep Wright-owned integration tests wright#371

Cross-repo implementation order should continue to respect ownership and dependencies rather than allowing consumer cleanup to redefine owner semantics.