ProofPilot

Evidence-first open-source guidance for turning uncertainty into a defensible venture decision.

ProofPilot helps a founder, small-business owner, student, community operator, technical builder, or non-technical entrepreneur discover an opportunity, validate it, plan the smallest useful test, review readiness, and prepare a pitch or application. It is not limited to software or hackathons.

What Makes It Different

ProofPilot is designed to resist confident but unsupported startup advice:

• facts, user claims, assumptions, inferences, and unknowns stay separate
• material external claims require direct sources and retrieval dates
• missing evidence produces a provisional conclusion, not a fabricated opportunity
• each experiment specifies success, inconclusive, and stop or pivot conditions
• Colosseum is the required research foundation; other service connections are optional
• coaching and formal evaluation use separate modes

How It Routes Work

ProofPilot classifies a request across independent dimensions instead of forcing it into one track:

• mode: coach or evaluator
• stages: discover, validate, plan, review, submit
• domains: web2, AI, web3, data/ML, community, no-code, physical-world business, or deep tech
• venture_type: startup, small business, public good, nonprofit, internal tool, or open source
• program_context: none, hackathon, grant, accelerator, incubator, or competition
• sensitivity: public, team, confidential, or restricted

A single request can run several stages. An AI and web3 startup preparing for a grant is no longer reduced to one ambiguous track.

Example Request

I do not know what to build.
I am good at sales and community, but I am not a developer.
I have two weeks and one technical friend.
Find directions I can test, check the riskiest assumptions, and plan the smallest MVP.

The response should include evidence and unknowns, three to five distinct directions, one recommended test, a measurable threshold, a stop condition, sources checked and not checked, credential requirements, and the next actions.

Install And Inspect

Requirements: Node.js 20 or later and npm. Live Colosseum access also requires curl.

Clone the repository and install dependencies:

npm install
npm test
node scripts/cli.js inspect

Install the self-contained skill for a supported agent runtime. Choose one target:

node scripts/cli.js install --target codex
node scripts/cli.js install --target claude
node scripts/cli.js install --target agents

Use --dir <path> for an exact custom destination and --force only when replacing an existing installation intentionally.

The CLI installs and validates the skill package. The selected agent runtime executes skills/proofpilot/SKILL.md. Restart or reopen the runtime after installation so it discovers the new skill.

First-Use Setup

Both repository installers print the next setup step. A plain skill installation cannot initiate an agent conversation on its own: invoke $proofpilot after installation, and the agent will check existing access and explain setup in your language. All six entrypoints share the same setup.

For higher-quality recommendations and assessments, ProofPilot recommends models in the SOL or Opus 5 class or higher, where available in your host. Weaker models may miss important details or draw incorrect conclusions; built-in checks cannot fully compensate for model limitations. This is a usage recommendation, not a benchmark guarantee or a setup requirement. Model access and costs depend on your host's plan.

Colosseum Copilot access is required for completed setup and is free for Arena members. Open Arena → Copilot to generate a PAT if you do not have one. The agent reuses existing credentials. Enter a new PAT directly into your own interactive terminal, never into chat or command arguments:

# Replace the directory with your actual installed skill path.
node "<installed skill directory>/scripts/setup.js" --configure-colosseum
node "<installed skill directory>/scripts/setup.js" --check-colosseum

proofpilot setup --status also works when the repository CLI is installed on PATH. Status is offline; --check-colosseum performs one read-only status request using curl. The local credential file is private (mode 600), not encrypted; a host-managed environment secret is also supported. Read onboarding for storage, renewal, error handling, and the explicit limited/offline-work exception.

The service access guide covers ten services: how to connect, what is free, and where charges begin. Other keys are requested only for a concrete capability. Additional OpenAI/Claude API keys are unnecessary just to use ProofPilot in an already authenticated agent. Host/model usage is billed separately from free Colosseum access.

Quick Start

In Codex, invoke ProofPilot explicitly with $proofpilot:

$proofpilot

I do not know what to build. I have two weeks, access to university communities,
and no technical cofounder. Find directions worth testing and plan the smallest
experiment that could disprove the strongest idea.

Other supported runtimes can invoke the installed proofpilot skill through their normal skill picker or invocation syntax. ProofPilot may also activate from a matching founder, validation, MVP, review, pitch, grant, accelerator, or hackathon request.

Optional Stage Profiles

The repository also ships five opt-in stage profiles. They are not installed by the default CLI because the main skill already routes every stage and broad implicit triggers can compete. Install them deliberately when a runtime benefits from direct stage invocation:

Profile	Direct stage
proofpilot-idea-discovery	Discover opportunities from constraints and access
proofpilot-venture-validation	Test demand and risky assumptions
proofpilot-mvp-planner	Plan the smallest useful experiment or MVP
proofpilot-readiness-review	Review evidence and readiness with a rubric
proofpilot-submission-builder	Prepare a pitch, grant, accelerator, or hackathon application

For a fresh Codex installation, use the profile installer instead of the main-only install --target codex command above. It installs the main router and all five profiles together:

npm run install:profiles -- --copy
npm run validate:profiles

The profile installer targets $CODEX_HOME/skills or ~/.codex/skills by default. Pass --target <skill-root> for another runtime. Add --force only when intentionally replacing an existing ProofPilot installation. The profiles share the canonical references from skills/proofpilot/references; they do not maintain a second registry.

Focused Extensions

The self-contained skill includes focused playbooks in the v0.3 evidence model:

• Solana tasks can use solana-new.md to discover installed journey skills, local knowledge, scaffold guidance, Colosseum context, and DefiLlama research paths.
• Accelerator work uses current program profiles for YC, Techstars, 500 Global, Antler, Entrepreneurs First, and Sequoia Arc, while requiring official-page refresh before final advice.
• Pitch and presentation work selects a hackathon, investor, angel, accelerator, grant, or partner deck before drafting.
• honest-evaluation.md calibrates positive and negative verdicts to evidence and the actual program stage.
• product-market-fit.md adapts Sequoia questions and archetypes into value, behavior, retention, and payment tests. Methodology is not project evidence.
• ai-product-validation.md checks accepted outcomes, repeated trials, human fallback, and full variable delivery cost.
• decisions.md separates test, build, application, and artifact decisions; mandatory gates remain independent of quality scores.
• quality.md applies to substantial prose assessments too: freeze source-backed facts, check arithmetic and gates, review exact claims, and permit at most two repairs. A narrow edit keeps only applicable checks.
• Source playbooks start relevant project/archive research with Colosseum, then add local, public and optional domain sources. Corpus coverage and the user's evidence remain explicit.

Inspect which supported local skills, shared source packs, and credential classes are available without printing secret values:

npm run discover:sources

Coach And Evaluator Modes

coach mode helps improve a project and plan the next experiment.

evaluator mode freezes the rubric, evidence cutoff, and allowed sources before scoring. It does not silently rewrite a submission or use private coaching history. Scores are reported with evidence coverage and confidence so Superteam or another operator can audit the result.

Hackathons And Workshops

Use the same $proofpilot entrypoint for organizers, judges and participants. The agent selects evaluator for formal judging or coach for a participant's self-review. An organizer can ask for a rubric and submission form before any projects exist.

Two versioned ProofPilot defaults are included: solana_hackathon (product-focused, 25/25/15/10/10/10/5 points) and learning_workshop (working assignment, technology use, understanding, usability and task value: 40/20/20/10/10). A specific event's published or organizer-supplied rules take precedence. These presets are not official Colosseum criteria; unspecified official weights are never invented.

$proofpilot Оцени проекты Solana-хакатона по нашему базовому профилю.
$proofpilot Подготовь форму заявки и правила оценки для учебного воркшопа по API.
$proofpilot Проверь мою заявку перед хакатоном в режиме coach по приложенным правилам.

See the event workflow and five-part intake. Event scorecards preserve evidence status and admission separately. Missing criteria keep their weight: partial results show earned points and unresolved coverage, with no final /100 total or normalized ranking. Video transcripts, recorded demos and independently reproduced behavior are distinct evidence.

The agent handles the internal files. For a direct local check:

node scripts/cli.js event list
node scripts/cli.js event init learning_workshop /absolute/new-workshop-card.json
node scripts/cli.js event check /absolute/new-workshop-card.json

Fill the generated card from actual evidence before checking it. The portable installed helper is scripts/event-score.js and needs no additional API key. Mechanical checks verify consistency, not source authenticity or judging quality. Formal evaluation still requires the shared quality review. The event card is separate from the general venture response schema.

Tools And Credentials

ProofPilot requires Colosseum read access for completed setup and uses public/local sources before other optional connections. A tool can expose several capabilities with different controls: public research may need no secret, private repository access may need a scoped token, and deployment may remain deferred.

The generated tool catalog distinguishes:

• public references available now
• connector specifications that do not yet ship a live adapter
• a bounded Colosseum read helper sharing the secure setup credential resolver
• catalogued candidates requiring verification
• deferred actions requiring stronger permission, cost, or security controls

ProofPilot never asks for a private key, seed phrase, mnemonic, or raw API secret in chat.

Repository Layout

skills/proofpilot/
  SKILL.md                 Canonical routing and safety instructions
  agents/openai.yaml       Agent UI metadata
  references/              Workflows, registries, rubrics, and schemas
  scripts/                 Setup, Colosseum reads, discovery, response validation
docs/                      Human-facing architecture and generated tool catalog
examples/                  Requests, structured output, and evaluation cases
scripts/                   CLI, documentation generator, and validator

The skills/proofpilot directory is self-contained and can be installed without copying sibling skills or repository-level data files.

Development

npm run generate:docs
npm test
npm pack --dry-run

npm test validates JSON Schemas, cross-registry references, unique IDs, rubric weights, workflow links, example responses, score arithmetic, evidence/gate semantics, routing contracts, installation modes, and generated documentation. These are deterministic checks, not a measurement of model quality.

Use node scripts/cli.js validate-response response.json for full Ajv and semantic validation. An installed skill can use its own scripts/validate-response.js with no npm dependencies. For actual model comparisons, see evaluation.md; recorded outputs and explicit judgments are required.

Use proofpilot quality --help (or the installed scripts/quality.js --help) for the local quality workflow. It snapshots evidence, retains drafts and reviewer findings, checks source-quote bindings, arithmetic, word limits and recorded action prerequisites, and stops on unresolved review or the repair limit. Follow quality.md for the compact packet and review process. It does not call model APIs, certify source truth, or secretly switch models. Separate review uses the host's available, authorized resources; self-review is recorded as such.

New quality packets declare mode: coach or mode: evaluator. Application judgments and evaluator mode automatically require separate-context review; material source conflicts additionally set requires_independent_review: true. If only self-review is available for those runs, the helper returns needs_review; it does not turn unanimous self-checks into acceptance. Ordinary bounded validation can still use explicitly labeled self-review.

The registries include a scoped freshness audit dated 2026-09-05: 50 tools, 18 sources and 6 accelerator profiles. Review dates include partial checks. Follow-up documented curl calls authenticated and searched Colosseum successfully with the existing PAT; earlier Python 403 responses did not prove the skill path unavailable. Replit has a documented optional MCP route; account access remains untested. The package now includes a bounded Colosseum read helper; its operation-specific checks and limitations are recorded separately. Read freshness.md and the per-entry evidence before reusing volatile claims.

Version 0.3 changes the response contract: scoped target/action decisions replace legacy verdicts; evaluator snapshots and evidence-backed mandatory checks are required. Regenerate old responses from their evidence rather than inventing missing fields.

Status

Alpha. Version 0.3 is a portable skill with executable response validation, mandatory Colosseum setup and a bounded Colosseum read helper. Other research uses the host agent's tools. There is no hosted UI, encrypted secret vault or hosted persistence layer.

License

MIT