SemaPact

Deterministic lifecycle governance and production assurance for ODCS data contracts.

SemaPact is an open-source, change-driven governance layer for evolving data products safely. It uses the Open Data Contract Standard (ODCS) as its canonical governed representation and keeps lifecycle policy deterministic, reviewable, and platform-neutral.

SemaPact is not another metadata catalog or CRUD editor. It governs how data products are allowed to change, turns approved contract releases into explicit deployment intent, and verifies how governed desired state compares with observed platform state.

Installation

Install the platform-neutral core:

pip install semapact

For Databricks / Unity Catalog integration:

pip install "semapact[databricks]"

Other integrations are exposed as optional extras so the base package does not require unrelated platform or UI dependencies.

pip install "semapact[sql]"
pip install "semapact[delta]"
pip install "semapact[quality]"
pip install "semapact[llm]"
pip install "semapact[tui]"

Verify the CLI:

semapact --help

The Problem SemaPact Solves

Validating one contract file is the easy part. Production governance becomes harder when the data product evolves:

a column disappears;
a required field becomes optional;
a physical type changes;
decimal precision or scale is reduced;
an active field needs to be deprecated;
a contract is retired and must become immutable;
an approved contract release needs to become runtime state safely;
production state no longer matches the governed desired state.

SemaPact treats these as governance, convergence, and assurance problems, not YAML editing operations.

Current Governed Contract
          +
Candidate Contract
          ↓
Canonical Identity
          ↓
Change Analysis
          ↓
Lifecycle + Version Policy
          ↓
GovernanceDecision
          ↓
Governance Gate

The exact governed decision then feeds either candidate deployment or an explicit formal release:

GovernanceDecision
→ ChangeSet
   ├─ candidate deployment
   │    → DeploymentSourceSnapshot(source_kind=candidate)
   │    → DeploymentPlan
   │    → DeploymentBundle
   │
   └─ formal release
        → ReleasePlan
        → VersionResolution
        → ReleaseSnapshot
        → ReleaseBundle
        → ContractRelease
             ↓
          + target
             ↓
        DeploymentSourceSnapshot(source_kind=contract_release)
        → DeploymentPlan
        → DeploymentBundle

Release approval protects formal publication. Runtime deployment permission belongs to the surrounding protected execution context; ContractRelease is provenance, not DEPLOY authorization.

For production assurance:

Governed Desired State
          +
ObservedPlatformState
          ↓
Deterministic Reconciliation
          ↓
IN_SYNC / DRIFT / INDETERMINATE

The governed desired state is an authoritative ODCS revision selected by an upstream governance / release / authorization process. approved is not an ODCS lifecycle status and reconciliation does not invent one.

Core Principles

Change-driven, not CRUD

Governance begins with the difference between a governed base revision and a proposed candidate revision. SemaPact is designed around change analysis rather than directly editing canonical state in place.

Deterministic by default

The same inputs and governance context should produce the same result. Lifecycle policy belongs in deterministic code, not in UI state or LLM reasoning.

Canonical identity is explicit

For the current governance model:

schema identity   = lowercase(schema.name)
property identity = lowercase(schema.name) + lowercase(property.name)

physicalName is not identity.

Lifecycle and authorization are separate concepts

SemaPact models lifecycle states such as:

DRAFT → ACTIVE → DEPRECATED → RETIRED

Lifecycle status does not itself mean that a revision has been authorized for release.

Side effects have explicit trust boundaries

SemaPact distinguishes pure planning/materialization from external side effects. Formal release PUBLISH approval is recorded explicitly when governance requires review. Runtime DEPLOY permission belongs to the surrounding protected execution boundary (for example a GitHub or Azure DevOps Environment), not to the ContractRelease artifact. Release state therefore cannot authorize runtime mutation by itself.

Runtime-aware without becoming platform-owned

Platforms such as Databricks Unity Catalog describe what exists now. SemaPact keeps its governance kernel and deployment intent provider-neutral, while provider adapters translate supported runtime operations explicitly.

AI can consume governance; AI does not become governance authority

Agents may consume governed contracts, decisions, reason codes, release/deployment artifacts, and semantic context. Deterministic governance policy remains authoritative.

Current Capabilities

Deterministic lifecycle governance

SemaPact currently supports deterministic change analysis and lifecycle-aware policy including:

canonical schema and property identity;
active / draft / deprecated / retired lifecycle semantics;
retired-state immutability;
active-field removal handling and governed deprecation;
physical type change detection;
logical type incompatibility checks;
decimal precision / scale reduction checks;
required / nullability tightening;
relationship change handling;
version-policy classification;
deterministic GovernanceDecision artifacts;
centralized governance and release-approval boundaries, with runtime deployment permission delegated to the protected execution environment.

Canonical ContractOps release planning

The release planning path evaluates governance once and produces exact deterministic artifacts:

GovernanceDecision
→ ChangeSet
→ ReleasePlan
→ VersionResolution

semapact release plan requires explicit base and candidate revision references so downstream authorization and application can refer to the exact workflow revisions rather than mutable file paths.

SemaPact supports two version-authority modes:

semapact — independently version contracts from their governed change requirements;
git — validate an explicitly supplied product/repository release version against the governance-required minimum bump.

See docs/contractops_phases.md and docs/version_authority.md.

Contract release and runtime deployment

SemaPact treats formal contract release and runtime deployment as separate lifecycles.

A formal release is target-neutral:

release assess
→ GovernanceDecision
→ ChangeSet
→ ReleasePlan
→ VersionResolution
→ ReleaseSnapshot
→ ReleaseBundle

REVIEW → exact release approval
        ↓
release finalize
→ ContractRelease
→ materialize the selected version back to ODCS

The selected version is calculated once. The resulting ContractRelease can then be deployed to any number of runtime targets without another version bump:

orders@1.4.0
├── dev
├── test
└── prod

Deployment is target-specific:

deployment assess --release ./artifacts/contract-release.json
→ DeploymentSourceSnapshot
→ DeploymentPlan
→ fresh runtime preview
→ DeploymentBundle

deployment deploy
→ fresh preview → execute → fresh verify
→ IN_SYNC / DRIFT / INDETERMINATE

Candidate/non-release deployment remains available directly from base + candidate and does not create a new contract version or release history.

Deployment telemetry is disabled by default. High-frequency execution history can be enabled once in typed .semapact.yaml configuration with a SQLite or Delta backend; it is not written into Git governance history. The --operational-history CLI option is only an override.

For Databricks, once deployment of a finalized formal release verifies IN_SYNC, SemaPact projects the finalized release provenance to governed Unity Catalog tables using reserved tags:

semapact_contract_id
semapact_contract_version
semapact_release_id
semapact_source_revision

Candidate deployments do not publish formal version/release tags. Business classifications or ABAC tags are not automatically mapped.

See docs/deployment_plans.md.

Databricks discovery and observation

With the databricks extra, SemaPact provides a thin read-side integration using the official Databricks SDK:

Databricks
   ├── Discovery
   │     → asset identities
   │
   └── Observation
         → ObservedPlatformState

Discovery identifies assets in a requested scope. Observation captures platform-neutral physical schema state.

Stable observation fingerprint

Observed physical schema state can be represented by a deterministic fingerprint over the current obs-v1 semantic payload:

platform
asset identity
asset type
property identity
physical type
nullability

Volatile envelope fields such as capture time and source location are excluded from the content fingerprint.

Runtime reconciliation and convergence verification

SemaPact deterministically compares governed desired state with ObservedPlatformState and reports factual differences for semantics represented on both sides today:

missing / unexpected assets;
missing / unexpected properties;
physical type mismatch;
required / nullability mismatch.

Reconciliation classifies the result as IN_SYNC, DRIFT, or INDETERMINATE. Deployment verification reuses the same reconciliation authority; it does not invent a separate convergence status machine.

See docs/runtime_reconciliation.md.

SemaPact + Databricks Unity Catalog

Unity Catalog and SemaPact solve different parts of the problem.

Unity Catalog tells you what exists. SemaPact governs desired-state evolution, executes explicitly supported governed mutations, and verifies observed state against that governed intent.

Git / ODCS
Governed Desired State
        │
        ▼
   ┌──────────┐
   │ SemaPact │
   └──────────┘
      │     ▲
      │     │ observation / reconciliation
      ▼     │
Governed runtime mutation
      │     │
      ▼     │
Databricks / Unity Catalog

Unity Catalog remains responsible for runtime assets, access control, lineage, metadata, and platform enforcement. SemaPact does not aim to replace it.

Quick Start

Inspect the CLI

pip install semapact
semapact --help

Import from SQL

Install SQL support:

pip install "semapact[sql]"

Then import into ODCS:

semapact import \
  --format sql-folder \
  --source ./ddl \
  --output ./contracts/orders.yaml

Analyze / merge governed contract evolution

semapact merge \
  --base ./generated.yaml \
  --business ./contracts/orders.yaml \
  --output ./contracts/orders.merged.yaml \
  --effective-date 2026-09-03

Build canonical release planning artifacts

semapact release plan \
  --base ./contracts/orders.yaml \
  --candidate ./contracts/orders.candidate.yaml \
  --base-revision-ref git:abc123 \
  --candidate-revision-ref git:def456 \
  --effective-date 2026-09-11

The output contains the canonical GovernanceDecision, ChangeSet, ReleasePlan, and VersionResolution artifacts.

Databricks integration

pip install "semapact[databricks]"

The Databricks SDK owns authentication-provider selection. SemaPact forwards supported connection hints rather than implementing a separate credential system.

Assess a candidate deployment without creating a new contract version:

semapact deployment assess \
  --base ./contracts/orders.yaml \
  --candidate ./contracts/orders.candidate.yaml \
  --base-revision-ref git:abc123 \
  --candidate-revision-ref git:def456 \
  --server development \
  --bundle-out ./artifacts/orders-dev.bundle.json

Build a target-neutral formal release separately:

semapact release assess \
  --base ./contracts/orders.yaml \
  --candidate ./contracts/orders.candidate.yaml \
  --base-revision-ref git:abc123 \
  --candidate-revision-ref git:def456 \
  --bundle-out ./artifacts/orders.release.bundle.json

For REVIEW releases, record the exact external approval and then finalize the release:

semapact release approve \
  --bundle ./artifacts/orders.release.bundle.json \
  --actor-reference github-environment:contract-release \
  --recorded-at 2026-09-20T10:00:00+10:00 \
  --approval-out ./artifacts/orders.approval.json

semapact release finalize \
  --bundle ./artifacts/orders.release.bundle.json \
  --approval ./artifacts/orders.approval.json \
  --output-contract ./contracts/orders.yaml \
  --release-out ./artifacts/orders.contract-release.json

Finalization writes the selected semantic version back to the ODCS contract and records the immutable ContractRelease in the Git governance ledger. The release records the source revision from which it was derived; the finalized release identity is the immutable released-contract snapshot, not a claim that the source revision already contained the materialized version bump.

Deployment then consumes the finalized release artifact directly:

semapact deployment assess \
  --release ./artifacts/orders.contract-release.json \
  --server production \
  --bundle-out ./artifacts/orders-prod.deployment.bundle.json

semapact deployment deploy \
  --bundle ./artifacts/orders-prod.deployment.bundle.json \
  --warehouse-id <databricks-sql-warehouse-id>

Candidate deployments do not require release approval.

Operational deployment history is configured project-wide rather than repeated on every deploy:

history:
  operational:
    backend: sqlite
    path: .semapact/operational.db

For a shared Delta sink:

history:
  operational:
    backend: delta
    table_uri: s3://governance/semapact/operational-history

The config is fail-closed against the typed SemaPactConfigSchema. --operational-history remains available only as a per-invocation override. If neither config nor override is present, operational persistence stays disabled. See docs/configuration.md for precedence and schema details.

Optional Dependencies

Extra	Purpose
`sql`	SQL parsing and SQL-folder workflows
`delta`	Delta table support
`databricks`	Databricks / Unity Catalog observation and governed deployment
`quality`	Great Expectations integration
`graph`	Graph export support
`llm`	Optional LLM-assisted semantic enrichment
`azure`	ADLS2 access
`s3`	Amazon S3 access
`tui`	Textual terminal interface
`all`	All currently supported optional integrations

Optional extras are intentionally separate from the base distribution. If an integration is not listed here, it is not part of the supported public extra surface.

Package Architecture

semapact/
  core/             # loading, configuration and validation
  lifecycle/        # canonical identity, lifecycle and change policy
  governance/       # GovernanceDecision and centralized gate
  contractops/      # deterministic release planning, approval and release artifacts
  deployment/       # provider-neutral deployment source, plan and preview contracts
  runtime/          # provider-neutral governed runtime asset projection
  application/      # interface-independent use-case models + orchestration
    models/         # application result DTOs; no domain authority
    services/       # thin orchestration over canonical domain rules/ports
  observation/      # platform-neutral observed state + fingerprint
  reconciliation/   # governed desired vs observed comparison
  platforms/        # provider adapters such as Databricks
  importers/        # external metadata → ODCS projection
  exporters/        # SQL / graph and other outputs
  quality/          # quality intent adapters
  interfaces/       # CLI and user-facing boundaries
  devops/           # Git / CI integration helpers

A central architectural rule is:

Interfaces parse and render. Application services orchestrate. Domain packages own business meaning. Platform adapters own provider-specific effects. There is one canonical workflow per lifecycle.

See ARCHITECTURE.md for package/model placement rules.

What SemaPact Does Not Try to Replace

SemaPact is not intended to replace:

Databricks Unity Catalog or another metadata catalog;
dbt, Spark, Lakeflow, or another transformation engine;
Great Expectations or another data-quality execution runtime;
Terraform / Databricks Asset Bundles as general infrastructure tooling;
Git review and human authorization processes.

SemaPact provides a deterministic governance, convergence, and assurance layer around those systems.

Development

Clone the repository and install the development environment with all supported extras:

uv sync --all-extras --group dev --frozen

Run tests:

uv run pytest

Build the Python distribution:

uv build

Product Model

SemaPact is organized as a governed desired-state control plane:

GOVERN
Can this contract change be allowed?

CONVERGE
Can the exact governed release safely become runtime state?

ASSURE
Does observed runtime state match governed desired state?

PROVE
What happened, why, and through which decision / release / deployment / observation?

Governance, release planning, guarded deployment, and runtime reconciliation are implemented as separate boundaries so persistence, history, additional provider capabilities, and audit surfaces can evolve without collapsing these responsibilities into one layer.

Open Data Contract Standard

SemaPact uses the Open Data Contract Standard (ODCS) as its canonical contract representation rather than introducing a proprietary data-contract schema.

Contributing

SemaPact is developed in the open. Contributions are welcome, including bug reports, integrations, architecture discussions, governance scenarios, Databricks / Unity Catalog cases, and schema-evolution edge cases.

See CONTRIBUTING.md and SECURITY.md for project guidance.

License

Licensed under the Apache License, Version 2.0 (Apache-2.0). See LICENSE.

Name		Name	Last commit message	Last commit date
.agents		.agents
.codex/skills		.codex/skills
.github		.github
.vscode		.vscode
docs		docs
examples		examples
schemas		schemas
semapact		semapact
tests		tests
.gitignore		.gitignore
.python-version		.python-version
.semapact.yaml		.semapact.yaml
AGENTS.md		AGENTS.md
ARCHITECTURE.md		ARCHITECTURE.md
CHANGELOG.md		CHANGELOG.md
CLAUDE.md		CLAUDE.md
CONTRIBUTING.md		CONTRIBUTING.md
GOVERNANCE_CONTEXT.md		GOVERNANCE_CONTEXT.md
GOVERNANCE_REASON_CODES.md		GOVERNANCE_REASON_CODES.md
LICENSE		LICENSE
README.md		README.md
RUNTIME_REASON_CODES.md		RUNTIME_REASON_CODES.md
SECURITY.md		SECURITY.md
TUI_SPEC.md		TUI_SPEC.md
pyproject.toml		pyproject.toml
uv.lock		uv.lock

Latest commit

History

Folders and files

Repository files navigation

SemaPact

Installation

The Problem SemaPact Solves

Core Principles

Change-driven, not CRUD

Deterministic by default

Canonical identity is explicit

Lifecycle and authorization are separate concepts

Side effects have explicit trust boundaries

Runtime-aware without becoming platform-owned

AI can consume governance; AI does not become governance authority

Current Capabilities

Deterministic lifecycle governance

Canonical ContractOps release planning

Contract release and runtime deployment

Databricks discovery and observation

Stable observation fingerprint

Runtime reconciliation and convergence verification

SemaPact + Databricks Unity Catalog

Quick Start

Inspect the CLI

Import from SQL

Analyze / merge governed contract evolution

Build canonical release planning artifacts

Databricks integration

Optional Dependencies

Package Architecture

What SemaPact Does Not Try to Replace

Development

Product Model

Open Data Contract Standard

Contributing

License

About

Topics

Resources

Contributing

Security policy

Stars

Watchers

Forks

Releases

Packages

Used by

Contributors

Languages