MatSage

MatSage is a terminal agent for exploring inorganic crystalline materials. Ask a question in plain English — "which stable oxides have a band gap between 2 and 3 eV?" — and it runs a multi-turn tool-calling loop against the Materials Project API, matminer experimental datasets, published literature and any documents you have ingested locally, then writes an answer that cites what it actually retrieved. It is for materials engineers and researchers who want database-grounded answers without hand-writing mp-api queries, and it runs on four LLM providers so the same agent works on a cheap open model or a frontier one.

Features

• Multi-turn tool calling across four providers — Anthropic, OpenAI, OpenRouter and DeepSeek share one loop implementation in agent/adapter_base.py; adapters only handle API serialization.
• 18 domain tools in a single registry (agent/tools.py) whose schemas, handlers and system-prompt inventory are cross-checked at import, so they cannot drift apart.
• Three processing modes: fast (execute only), standard (plan → execute), thorough (plan → execute → assess), implemented by agent/planner.py and agent/assessor.py.
• Rule-based answer verification (agent/verifier.py) — checks that every mp- ID in the answer came back from a tool, and flags mechanical-property and DFT band-gap claims that need a caveat.
• Persistent, replayable sessions (sessions.py) — the TUI, the session store, the notebook exporter and the eval harness consume the same typed StreamEvent stream, so a resumed session re-renders exactly as it looked live; notebook/export.py turns one into a runnable marimo notebook.
• Local document store (datastore/) — ingest PDF, CSV, Excel or Word files into SQLite and search them with FTS5 alongside the remote sources.
• On-demand domain knowledge (knowledge/skills/) — material-selection (Ashby's methodology: selection process, material indices, property charts) and property-handbook skills, loaded a chapter at a time rather than pasted into every prompt. Every run lands in a SQLite trace database (eval/trace_logger.py) with tokens, cost, tool calls and latency.

Architecture

 Textual TUI (tui/)   ·   matsage ask / evaluate (cli.py, eval/)
                \         /
          AgentState.run_query (agent_state.py)  — one path, one StreamEvent stream
                        |
      router -> planner -> provider adapter loop (agent/adapter_base.py)
                        |
                TOOL_REGISTRY (agent/tools.py)
                        |
   Materials Project · matminer experimental · Semantic Scholar · DuckDuckGo
   (materials/)                                    · local datastore (FTS5)
                        |
        assessor (thorough mode) -> verifier -> answer
                        |
   session store (sessions.py) · trace logger (traces/traces.db) · marimo notebook

Package	Purpose
agent/	Adapters, tool registry, planner, assessor, verifier, router, context handoff, stream event types
materials/	Materials Project client with rate limiting and disk cache, matminer experimental lookups, metallurgical calculations, Semantic Scholar and web search
knowledge/	Progressive skill loader and the Markdown skill/chapter files
datastore/	File ingestion, SQLite store, FTS5 search over user documents
sessions.py	Records the event stream per session and replays it back into the same renderer
notebook/	Exports one session as a marimo notebook
eval/	Benchmark queries, run harness, LLM-as-judge scorer, trace database
tui/	Textual application, slash commands, renderers, input and dialog widgets
viz/	Plotly comparison, radar and Ashby-style charts, used by both the dashboard and the notebook export
app.py	marimo analytics dashboard over the trace database, launched by matsage dashboard
agent_state.py / config.py	The single orchestration path shared by TUI, CLI and eval harness; provider config, models.json, pricing and spend caps

Installation

Requires Python 3.12+ and uv.

git clone https://github.com/sleipnir029/MatSage.git && cd MatSage
cp .env.example .env              # then fill in your keys
uv sync                           # add --extra datastore for PDF/Excel/Word ingestion
uv run matsage --help

A Dockerfile is included and builds an image whose entrypoint is matsage.

Environment variables

All are read from .env or the process environment. .env.example ships the keys you are most likely to need; the full set is below.

Variable	Required	Purpose
MP_API_KEY	Yes	Materials Project API key (free) — without it the database tools cannot run
OPENROUTER_API_KEY	One provider key required	Enables the openrouter provider and the 14 models in models.json
ANTHROPIC_API_KEY	"	Enables the claude provider (claude-sonnet-4-20250514)
OPENAI_API_KEY	"	Enables the openai provider (gpt-4o)
DEEPSEEK_API_KEY	"	Enables the deepseek provider (direct API, peak/off-peak pricing)
S2_API_KEY	Optional	Semantic Scholar key for search_papers (higher rate limits)
MAX_SPEND_USD	Optional	Global lifetime spend cap across all runs (default 1.0)
SYNTHESIS_RESERVE_USD	Optional	Budget held back so a final synthesis call still fits (default 0.02)
MARIMO_URL	Optional	marimo server used by execute_in_notebook (default http://localhost:3000)
JUDGE_MODEL	Optional	Model used by the LLM-as-judge scorer (default qwen/qwen3-32b)
MATSAGE_LOG_LEVEL	Optional	Log level (default INFO)
MATSAGE_TRACE_RETENTION_DAYS	Optional	Age at which JSONL trace tapes are pruned (default 30)

models.json holds the OpenRouter model table (key, model id, per-1k pricing, context window), the default_model, and the router tiers used when auto-routing is on.

Usage

Launch the TUI:

uv run matsage                  # new session
uv run matsage --continue       # reopen the most recent session with runs
uv run matsage --resume 7a3f    # resume by session id or unique prefix

Slash commands

Group	Commands
Session	/new [name], /sessions, /resume <id>, /rename <name>, /branch, /merge, /tree, /quit (aliases /exit, /q)
Model	/model <key>, /provider <claude|openai|openrouter|deepseek>, /models, /router, /reasoning <off|low|medium|high>
Run controls	/mode <fast|standard|thorough>, /budget <usd>, /turns <n>, /detail <brief|standard|detailed>, /settings, /compact, /clear
Data & inspection	/ingest <path>, /datastore, /trace, /runs, /show <run_id>, /cost
Notebook	/notebook, /notebook open, /notebook exec on|off
Other	/help, /sidebar

Typing / opens a completion popup; Tab completes, and argument values (models, providers, modes) come from the definitions that own them.

Keyboard shortcuts

Key	Action
Enter	Send the query (or complete the highlighted command when the popup is open)
Ctrl+J / Shift+Enter / Alt+Enter	Newline inside the input
Up / Down	Walk the input history, shell style — only from the first/last line of the input
Esc	Cancel the running query, or close the popup/dialog
Ctrl+N / Ctrl+O / Ctrl+P / Ctrl+B	New session / session picker / command palette / toggle sidebar
Ctrl+Up / Ctrl+Down (or Ctrl+PgUp / Ctrl+PgDn)	Scroll the transcript half a page

CLI subcommands

ask             Run a single query and print the answer
runs            List recent runs
show            Show the trace and answer of one run
sessions        List sessions that have runs
models          List configured providers and available models with pricing
cost            Show total spending against the cap
notebook        Export a session as a marimo notebook
evaluate        Run the evaluation benchmark across models
benchmark-list  List all benchmark queries
dashboard       Launch the marimo analytics dashboard

uv run matsage ask "What is the band gap of GaAs?" --model deepseek-v4-flash --mode standard
uv run matsage notebook 7a3f --open
uv run matsage runs -n 20

Example session (illustrative)

Shape of a standard-mode turn — phase labels, tool sequence and the closing stats line are real parts of the stream; every number below is a placeholder, not a recorded run.

matsage ❯ Which stable oxides have a band gap between 2 and 3 eV?

  planning   screen by band gap and stability, then cross-check computed gaps against measured ones
  Working…   search_materials  →  lookup_experimental  →  get_material_details
  verifying  every mp- id traced to a tool result; DFT band-gap caveat present

  <answer: the shortlist, with the caveats the verifier checked for>
  deepseek-v4-flash · 12,400↓ 1,800↑ · $0.00xx · 3 tools · x.xs

Reliability and cost controls

• Per-query budget — QuerySettings.query_budget (default $0.05, /budget to change); the loop stops short of the cap, holding back SYNTHESIS_RESERVE_USD (at most 20% of the budget) so one final synthesis call over the data already gathered still fits.
• Global spend cap — MAX_SPEND_USD is checked against the lifetime total in the trace database before a query starts; the effective budget is the smaller of the remaining cap and the query budget.
• Turn limit with forced synthesis — max_turns (default 20, hard ceiling 50); one turn before the limit the agent is told to synthesize now, and a short answer is recovered rather than discarded.
• Retry and circuit breaker — transient tool errors (network, rate limit) are retried, deterministic ones are not; after 3 consecutive failures a tool is disabled for the rest of the run and the model is told to route around it (agent/adapter_base.py).
• Context handoff — agent/context.py tracks usage against the model's window, warns at 75%, and at 85% summarizes the conversation and starts a fresh transcript.
• Session persistence and replay — every UI-level block is stored as an event row and replayed through the same renderer on resume (sessions.py).
• Verification — agent/verifier.py checks the final answer: unsourced MP IDs are hard failures; missing mechanical-property and DFT band-gap caveats are soft warnings. Both are shown in the TUI and in matsage ask.

Evaluation

src/matsage/eval/ holds 22 benchmark queries across 7 categories (property lookup, application selection, multi-constraint selection, limitation awareness, cross-reference, processing/synthesis, structural exploration), each with expected tools and ground truth. The harness runs them through one or more models, applies automated checks (such as whether the expected tools were called), prints a summary, and writes a scoring sheet for a five-dimension rubric — factual grounding, reasoning quality, limitation awareness, tool strategy, answer completeness — each scored 0–3.

uv run matsage benchmark-list
uv run matsage evaluate --models deepseek-v4-flash,gpt-oss-120b
uv run matsage evaluate --queries PL-01,AS-01 --mode thorough

eval/judge.py scores stored answers with a separate LLM (JUDGE_MODEL) so the agent does not grade itself; its reports are written into docs/ locally and are gitignored, not part of the repo. uv run matsage dashboard opens a marimo dashboard over the trace database for runs, tools and cost.

Tools

Category	Tool	Purpose
Materials Project (DFT, 0 K)	search_materials	Search the database for materials matching property criteria
	get_material_details	Detailed properties for one material by MP ID
	compare_materials	Compare several materials side by side on selected properties
	find_similar_materials	Find materials structurally similar to a given one
	get_crystal_description	Natural-language structure descriptions from Robocrystallographer
	search_synthesis	Synthesis recipes extracted from published literature
Experimental datasets (measured)	lookup_experimental	Measured (not computed) properties from matminer: expt_gap, steel_strength, citrine_thermal_conductivity
Calculations	calculate_steel_property	Carbon equivalent (IIW), martensite start (Andrews), Hall–Petch, Ae3, solid-solution strengthening
Domain knowledge	load_skill	Load a skill's core framework document
	load_chapter	Load one chapter of a skill on demand
Literature and web	search_papers	Peer-reviewed paper search via Semantic Scholar
	search_web	Web search for cost, availability, standards and processing guidance
Local data store	ingest_file	Import a PDF, CSV, Excel or Word file into the local store
	search_local_data	Full-text search over ingested documents
	get_local_document	Fetch the full text of one stored chunk
	list_local_data	List stored documents with previews
Utilities	get_cached_result	Retrieve the full, untruncated result of an earlier tool call
Notebook (marimo)	execute_in_notebook	Run Python in a live marimo server (off unless /notebook exec on)

Development

uv sync --group dev
uv run pytest                       # 328 tests, no API keys needed
uv run ruff check src/ tests/
uv run ruff format src/ tests/

Adding a tool: add the schema to _TOOL_SCHEMA_LIST and a category to TOOL_CATEGORIES in src/matsage/agent/tools.py, then a handler decorated with @tool_handler("<name>") in the same file. Import fails loudly if the two sets disagree, and tests/test_tools.py::TestRegistry guards the invariant.

Runtime state is gitignored: traces/traces.db (runs, tool calls, sessions), datastore/store.db (ingested documents), notebooks/ (exported sessions), .cache/ (Materials Project responses). Further reading: docs/audit-2026-09-04.md, a self-audit listing the defects found and their status, and docs/engineering-journal.md.

Known limitations

• No automated selection ranking. Ashby's methodology is available to the model as a loadable knowledge skill, but there is no engine that computes material indices and orders candidates; ranking is left to the model's prose. See D9 in the audit.
• DFT data is computed at 0 K. Materials Project band gaps (GGA-PBE) are systematically underestimated; the verifier nudges the model to say so and lookup_experimental cross-references measured values where a dataset covers the material. No temperature-dependent behaviour, fatigue, creep, corrosion or cost data.
• Context-window values are approximate — from models.json where present, otherwise a substring lookup table in agent/context.py.
• Auto-routing is OpenRouter-only — /router is a no-op on claude, openai and deepseek (agent_state.py:539).
• Notebook execution is off by default. execute_in_notebook is offered to the model only after /notebook exec on, and needs a marimo server already running at MARIMO_URL.
• Scope is inorganic crystalline materials and steels. No polymers, composites or biological materials. Terminal only: no REST API or web UI, and the marimo dashboard is analytics, not chat.

License

MIT. See LICENSE.

Author: Rakibuzzaman Rahat