representation-geometry

Trajectory Geometry of Transformer Representations Across Layers

[EMNLP 2026 Findings] Decomposing Transformer updates into parallel & perpendicular subspaces for geometric probing, compression, and training dynamics.

Prosthetic cognition architecture for AI agents. Deterministic scaffolding over probabilistic reasoning.

Corpus operators recover different structures, and task-aligned conditionals predict held-out model behaviour — with musical keys as the instrument. Code, results and audit trail for the paper.

Mechanistic interpretability of multilingual reasoning in transformers. 170+ causal intervention experiments across 4 model families.

Code, results and paper for "The geometry of single-cell foundation models: what they inherit, what they add, and what shapes it"

Effective rank, RankMe, E1, CKA and anisotropy on transformer hidden states are determined by one direction. The exact identity, and the attention sink behind it.

Mechanistic interpretability of transformer hallucinations via attention flow, residual stream geometry, and head-level attribution analysis.

Visualizing Modern LLM Mechanics, Loss Landscapes & HPC Topologies

A 2-D map of GPT-2's token embeddings you can poke at. Pick two axes, project the vocabulary, and see where analogies, cyclic features, and hubness show up (and where the 2-D view is lying to you). Companion to a writeup on which "cyclic" concepts actually form circles.

Code for 'Exploring the Impact of a Transformer's Latent Space Geometry on Downstream Task Performance' (arXiv:2406.12159)

Reproducibility package for "Context Is King: How In-Context Specification Shapes the Geometry of Concepts" — code, cached data, and interactive 3D explorers.

Personal learning notebook on latent space engineering, representation geometry, and exploratory research notes.

Code for 'Reliable Measures of Spread in High Dimensional Latent Spaces' (ICML 2023)

Com la tokenització fractura la morfologia catalana i si una segmentació conscient dels morfemes recupera la geometria. Provat en 3 llengües indoeuropees (català, castellà, anglès): el català es fragmenta ~1,7× més que l'anglès; forçar el tall morfèmic recupera la composicionalitat (robust a portadora i replicat en castellà).

An empirical and geometric analysis of Neural Collapse under different optimizers on CIFAR-10