📰 Must-read papers and blogs on LLM based Long Context Modeling 🔥
Methods and code for extending the context length of diffusion language models
An imaginary extension of rotary position embeddings for long-context language models
Context-aware Biases for Length Extrapolation
A training-free method for extending the maximum context length of language models
Controlled experiments on recurrent-depth (looped) transformers — length extrapolation, adaptive test-time compute, and honest sub-1B negatives.
A constant-memory recurrent stream coupled to a growing .causal knowledge index, with a measured capacity threshold in the gated readout. Builds on GSSM.
The O(1)-state organism: constant-memory unbounded streaming, surprise-gated plasticity, holographic memory through silence, dosed sleep, a runtime knowledge index — and live cross-architecture migration. Seven disclosed foundations, 38 pre-registered predictions, falsifications kept.
To associate your repository with the length-extrapolation topic, visit your repo's landing page and select "manage topics."