Since Google’s 2017 “Attention Is All You Need” paper, the Transformer architecture has been the default foundation for large language models (LLMs). It powers systems like ChatGPT and Gemini, but its quadratic compute and heavy memory footprint make large-scale inference… Read More »Mamba‑3: Open Source State Space Models Challenge Transformers on Speed, Cost and Reasoning
MIT CSAIL researchers are proposing a different answer to the long-context problem in large language models: don’t keep stretching the context window—change how the system uses it. Their new Recursive Language Models (RLMs) framework treats long prompts as an external… Read More »MIT’s Recursive Language Models: A Systems Approach to 10M-Token Contexts
NeurIPS has long been the place where new architectures, training tricks and evaluation benchmarks quietly change how real systems are built. The 2025 edition continued that pattern — but with a sharper message for anyone working on LLMs, agentic systems… Read More »NeurIPS 2025: Five Papers That Show Why AI Progress Is Now Systems-Limited