A startup claims it broke through a bottleneck that’s holding back LLMs Miami-based AI startup Subquadratic has emerged from stealth mode, announcing a breakthrough in large language models (LLMs) with their new model, SubQ. SubQ reportedly offers significant improvements in speed, cost, and energy efficiency by using sparse attention instead of the conventional dense attention found in transformers. Independent tests conducted by Appen appear to validate many of Subquadratic’s ambitious claims, suggesting SubQ could rival top models in performance while being vastly more efficient for specific tasks.

• Subquadratic claims its new LLM, SubQ, is faster, cheaper, and more energy-efficient than existing models.
• SubQ can process up to 12 times more text at once, enabling complex data-heavy tasks.
• The model utilizes sparse attention, moving away from the dense attention mechanism in traditional transformers.
• Independent tests by Appen show SubQ to be significantly faster than previous sparse-attention techniques and competitive with top models in coding tasks.
• Subquadratic claims SubQ can handle tasks at a fraction of the cost of current leading LLMs.
• The model boasts a context window up to 12 million tokens, substantially larger than most current top models.
• Despite positive test results, some skepticism remains due to the model’s partial reliance on existing weights from another model. Continue reading https://www.technologyreview.com/2026/06/19/1139313/a-startup-claims-it-broke-through-a-bottleneck-thats-holding-back-llms