Hybrid SSM/Transformers > Pure SSMs/Pure Transformers

**NVIDIA**'s Bryan Catanzaro highlights a new paper on **Mamba models**, showing that mixing Mamba and Transformer blocks outperforms either alone, with optimal attention below **20%**. **Mixture-of-Agents (MoA)** architecture improves LLM generation quality, scoring **65.1% on AlpacaEval 2.0** versus **GPT-4 Omni's 57.5%**. The **LiveBench AI benchmark** evaluates reasoning, coding, writing, and data analysis. A hybrid **Mamba-2-Hybrid** model with **7% attention** surpasses a Transformer on MMLU accuracy, jumping from **50% to 53.6%**. **GPT-4** performs better at temperature=1. **Qwen 72B** leads open-source models on LiveBench AI. **LaminiAI Memory Tuning** achieves **95% accuracy** on a SQL agent task, improving over instruction fine-tuning. **Sakana AI Lab** uses evolutionary strategies for preference optimization. **Luma Labs Dream Machine** demonstrates advanced text-to-video generation. The **MMWorld benchmark** evaluates multimodal video understanding, and **Table-LLaVa 7B** competes with GPT-4V on multimodal table tasks.