DeepSeek-V2 beats Mixtral 8x22B with >160 experts at HALF the cost

**DeepSeek V2** introduces a new state-of-the-art MoE model with **236B parameters** and a novel Multi-Head Latent Attention mechanism, achieving faster inference and surpassing GPT-4 on AlignBench. **Llama 3 120B** shows strong creative writing skills, while Microsoft is reportedly developing a **500B parameter** LLM called **MAI-1**. Research from Scale AI highlights overfitting issues in models like **Mistral** and **Phi**, whereas **GPT-4**, **Claude**, **Gemini**, and **Llama** maintain benchmark robustness. In robotics, **Tesla Optimus** advances with superior data collection and teleoperation, **LeRobot** marks a move toward open-source robotics AI, and **Nvidia's DrEureka** automates robot skill training. Multimodal LLM hallucinations are surveyed with new mitigation strategies, and **Google's Med-Gemini** achieves SOTA on medical benchmarks with fine-tuned multimodal models.