← All news

not much happened today

The AI news recap highlights several key developments: **nanoMoE**, a PyTorch implementation of a mid-sized Mixture-of-Experts (MoE) model inspired by Andrej Karpathy's nanoGPT, enables pretraining on commodity hardware within a week. An agentic leaderboard ranks LLMs powering **smolagents CodeAgent**, with **GPT-4.5** leading, followed by **Claude-3.7-Sonnet**. Discussions around **DeepSeek-R1** emphasize AI model commoditization, with DeepSeek dubbed the "OpenAI of China." **Q-Filters** offer a training-free method for KV cache compression in autoregressive models, achieving **32x compression** with minimal perplexity loss. The **PokéChamp** minimax language agent, powered by **GPT-4o** and **Llama-3-8b**, demonstrates strong performance in Pokémon battles. Other notable models include **TinyR1-32B-Preview** with Branch-Merge Distillation, **R1-Searcher** incentivizing search capability via reinforcement learning, and the **Forgetting Transformer** using a Forget Gate in softmax attention. These advancements reflect ongoing innovation in model architectures, compression, reinforcement learning, and agentic AI.
Read original at AINews / smol.ai →