Mergestral, Meta MTIAv2, Cohere Rerank 3, Google Infini-Attention

**Meta** announced their new **MTIAv2 chips** designed for training and inference acceleration with improved architecture and integration with PyTorch 2.0. **Mistral** released the **8x22B Mixtral** model, which was merged back into a dense model to effectively create a 22B Mistral model. **Cohere** launched **Rerank 3**, a foundation model enhancing enterprise search and retrieval-augmented generation (RAG) systems supporting 100+ languages. **Google** published a paper on **Infini-attention**, an ultra-scalable linear attention mechanism demonstrated on 1B and 8B models with 1 million sequence length. Additionally, **Meta's Llama 3** is expected to start rolling out soon. Other notable updates include **Command R+**, an open model surpassing GPT-4 in chatbot performance with 128k context length, and advancements in Stable Diffusion models and RAG pipelines.