Behold the Megatron: Microsoft and Nvidia build massive language processor - The Register

Megatron gets a tune-up courtesy of Microsoft and NVIDIA Windows Central Microsoft and Nvidia create 105-layer, 530 billion parameter language model that needs 280 A100 GPUs, but it's... ZDNet Microsoft and Nvidia created the world's largest, most powerful language model to date, but it's still biased TechSpot NVIDIA, Microsoft Introduce New Language Model MT-NLG With 530 Billion Parameters, Leaves GPT-3 Behind Analytics India Magazine
https://bit.ly/3iXlnBu