#237 - Nemotron 3 Super, xAI reborn, Anthropic Lawsuit, Research!!! - Last Week in AI Recap

Podcast: Last Week in AI

Published: 2026-03-16

What Happened

NVIDIA's Nematron 3 Super, with 120 billion parameters and a 1 million token context window, is optimized for Blackwell GPUs using four-bit arithmetic and latent MOE for efficient compute. However, NVIDIA has halted H200 chip production for China, reallocating resources to the American and Western markets due to supply constraints.

Anthropic is facing a lawsuit against the Department of Defense, challenging its designation as a supply chain risk. This legal battle could set significant precedents for military technology, with Anthropic's lawsuit arguing unconstitutional retaliation for protected speech. An amicus brief supporting Anthropic includes signatures from 37 individuals from companies like Google and OpenAI.

XAI is undergoing a transformation after co-founders Jiang Dai and Guodong Zhang left the company. Elon Musk commented on rebuilding XAI differently, leveraging SpaceX's infrastructure for data centers in space. New members Jason Ginsberg and Andrew Millich from Cursor have joined XAI, indicating a strategic shift.

Yann LeCun's AMI Labs raised $1.3 billion to focus on building fundamental AI models rather than products, marking the largest AI research funding in European history. Valued at a $3.5 billion pre-money valuation, AMI Labs aims to make 'world models' a buzzword in AI, focusing on the future of AI research rather than immediate applications.

Research on activation steering in language models reveals their ability to manipulate internal states, such as focusing on specific topics like the Golden Gate Bridge. The LAMA 3.370B model shows higher controllability, resisting internal state manipulation while maintaining safety mechanisms to prevent off-topic content generation.

A paper titled 'Beyond Language Modeling, an Exploration of Multimodal Pre-Training' proposes integrating text, images, and video into a single pre-training pipeline. This approach uses a diffusion strategy for image processing, showing that adding visual data doesn't hurt language performance but instead offers positive transfer and world modeling capabilities.

The podcast also covers advancements in CUDA kernel optimization, where reinforcement learning is used to generate synthetic training problems. This new approach outperforms traditional methods, achieving state-of-the-art results on kernel benchmarks by optimizing memory management and operation fusion.

Drone strikes on AWS data centers in the UAE highlight the vulnerabilities in data center security, raising concerns about national security and the implications for AI hub ambitions. The increasing role of data centers as frontline assets in warfare presents new challenges in the development of tools of war.

Key Insights

• NVIDIA's Nematron 3 Super features 120 billion parameters and a 1 million token context window, optimized for Blackwell GPUs using four-bit arithmetic and latent MOE, enhancing computational efficiency.
• Anthropic's lawsuit against the Department of Defense challenges its designation as a supply chain risk, arguing unconstitutional retaliation for protected speech, with significant military technology implications.
• Yann LeCun's AMI Labs raised $1.3 billion, the largest AI research funding in Europe, to focus on fundamental AI models and make 'world models' a focal point in future AI research.
• Reinforcement learning is used in CUDA kernel optimization, outperforming traditional methods by generating synthetic training problems and achieving state-of-the-art results on kernel benchmarks, improving memory management and operation fusion.