What NVIDIA’s bet on OpenClaw means for the future of AI and your token budget
Azeem Azhar's Exponential View Podcast Recap
Published:
What Happened
Nvidia's latest GTC Jamboree showcased AI inference as a pivotal area for future development. The company's valuation stands at $4 trillion, affirming its dominance in the AI accelerator chip market. Nvidia has $1 trillion in committed AI chip orders, a significant leap from $500 billion the previous year.
OpenClaw, an open-source platform enabling AI agents reminiscent of 1980s science fiction, is gaining rapid popularity. Within 45 days, it surged from 5,000 GitHub stars to becoming a major focal point. Exponential View has increased its compute capacity by 50% to accommodate these agents, highlighting the growing demand for AI inference.
Inference is now the primary driver of AI compute demand, shifting away from training. Nvidia's acquisition of Groq for $20 billion aims to bolster their inference capabilities. The planned release of new chips with Groq technology promises a 35-fold improvement in throughput per megawatt.
Exponential View's AI token usage has skyrocketed, jumping from 100 million to 870 million tokens daily in a matter of weeks. This increase is attributed to the deployment of various AI models for simulations, coding, documentation, and security audits. A model registry ensures these AI systems remain up-to-date.
AI agents like R. Mini Arnold and R. Veblen currently trigger inference workloads at Exponential View. Although these processes require manual approval, there is a noticeable trend towards allowing agents to autonomously initiate large token workloads. Governance over these agents and their compute usage is deemed necessary to maintain efficiency.
Token budgets have become a critical consideration, especially in smaller organizations. Jensen suggests allocating half of an engineer's salary to their token budget, underscoring its importance. In many companies, IT departments manage token budgets, but this may not be the ideal approach.
The inference economy is experiencing a significant shift, having grown a million-fold over the past two years. Nvidia and its partners anticipate continued growth in this sector. The transition from a training economy to an inference economy marks a new era in AI development.
Key Insights
- Nvidia's commitment to AI inference is reflected in its $1 trillion in orders for AI chips, which is double what it was last year.
- OpenClaw's rapid rise in popularity among developers highlights the increasing interest in open-source AI platforms. Its GitHub stars grew from 5,000 to a major focus within just 45 days.
- The shift from training to inference in AI computing demands is evident, with a reported 10,000-fold increase in compute demand per user.
- AI token management is becoming crucial for companies. Jensen advocates for allocating half of an engineer's salary to their token budget, emphasizing the growing financial importance of AI operations.