#323 David Ha: Why Model Merging Could Be the Next AI Breakthrough - Eye On A.I. Recap

Podcast: Eye On A.I.

Published: 2026-02-24

Duration: 57 min

Summary

In this episode, David Ha discusses the potential of model merging as a breakthrough in AI, highlighting its implications for evolutionary strategies and the broader AI landscape. He emphasizes the importance of open-ended discovery in AI development.

What Happened

David Ha shares insights into the concept of model merging, explaining how it involves combining solutions from multiple agents tackling the same problem. He notes the challenges inherent in this approach, particularly the impracticality of accessing model weights, especially when dealing with closed frontier models. The discussion also touches upon the significance of having diverse architectures among models, which complicates the merging process further.

Ha delves into his background in finance and how the financial crisis of 2008 shaped his understanding of extreme events. This experience led him to explore the evolutionary strategies in AI, particularly through neuroevolution. He contrasts the narrow focus of traditional neural networks on classification tasks with the broader potential of evolving networks that can create art or optimize gameplay. The conversation reveals Ha's journey from finance to AI and his fascination with the evolutionary aspect of intelligence development, which he believes can foster innovation in artificial intelligence.

Key Insights

Key Questions Answered

What is model merging in AI?

Model merging refers to the process of combining solutions from multiple AI agents that are all addressing the same problem. David Ha explains that this approach raises questions about how models combine their outputs and whether they can effectively merge more than two models at a time. However, a significant hurdle is that many advanced models are closed, making it difficult to access the necessary weights for merging.

How does David Ha connect his finance background to AI?

David Ha draws a direct connection between his experiences in finance, particularly during the 2008 financial crisis, and his current work in AI. This crisis helped him understand the nature of extreme events and tail effects, influencing his views on civilization and intelligence. His financial background instilled in him a recognition of the unpredictable elements in both finance and AI development.

What are evolutionary strategies in AI?

Evolutionary strategies in AI involve using concepts from biological evolution to optimize artificial intelligence systems. Ha highlights that rather than focusing solely on predefined objectives, these strategies encourage the exploration of new objectives and novelty. He believes that this broader approach has the potential to lead to more innovative AI solutions, as it embraces open-ended discovery.

How has evolutionary computation influenced AI in recent years?

According to Ha, many concepts from evolutionary computation have gained traction within the broader AI community over the past decade. When he joined Google in 2016, he observed the integration of evolutionary principles into deep learning, particularly in areas like neural architecture search. This has allowed AI researchers to optimize neural network architectures and develop more complex, capable AI systems.

What role does open-ended discovery play in AI?

Open-ended discovery is a critical concept in AI that encourages the exploration of new possibilities rather than adhering to fixed objectives. Ha emphasizes that this approach can lead to innovative breakthroughs in artificial intelligence, as it allows systems to evolve and adapt in ways that traditional methods may not facilitate. The notion of finding new objectives creates a dynamic landscape for AI development.