Reinventing the Python Notebook with Akshay Agrawal - Software Engineering Daily Recap

Podcast: Software Engineering Daily

Published: 2026-03-10

Duration: 46 min

Summary

Akshay Agrawal discusses the limitations of traditional Python notebooks and introduces Marimo, an open-source tool designed to improve reproducibility, version control, and interactivity for data scientists and researchers.

What Happened

In this episode, Akshay Agrawal shares his journey from Google Brain to creating Marimo, a next-generation Python notebook aimed at addressing the shortcomings of traditional notebooks like Jupyter. He highlights the popularity of interactive notebooks in data science and research but points out that as projects become more complex, traditional notebooks often encounter significant issues such as hidden state, non-reproducible execution, and poor version control. These limitations can hinder the transition from exploratory analysis to production environments.

Akshay elaborates on the specific challenges he faced while using traditional notebooks, particularly the concept of hidden state. He explains how running cells in an imperative paradigm can lead to inconsistencies and confusion, especially when working on more extensive projects. Additionally, he discusses the ergonomics of notebook usage within modern software projects, noting that the default file format for Jupyter notebooks complicates version control and code reuse. To combat these issues, Marimo is designed to be stored as pure Python, allowing for better integration with Git and more robust software engineering practices.

The episode also delves into the shareability of notebooks, where Akshay emphasizes that traditional notebooks require collaborators to recreate entire environments, which can slow down interactivity and feedback. Marimo addresses this by enabling notebooks to function as interactive web apps, allowing users to manipulate variables through UI components like sliders. This approach not only enhances collaboration but also retains the exploratory vibe of traditional notebooks while elevating them to a more production-ready state.

Key Insights

Key Questions Answered

What are the limitations of traditional Python notebooks?

Akshay outlines several key limitations of traditional Python notebooks, particularly focusing on the concept of hidden state. In an imperative environment like Jupyter, running a cell can mutate memory, leading to a scenario where variables in memory do not match the code on the page. This can result in significant confusion and errors, especially in complex projects where the relationships between cells become obscure. Additionally, the default file format of Jupyter notebooks complicates version control and makes it challenging to reuse code across different projects.

How does Marimo improve upon traditional notebooks?

Marimo is designed to address the issues faced with traditional notebooks by storing notebooks as pure Python scripts. This allows users to version control their work using Git, making it easier to manage changes and collaborate with others. Furthermore, Marimo enhances reproducibility, offering a more structured environment for users who need to transition from exploration to production. The ability to share notebooks as interactive web apps also sets Marimo apart, enabling others to engage with the content without needing to recreate the entire Python environment.

Why is hidden state a problem in traditional notebooks?

Hidden state refers to the situation where the current state of a notebook does not reflect the actual code being executed, leading to inconsistencies. Akshay explains that as users run cells in a Jupyter notebook, they may forget to execute other dependent cells, causing discrepancies between the displayed code and the underlying data. This can become particularly frustrating when working on long analyses, as users may not realize that changes in one cell can affect the output in another, leading to wasted time and effort.

What role does collaboration play in the functionality of notebooks?

Collaboration is a significant challenge with traditional notebooks, as Akshay points out that sharing results often requires collaborators to have the same Python and Jupyter setup. This can be a barrier to effective teamwork, as it slows down feedback loops and interactivity. Marimo addresses this by allowing notebooks to function as interactive web apps, which means collaborators can engage with the analysis without needing to set up their environments, thus streamlining the collaborative process.

How does Marimo facilitate better code reuse?

One of the key features of Marimo is that it allows notebooks to be stored as standard Python files. This design choice improves code reuse by enabling users to extract functions and modules from notebooks and integrate them into other projects seamlessly. Akshay mentions that traditional notebooks often lead to duplicated code, making it difficult to maintain and update analyses. By structuring notebooks as pure Python, Marimo promotes better software engineering practices, making it easier to manage code across different notebooks and applications.