Why You Can't Go Off Vibes in Software Development with Stephen Poletto, Field CTO at Span - Modern CTO Recap

Podcast: Modern CTO

Published: 2026-03-19

Duration: 3350

Guests: Stephen Poletto

What Happened

Stephen Poletto, Field CTO at Span, argues against the trend of making software development decisions based on hype or 'vibes' rather than data. He introduces the concept of 'harness engineering' to ensure AI agents operate within desired constraints, addressing the issue of AI-generated code causing production incidents, such as those experienced by AWS.

Span assists engineering leaders in adapting to the rapid changes in software development practices brought about by AI. Poletto cites Ramp as an innovative organization effectively utilizing AI, while also noting Jack Dorsey's controversial layoffs at Block, attributing them to perceived AI efficiency. This dichotomy highlights the debate between reducing workforce for efficiency or using AI to tackle previously unfeasible projects.

The potential of AI to compress the software development cycle by reducing handoffs and coordination costs is discussed. However, organizations face challenges in quantifying the ROI of AI tools and understanding their true impact on productivity. Poletto mentions a study by Meter, which found a discrepancy between engineers' perceived productivity with AI tools and actual metrics showing a decrease.

Span's approach involves integrating with development toolchains to analyze code workflow and AI tool engagement, helping identify bottlenecks and optimize processes. They offer both software tools and consultative services, providing targeted recommendations based on industry benchmarks and customer data.

Stephen Poletto, with a background at companies like Lattice, Dropbox, and Apple, joined Span due to his relationship with CEO Jayzak Stein. His role involves sales, customer support, and consulting, rather than leading engineering efforts.

The podcast touches on the existential question of human value in the face of advancing AI, with references to Yuval Noah Harari's 'Homo Sapiens' highlighting human intellect and coordination as key factors in human dominance over other species. There is concern about a future where autonomous robots could outnumber humans if controlled by bad actors.

Key Insights

• Stephen Poletto emphasizes the need for data-driven decision-making in software development, cautioning against relying on hype or doomsday predictions. This approach is crucial as AI continues to reshape development practices.
• The concept of 'harness engineering' is introduced to create constraints for AI agents, ensuring they operate effectively and prevent issues such as the production incidents experienced by AWS with AI-generated code.
• Span integrates with development toolchains to analyze code workflows, providing insights into AI tool engagement and identifying bottlenecks. This helps organizations optimize their AI development processes.
• A study by Meter found a discrepancy between engineers' perceived productivity with AI tools and actual productivity metrics, highlighting the challenge organizations face in quantifying the ROI and true impact of AI tools.