We've all done RAG, now what? - Practical AI Recap

Podcast: Practical AI

Published: 2025-09-29

Duration: 44 min

Guests: Rajeev Shah

Summary

The episode discusses the evolution and current state of Retrieval-Augmented Generation (RAG) in AI, emphasizing its widespread use in businesses for internal information retrieval and customer support. Rajeev Shah shares insights on the challenges of integrating AI solutions into organizational workflows.

What Happened

Rajeev Shah, Chief Evangelist at Contextual AI, joins Daniel Wightnack to explore the advancements in Retrieval-Augmented Generation (RAG) and its applications in AI. They reflect on the excitement around AI two and a half years ago, noting the practical developments in areas like code completion and chatbots. Shah highlights that while AI capabilities have dramatically improved, challenges remain in integrating AI solutions into workflows effectively.

The episode delves into the misconception around training AI models with company-specific data. Shah explains that instead of training models, it's more about retrieving the relevant information and providing context to the AI to generate appropriate responses. This approach is crucial for applications in different industries, such as healthcare, where guidelines can vary dramatically depending on context.

Shah elaborates on the concept of context engineering, explaining how it differs from traditional model training. By manipulating inputs and using retrieval methods, organizations can better tailor AI outputs to their specific needs without extensive retraining. This approach underscores the importance of understanding the broader system in which AI operates.

Discussing the practical challenges in deploying RAG at scale, Shah notes that while it's easy to build a proof of concept, scaling to handle large volumes of data or diverse queries presents significant hurdles. He emphasizes the need for continuous evaluation and iteration to maintain accuracy and performance in production environments.

Shah addresses the hype around generative AI and the importance of focusing on practical applications that provide real value to organizations. He cautions against being drawn into flashy demos and urges businesses to concentrate on solving actual problems with AI rather than pursuing science experiments that don't integrate well into existing systems.

The conversation touches on the evolving role of data scientists in the age of AI, with Shah expressing optimism about the potential for reasoning models to assist in complex problem-solving. He also highlights the ongoing need for human intervention to bridge gaps in AI workflows, particularly in evaluation and decision-making processes.

Key Insights