fbpx
Podcast: How to Evaluate LLMs and RAG Applications with Pasquale Antonante Podcast: How to Evaluate LLMs and RAG Applications with Pasquale Antonante
Learn about cutting-edge developments in AI and data science from the experts who know them best on ODSC’s Ai X Podcast.... Podcast: How to Evaluate LLMs and RAG Applications with Pasquale Antonante

Learn about cutting-edge developments in AI and data science from the experts who know them best on ODSC’s Ai X Podcast. Each week we release an interview with a leading expert, core contributor, experienced practitioner, or acclaimed instructor who is helping to shape the future of the AI industry through their work or research.

In this episode, Pasquale Antonante, Co-Founder & CTO of Relari AI, joins us to discuss evaluation methods for LLM and RAG applications. Since his time as a PhD student at MIT, Pasquale has been interested in understanding reliability in complex AI systems. Now, at Relari AI, they are building an open-source platform to simulate, test, and validate complex generative AI (GenAI) applications.

Inspired by the testing methodologies used in the autonomous vehicle industry, Relari AI’s is creating an innovative approach to improving generative AI and RAG applications. 

During this discussion, you’ll hear about topics like the complexity of GenAI workflows, the challenges in evaluating LLM and RAG systems, and various evaluation methods such as reference-based, and synthetic data-based approaches. You’ll also explore metrics like precision, recall, faithfulness, and relevance, and compare GPT auto-evaluators with simulated user feedback.

Podcast: How to Evaluate LLMs and RAG Applications with Pasquale AntonanteLastly, we’ll highlight Relari’s continuous-eval open-source project and explore the future of leveraging synthetic data for LLM finetuning.

Start listening now to get the full impact of Pasquale’s extensive knowledge and expertise in evaluating LLM and RAG applications and don’t forget to subscribe to ODSC’s Ai X Podcast to ensure you never miss an episode. Finally, like what you hear? Leave a review or share it with a friend! You can listen on Spotify, Apple, and SoundCloud.

To take an even deeper dive into AI topics and tools, and their effects on society at large, join us at one of our upcoming conferences, ODSC APAC (August 13th, Virtual), ODSC Europe (September 5-6, Hybrid, or ODSC West (October 29-31, Hybrid). 

ODSC Team

ODSC Team

ODSC gathers the attendees, presenters, and companies that are shaping the present and future of data science and AI. ODSC hosts one of the largest gatherings of professional data scientists with major conferences in USA, Europe, and Asia.

1