Don't Ship Skills Without Evals — Philipp Schmid, Google DeepMind

Summary

This talk emphasizes the critical need for evaluation (evals) when integrating skills into AI agents, especially those built for consumers. The speaker highlights that while many developers build AI-powered skills, a significant portion lack proper testing, leading to unpredictable failures in production due to the non-deterministic nature of agents. The key takeaway is that robust evaluation processes are essential to ensure the reliability and effectiveness of AI skills for end-users.