Imagine having an AI assistant that could listen to your meetings and generate perfect summaries. Sounds like a dream, right? While Large Language Models (LLMs) are getting better at summarizing text, spoken documents present a unique challenge. Think about it: ASR transcription errors, speaker misattributions, and the nuances of human conversation can easily trip up even the most sophisticated AI. So, how do we ensure these AI-generated summaries are truly helpful? New research explores the critical role of human feedback in optimizing these systems. Instead of relying solely on automated metrics like ROUGE or BERTScore, which struggle to capture the subtleties of spoken language, researchers are turning to human evaluators. They've developed specific evaluation criteria, including things like accuracy, relevance, and even the tone of the summary. This allows them to go beyond simply checking if the AI grabbed the right keywords and delve into whether the summary truly reflects the meeting's essence. Through two real-world case studies at Cisco, this research demonstrates how human input refines AI models. For example, evaluators might listen to a meeting and compare summaries generated by two different AI models, providing feedback on which summary captures the meaning and action items better. This feedback loop helps train the AI to understand not only *what* was said but *how* it was said. This research paves the way for creating smarter, more reliable AI assistants that can truly grasp the complexities of human communication. As AI-powered meeting summaries become integrated into collaborative applications, human feedback will play an increasingly important role in creating tools that boost productivity and communication.