Case Study: An AI Meeting Assistant

An AI meeting assistant joins a call, writes live captions, and afterward produces a summary, a list of decisions, and action items with owners and dates. It also lets you search everything said across past meetings. It is a nice case because it chains several models, speech and language, and each link has its own error rate that multiplies into the final quality.

A meeting has two helpers. The court reporter types every word as it is said, quickly, and with some mistakes. The chief of staff later reads the transcript and writes the one-page summary, with who agreed to do what by when. If the reporter mishears a name, the chief of staff's summary inherits the error. Pipelines of models have the same property.

• -Audio capture: from the meeting platform, per speaker if possible.
• -Streaming speech recognition (ASR): turns audio into text in near real time, with a delay of under about a second for live captions.
• -Speaker diarization: labels who is speaking, so the transcript says who said what.
• -Transcript store: text with timestamps and speakers, searchable.
• -Summarization and extraction: after the meeting, an LLM writes a summary, decisions, and action items with owners and due dates.
• -Search and question answering: retrieval over past transcripts, with permissions, as in other RAG designs.

Two latency tiers matter. Live captions need a delay of a second or so and cannot wait for a large model. The summary can take a minute after the meeting ends, so it can use a bigger model and more careful steps.

Speech recognition is scored by word error rate, or WER: the number of word substitutions, deletions, and insertions needed to turn the system's transcript into the correct one, divided by the number of words in the correct transcript. The code implements it with the standard edit-distance method, then looks at how big a long meeting is for summarization. Press Run and compare the three systems.

System A is perfect, at 0%. System B has a word error rate of 12%: it wrote "building" for "billing" and dropped "on" before "monday", two errors in 16 words. System C scores 38%, with several dropped words and an extra one. A single error can matter more than a percentage suggests: "building page" changes the meaning, while a missing "on" does not. That is why WER is a starting point, and teams also look at errors on names, numbers, and jargon, often fixed with a custom vocabulary. The second part shows size. A 60-minute meeting is about 12,000 tokens, and a 3-hour meeting about 36,000. Modern models can often take that in one call, but for very long meetings, or to save cost, you can summarize pieces and combine them, which also lets you organize by topic.

• -ASR: word error rate overall and on names, numbers, and domain terms, across accents, noisy rooms, and crosstalk.
• -Diarization: how often words are assigned to the wrong speaker.
• -Summaries: faithfulness (nothing invented) and coverage (key points not missed), using a judge validated against human labels.
• -Action items: recall and precision against action items that a person listed, including whether the owner and date are right.
• -Online: how often users edit or delete the summary, and whether they open it again.

• -Consent and legal rules for recording differ by place. Announce recording, get consent where required, and give admins controls.
• -Retention and access: store transcripts only as long as needed, restrict who can read them, and redact sensitive data.
• -Hallucinated action items: the model invents a commitment nobody made. Require each item to link to the transcript line it came from.
• -Wrong attribution: assigning a task to the wrong person can cause real harm. Show the evidence, and let users correct it.
• -Noise and crosstalk: expect lower quality, and show a confidence indicator.