Worked Example: Autonomous & Multi-Modal Agents

A trading desk wants an AI that reads news, analyzes earnings, and suggests trades, with strict risk controls, because one bad trade can lose millions. Read that requirement closely: it explicitly asks for an AI co-pilot, not an autonomous trader. That distinction is the entire design.

This is Tier 9's ReAct and human-in-the-loop chapters, composed directly, with zero new mechanics needed. The agent reads real-time market data and news through tools, reasons about what it finds, and produces a recommendation with its confidence and reasoning attached, exactly the human-in-the-loop pattern from Tier 9. It never executes a trade autonomously. A human always makes the final call.

One hundred percent tracing here is not a best practice, it is a compliance requirement: every piece of data the agent saw, every reasoning step, every recommendation, timestamped and retained, because regulators will audit it. This is Tier 9's agent evaluation chapter's tracing discipline, applied at its highest possible stakes. A content classifier specifically screens for MNPI, material non-public information, before it ever reaches the agent's reasoning, since even surfacing it internally can itself be a compliance violation.

A media company with 10 million assets, images, video, documents, audio, needs a query like "find the product photo where someone is holding our blue widget next to a laptop" to actually work. Text search cannot do this. The answer is a multi-modal embedding model, CLIP-style, that projects both text and images into the same vector space, so a text query and a visually matching image end up as nearby vectors, the exact embedding-space intuition from Tier 7, extended across modalities instead of staying within text alone.

• -Video cannot be embedded as a whole. Extract keyframes, roughly one per second, and embed each one individually alongside the video's transcript.
• -A query then matches against both frame embeddings and transcript embeddings, since the same moment in a video might be findable by what is visually shown or by what is being said.
• -At 10 million assets, this is squarely Tier 8's vector database chapter's territory: an HNSW index is the only realistic choice for keeping search latency reasonable at this scale.