The Economics of LLM Calls: Tokens & Pricing

A CEO tells the team an AI assistant costing $52,000 a month needs to drop to $15,000, without users noticing any drop in quality. That specific number, a roughly 70% cut with no visible quality loss, sounds impossible until you notice the detail buried in the request: 70% of the actual queries are simple lookups that never needed the most expensive model to begin with. This chapter is about the handful of concrete levers that make that cut genuinely achievable.

This prints a roughly 10x cost ratio between tiers, a realistic order of magnitude in real pricing between a flagship model and a smaller, faster model from the same provider family. Multiply either number across a million monthly queries and the gap between "route everything to the premium model" and "route intelligently" becomes a very large line item.

Not every query needs the most capable model. A simple FAQ lookup and a complex multi-step financial analysis do not deserve identical treatment, one is a waste of capability and money, the other genuinely needs it. Tiered routing classifies incoming requests (often with a small, cheap classifier model, or even a rule-based heuristic on query length and keywords) and sends simple requests to a cheaper, faster model while reserving the expensive model for requests that actually need its extra capability.

If the exact same question gets asked repeatedly (a common FAQ, a popular product description), there is no reason to pay for a fresh model call every single time. An exact-match cache stores the response keyed by the literal input and serves repeats instantly and for free. A semantic cache goes further, using embeddings (from the previous chapter) to catch queries that are phrased differently but mean the same thing, "how do I get a refund" and "can I get my money back" hitting the same cached answer. Caching gets a full, dedicated treatment in the production tier ahead.

Work that does not need an instant response (nightly report generation, bulk content classification, offline data labeling) can often be submitted through a provider's batch API, processed asynchronously within a longer window (commonly 24 hours), at a meaningfully discounted rate, frequently 50% off standard pricing, since the provider can schedule that work into otherwise-idle capacity instead of guaranteeing it real-time priority.