Running LLM Workloads in Production: An Operations Playbook for Teams That Did Not Sign Up for This
Somewhere in the past two years, AI quietly became an operations problem. The proof of concept your product team shipped - a support-ticket summarizer, a natural-language search box, a code-review assistant - graduated into a production dependency, and now it pages you. The failure modes are unfamiliar: latency distributions with tails measured in tens of seconds, upstream providers that throttle without warning, costs that scale with user enthusiasm rather than infrastructure size, and outputs that can be wrong in ways a health check will never catch.
