Calling OpenAI, Anthropic, or Google directly is fine when you have one app, one model, and no real platform concerns. The moment you want fallback behavior, provider switching, budget controls, observability, or a clean way to mix hosted and self-hosted models, direct integrations start to feel brittle.
That is where AI LLM routers come in. In practice, teams use the terms LLM router, AI gateway, and model gateway almost interchangeably.