What is energy-aware routing?
Energy-aware routing is model routing with electricity as one of the inputs to the decision. Every generated token costs energy, and a large model costs many times more per token than a small one. The router sends each request to the least energy-intensive option that meets the quality requirement: a small language model for simple tasks, a quantized model where full precision is unnecessary, and the full model only when the task demands it. Non-urgent work can also be batched and scheduled for hours when the grid is cleaner.
The term falls under Green AI, the broader effort to reduce the energy and carbon footprint of AI.
Why energy-aware routing matters
Energy is a direct operating cost of private AI, because the company pays the power bill for its own GPUs, and an indirect cost of hosted AI through pricing. It is also a reporting item for companies with sustainability commitments, and a router’s logs supply the usage data that reporting needs. A county government that classifies public records requests overnight on a small model, reserving the large model for complex legal review, cuts both its power draw and its GPU requirement.
What to watch for
- Vendor energy claims are hard to verify, so ask for measurements per request or per thousand tokens on your own workload.
- Routing to a smaller model only saves energy if the answer is right the first time, so measure retries and escalations.
- Idle GPUs still draw power, so right-sizing the hardware matters as much as per-request routing.