What is elastic cloud computing?
Elastic cloud computing is the automatic scaling of rented computing resources up and down to match demand. A rule or a scheduler watches a metric such as request queue length or GPU utilization, starts more instances when the metric climbs, and shuts them down when it falls. The organization pays for the extra capacity only during the hours it was running.
Elasticity is what separates cloud computing from a rented server. A rented server has fixed capacity for the length of the lease. An elastic service can run on two machines at three in the morning and forty at noon.
Elastic cloud computing in practice
An online retailer runs a product-description assistant and a customer-service chatbot on a language model. Traffic in the week before a major holiday is many times the January baseline. With elastic scaling, the inference cluster adds GPU instances as the request queue lengthens and releases them after the rush, so the retailer neither pays for idle hardware all year nor turns customers away in December.
What to watch for
- GPU instances take minutes to start and load a model, so scaling rules must react before the queue is already full.
- GPU capacity in a region can run out at exactly the moment everyone else is scaling up, so reserve a baseline.
- Elastic bills are hard to predict, so set spending caps and alerts before the first peak.