What is Kubernetes?
Kubernetes is open-source software that runs containers across a group of machines and keeps them running as declared. An operator describes the desired state, for example three copies of an inference server with one graphics processing unit (GPU) each, and Kubernetes schedules the containers onto machines with free capacity, restarts any that crash, and replaces them when a machine fails. It also provides networking between services, storage mounts for model weights, and a place to hold configuration and secrets.
Most AI platforms use Kubernetes or something very like it to manage inference servers, vector databases, agent runtimes, and the web applications in front of them. The major clouds sell managed versions, and the same deployment manifests work on an on-premises cluster.
Why Kubernetes matters
For a business, Kubernetes is what turns a rack of GPU servers into a dependable service. A regional retailer running a product search model and a customer service assistant can scale the search service up before a holiday promotion and down afterward, roll out a new model version a few copies at a time, and survive a server failure overnight without a call to the on-duty engineer. The cost is complexity. A cluster needs people who understand it, and GPU scheduling, driver versions, and storage for large models each add setup work. Smaller deployments often run a single server with plain containers and adopt Kubernetes only when they need more than one machine.