Agent Observability
- Definition
- Agent observability is the practice of recording and inspecting what an AI agent did during a task, including each model call, tool call, intermediate result, cost, and failure, so teams can debug…
Showing 20 of 90 terms
Agent Runtime
- Definition
- An agent runtime is the software environment that executes an AI agent: it runs the reasoning loop, calls the model, dispatches tool calls, manages state and memory, and enforces limits and permissions.
AI Accelerator (NPU, VPU)
- Definition
- An AI accelerator is a specialized chip or chip section built to run neural network inference efficiently, such as the neural processing units (NPUs) in laptops and phones and the vision processing…
AI Agent Framework
- Definition
- An AI agent framework is a software library or platform that supplies the building blocks for agents, including the reasoning loop, tool definitions, memory, and connections to models, so teams do not…
AI Gateway
- Definition
- An AI gateway is a proxy that sits between applications and model endpoints to enforce authentication, routing, rate limits, logging, and data policies for every request to an AI model.
AI Ownership
- Definition
- AI ownership means running AI on model weights, infrastructure, and data paths a company controls, instead of renting a provider's model per token. It suits companies with sensitive data or steady volume.
AI Security
- Definition
- AI security is the practice of protecting AI systems, including models, data, prompts, and agents, from attack and misuse, and of preventing a manipulated AI system from harming the business that runs…
Air-Gapped AI Deployment
- Definition
- An air-gapped AI deployment runs models on a network with no connection to the internet or to other networks, so data cannot leave and updates arrive only through controlled physical transfer.
Application Programming Interface (API)
- Definition
- An application programming interface (API) is a defined set of requests and responses that lets one program use another's functions or data without access to its internal code.
Batch Inference
- Definition
- Batch inference is the practice of running a model over a large set of records at once on a schedule, such as scoring every account overnight, instead of answering requests one at…
Canary Deployment
- Definition
- A canary deployment releases a new model or software version to a small share of real traffic first, compares its behavior with the current version, and widens the rollout only when the…
Central Processing Unit (CPU)
- Definition
- A central processing unit (CPU) is the general-purpose processor in every computer, built to run a few tasks very quickly in sequence, and it handles most of the work around an AI…
Citizen Developer
- Definition
- A citizen developer is an employee outside the IT department who builds applications, automations, or AI agents for their own team using no-code or low-code tools that IT has approved.
Cloud Computing
- Definition
- Cloud computing is the delivery of servers, storage, networking, and software over the internet from a provider's data centers, rented on demand and billed by use instead of bought outright.
Cloud GPU
- Definition
- A cloud GPU is a graphics processing unit rented by the hour or by reservation from a cloud provider, giving an organization AI hardware without buying or housing it.
Compute
- Definition
- Compute is the processing capacity, measured in processor time and operations per second, that an AI system consumes to train a model or to produce answers from a trained one.
Computer Vision Platform
- Definition
- A computer vision platform is software that covers the full life cycle of vision models: collecting and labeling images, training, deploying to cameras or servers, and monitoring results in production.
Confidential Computing
- Definition
- Confidential computing protects data while it is being processed by running workloads in hardware-isolated, encrypted memory that the host operating system, the cloud operator, and other tenants cannot read.
Container
- Definition
- A container is a packaged unit of software that bundles an application with its libraries and settings so it runs the same way on any machine. In AI work, containers carry models,…
Continuous Batching
- Definition
- Continuous batching is an inference server technique that adds new requests to the running batch and removes finished ones at every generation step, keeping the GPU busy and raising throughput.
Data Pipeline
- Definition
- A data pipeline is an automated sequence of steps that moves data from where it is produced to where it is used, transforming, validating, and loading it along the way.
Deep Learning Framework
- Definition
- A deep learning framework is a software library that supplies the building blocks for defining, training, and running neural networks, handling the underlying math and GPU acceleration so developers describe the model…
Digital Twin
- Definition
- A digital twin is a software model of a physical asset, process, or facility, kept current with sensor and operational data, so planners can monitor it, test changes, and predict its behavior…
Edge AI
- Definition
- Edge AI is the running of machine learning models directly on local devices such as cameras, gateways, industrial PCs, and phones, so decisions are made where the data is captured.
Edge Computing
- Definition
- Edge computing is the processing of data on or near the devices that generate it, such as sensors, machines, and store systems, instead of sending everything to a central data center or…
Elastic Cloud Computing
- Definition
- Elastic cloud computing is the ability to add or remove computing resources automatically as demand changes, so capacity grows during a peak and shrinks back when the peak passes.
Energy-Aware Routing (Green AI)
- Definition
- Energy-aware routing is a form of model routing that includes energy use in the decision, sending each request to the smallest model, hardware, or time slot that meets its quality needs.
Enterprise AI Platform
- Definition
- An enterprise AI platform is the shared software layer on which a company builds and runs its AI applications, combining model access, data connections, identity and access control, agent tooling, monitoring, and…
Feature Store
- Definition
- A feature store is a data system that defines, stores, and serves the input variables machine learning models use, so the same feature definitions feed both model training and live predictions.
Governance Plane
- Definition
- A governance plane is the layer of an AI platform that enforces identity, permissions, policy, and audit logging on every model call and agent action, separate from the models themselves.
GPU Memory (VRAM)
- Definition
- GPU memory (VRAM) is the fast memory on a graphics card that holds a model's weights and working data during inference, and its size decides which models can run and how many…
Graphics Processing Unit (GPU)
- Definition
- A graphics processing unit (GPU) is a processor built to run thousands of simple calculations at once, which makes it the standard hardware for training and running neural networks.
Hosted Model
- Definition
- A hosted model is an AI model that a provider runs on its own infrastructure and offers to customers through an API, so the customer sends data in and receives results without…
Hybrid Deployment
- Definition
- A hybrid deployment runs part of an AI system on infrastructure the organization owns and part in a public cloud, placing each workload where its data sensitivity, cost, and capacity needs are…
Inference
- Definition
- Inference is the stage in which a trained model is used to produce predictions or generated output for new inputs, as distinct from training, which is where the model learns.
Inference Endpoint
- Definition
- An inference endpoint is the network address where a deployed model accepts requests and returns predictions or generated text, whether the model runs in a vendor's cloud or on the company's own…
Inference Server
- Definition
- An inference server is the software that loads a trained model onto GPUs, accepts requests over the network, and schedules them for efficient generation, turning model weights into a usable service.
Kubernetes
- Definition
- Kubernetes is open-source software that runs and manages containers across a cluster of machines, restarting failed workloads, scaling copies up and down, and assigning GPUs to the services that need them.
KV Cache
- Definition
- The KV cache is the GPU memory an inference server uses to store the attention keys and values of every token already processed, so each new token is generated without recomputing the…
LLM Observability
- Definition
- LLM observability is the practice of recording and analyzing what a language model application does in production, including prompts, responses, tool calls, latency, token usage, errors, and quality signals.
Low Code
- Definition
- Low code is a way of building applications with visual tools and prebuilt components while allowing developers to add hand-written code where the visual blocks cannot express the needed logic or integration.
Machine Learning Hardware Requirements
- Definition
- Machine learning hardware requirements are the processors, memory, storage, and network capacity a project needs, which depend on whether the work is classic machine learning, deep learning training, or language model inference.
Machine Learning Pipeline
- Definition
- A machine learning pipeline is an automated, repeatable sequence of steps that turns raw data into a trained, evaluated, and deployable model, so that retraining runs the same way every time.
Managed AI Service
- Definition
- A managed AI service is an arrangement in which a provider operates an AI system on the customer's behalf, handling hosting, updates, monitoring, security patches, and support, while the customer uses the…
MLOps
- Definition
- MLOps (machine learning operations) is the set of practices and tools for moving machine learning models into production and keeping them accurate, monitored, and governed over their working life.
Model Context Protocol (MCP)
- Definition
- The Model Context Protocol (MCP) is an open standard that lets AI applications connect to tools and data sources through one common interface, so any compliant agent can use any compliant server.
Model Deployment
- Definition
- Model deployment is the process of moving a trained model out of the lab and into a running system where it receives real requests, returns predictions, and is monitored, secured, and updated…
Model Drift
- Definition
- Model drift is the gradual or sudden decline in a deployed model's accuracy because the data it sees in production, or the patterns behind that data, no longer match what it learned…
Model Maintenance
- Definition
- Model maintenance is the ongoing work that keeps a deployed model accurate, secure, and compliant, including retraining, recalibration, software updates, revalidation, and eventual retirement.
Model Monitoring
- Definition
- Model monitoring is the ongoing tracking of a deployed model's inputs, outputs, accuracy, and operational health, with alerts that tell its owners when performance degrades or the data changes.
Model Registry
- Definition
- A model registry is a central catalog of trained models that records each version's lineage, evaluation results, approvals, and lifecycle stage, so teams know exactly which model is running where.
Model Routing
- Definition
- Model routing is the practice of sending each AI request to the most suitable model, local or cloud, based on the task type, cost, latency requirement, and sensitivity of the data involved.
Model Serving
- Definition
- Model serving is the software and infrastructure that keeps a trained model loaded in memory, accepts requests over a network, runs inference, and returns results reliably enough for applications to depend on.
Model Size
- Definition
- Model size is the scale of a model, usually expressed as its parameter count, which determines how much memory it needs, how fast it runs, and what hardware can serve it.
Model-Driven Architecture
- Definition
- Model-driven architecture is a way of building software in which a formal model of the business domain, its objects, relationships, and rules, is the primary artifact, and databases, interfaces, and application code…
Multi-Cloud
- Definition
- Multi-cloud is the use of services from two or more cloud providers within one organization, whether by deliberate design or because different teams and acquisitions chose different vendors.
Multi-Tenancy
- Definition
- Multi-tenancy is an architecture in which one running instance of a platform serves several separate customers or business units, called tenants, keeping each tenant's data, users, and configuration isolated from the others.
No Code
- Definition
- No code is an approach to building software through visual tools, forms, and prebuilt components instead of written programming, letting business users assemble applications, workflows, and AI agents themselves.
On-Premises AI
- Definition
- On-premises AI is AI that runs on servers physically located in an organization's own facilities, keeping models, prompts, and data inside buildings and networks the organization controls.
ONNX (Open Neural Network Exchange)
- Definition
- ONNX (Open Neural Network Exchange) is an open file format for trained machine learning models that lets a model built in one framework run in a different runtime, programming language, or device.
Open-Source Model
- Definition
- An open-source model is an AI model whose weights, training code, and ideally training data are published under a license that permits anyone to use, study, modify, and redistribute it.
Open-Weight Model
- Definition
- An open-weight model is an AI model whose trained weights are published for download and self-hosting, usually without the training data, and often under a license that restricts certain uses.
OpenAI-Compatible API
- Definition
- An OpenAI-compatible API is a model-serving interface that accepts the same request and response format as OpenAI's API, so most client libraries and applications can switch models by changing a URL.
Private AI
- Definition
- Private AI is artificial intelligence that runs on infrastructure the organization controls, using models it owns or has the rights to run, so that prompts, documents, and outputs never leave its environment.
Private AI Platform
- Definition
- A private AI platform is the software layer that runs models, agents, and integrations on an organization's own infrastructure, with identity, access control, and audit logging built in.
Production Environment
- Definition
- A production environment is the live set of servers, software, and data where an AI system serves real users and real business processes, as distinct from the development and test environments used…
Prompt Caching
- Definition
- Prompt caching stores the processed form of a prompt prefix, such as a long system prompt or document, so later requests that begin the same way skip that work and respond faster.
PyTorch
- Definition
- PyTorch is an open-source deep learning framework, started at Meta and now governed by the PyTorch Foundation, that has become the most widely used tool for building and training neural networks, including…
Quantization
- Definition
- Quantization stores a model's weights in lower-precision numbers such as FP8, INT8, or INT4 so the model needs less memory and runs faster, at the cost of a small loss in accuracy.
Rate Limiting
- Definition
- Rate limiting caps how many requests or tokens a user, application, or team may send to an AI model in a given period, protecting capacity and controlling cost.
Real-Time Inference
- Definition
- Real-time inference is serving a model's prediction on demand, within milliseconds to a few seconds of the request, so that the result can be used inside a live transaction or conversation.
Role-Based Access Control (RBAC)
- Definition
- Role-based access control (RBAC) is a method of managing permissions in which access rights are attached to job roles, and people receive access by being assigned to the roles that match their…
Scalable AI
- Definition
- Scalable AI is an AI deployment designed to keep performing as users, data volume, and use cases grow, through capacity planning, load testing, and architecture that adds compute without redesign.
Secrets Management
- Definition
- Secrets management is the practice of storing, distributing, rotating, and auditing the credentials that software and AI agents use to reach other systems, such as API keys, passwords, and tokens.
Self-Hosted AI
- Definition
- Self-hosted AI is AI software and models that an organization installs and operates on servers it manages, rather than consuming them as a vendor's hosted service. The servers can be on-premises or…
Shadow Mode
- Definition
- Shadow mode runs a new model on live production inputs alongside the current system, records its outputs for comparison, and takes no action on them, so the new model can be judged…
Single Sign-On (SSO)
- Definition
- Single sign-on (SSO) is an authentication method that lets people sign in once through a central identity provider and then use many applications, including AI tools, without separate passwords.
Single-Tenant Deployment
- Definition
- A single-tenant deployment is an AI system in which the servers, model instance, databases, and configuration serve one organization only, with nothing shared with other customers of the provider or platform.
Small Language Model (SLM)
- Definition
- A small language model (SLM) is a language model with a few billion parameters or fewer, sized to run on a single GPU, a CPU, or an edge device while handling narrow,…
Software Development Kit (SDK)
- Definition
- A software development kit (SDK) is a packaged set of libraries, tools, and documentation that lets developers build on a platform or call its API from a specific programming language.
Sovereign AI
- Definition
- Sovereign AI is AI capability that a nation or organization controls within its own legal jurisdiction, covering the models, the data, the infrastructure, and the rules that govern them.
Speculative Decoding
- Definition
- Speculative decoding speeds up text generation by letting a small draft model propose several tokens that the large model then verifies in one pass, producing the same output faster.
Streaming (LLM Responses)
- Definition
- Streaming is the delivery of a language model's response token by token as it is generated, so the user sees text appear immediately instead of waiting for the full answer.
Tensor Processing Unit (TPU)
- Definition
- A tensor processing unit (TPU) is a chip Google designed specifically for the matrix arithmetic of neural networks, available only through Google's cloud and used mostly for large-scale training.
TensorFlow
- Definition
- TensorFlow is an open-source deep learning framework developed by Google for building, training, and deploying neural networks, known for its production deployment tooling and its support for mobile and edge devices.
Time to First Token (Latency)
- Definition
- Time to first token is the delay between sending a prompt and receiving the first piece of the model's answer, the latency figure users notice most in an interactive AI feature.
Tokens per Second (Throughput)
- Definition
- Tokens per second measures how fast a language model produces text, either for a single response or as the total output of a server across all users at once.
Total Cost of Ownership (AI)
- Definition
- Total cost of ownership for AI is the full cost of running an AI capability over its life, including hardware or GPU time, power, people, software, model licenses, and support, rather than…
Type System
- Definition
- A type system in an AI platform is a catalog of formally defined object types, with attributes, relationships, and rules, that describes data, models, and services so applications are built against types…
Zero Trust
- Definition
- Zero trust is a security model in which no user, device, or service is trusted because of its network location, and every request to every resource is authenticated and checked against policy.
No terms match your search.
Try a broader keyword or another category.