Glossary

Agent

An AI system capable of taking autonomous actions—executing code, calling APIs, accessing databases, or interacting with external systems—based on its reasoning. Agents are distinguished from simple inference systems by their ability to act, not just respond. In security terms, agents are principals with delegated authority.

Agentic AI

AI systems designed to operate with significant autonomy, making decisions and taking actions with minimal human oversight. Agentic AI amplifies both capability and risk, as autonomous action can compound errors at machine speed.

Attack Surface

The sum of all points where an attacker could attempt to enter or extract data from a system. In AI systems, the attack surface extends beyond traditional application boundaries to include training data, model artifacts, inference APIs, integration points, and downstream consumers of AI outputs.

Blast Radius

The extent of damage or impact that results from a security incident. In AI systems, blast radius is determined by what data the system can access, what actions it can take, and what downstream systems trust its outputs. Architectural controls aim to limit blast radius.

Context Window

The amount of information an AI model can consider when generating a response. In security terms, everything in the context window is accessible to the model, making context injection a vector for information disclosure and manipulation.

Control Plane

The components of a system responsible for configuration, policy enforcement, and orchestration—as opposed to the data plane, which handles actual workloads. In AI systems, the control plane includes model registries, deployment pipelines, and governance enforcement mechanisms.

Data Lineage

The ability to trace data from its origin through all transformations to its current state. In AI security, data lineage is foundational—without it, you cannot assess training data integrity, respond to data-related incidents, or verify compliance with data governance policies.

Data Plane

The components of a system that process actual workloads—in AI systems, this includes inference services, retrieval systems, and the data flows that feed models. Security controls must exist in both control and data planes.

Data Poisoning

An attack that corrupts training data to influence model behavior. Poisoning can be targeted (causing specific misclassifications) or general (degrading overall model quality). Unlike inference-time attacks, poisoning affects the model persistently.

Emergent Privilege

Capabilities that arise from the combination of individually authorized actions. An agent with document retrieval and email access has exfiltration capability, even if neither permission was intended to enable exfiltration. Emergent privilege is a key risk in agentic systems.

Feedback Loop

A cycle where an AI system's outputs become inputs to subsequent reasoning or action. Unsupervised feedback loops can amplify errors, as each iteration builds on potentially flawed previous outputs without human validation.

Fine-Tuning

The process of further training a pre-trained model on domain-specific data to adapt it for particular tasks. Fine-tuning introduces supply chain considerations—the fine-tuned model inherits properties from both the base model and the fine-tuning data.

Governance Theater

Governance activities that produce documentation and compliance artifacts without actually constraining system behavior. Policies that aren't technically enforced, reviews that don't change outcomes, and frameworks that aren't operationalized are governance theater.

Hallucination

When an AI model generates outputs that are plausible-sounding but factually incorrect or fabricated. While often discussed as an accuracy problem, hallucinations are also a security concern when they affect decisions or when users cannot distinguish hallucinated content from factual output.

Identity (Principal)

In security, an entity that can be authenticated, authorized, and held accountable for actions. AI systems—particularly agents—should be treated as principals with their own identities, distinct from the users or applications they serve.

Inference

The process of using a trained model to generate outputs from inputs. Inference is when AI systems interact with the world and when most security controls must operate. Also called "prediction" or "serving."

Inference Attack

An attack that extracts information from a model through its outputs—membership inference (determining if specific data was in training), model extraction (reconstructing the model), or attribute inference (deducing sensitive attributes). Inference attacks exploit what models inadvertently memorize.

Jailbreak

An attempt to bypass an AI model's safety constraints or content policies through crafted inputs. Jailbreaks exploit the gap between what models are instructed to refuse and what they can be manipulated into producing.

Least Privilege

The principle that any entity should have only the minimum permissions necessary to perform its function. In AI systems, least privilege applies to data access, tool capabilities, network connectivity, and integration permissions.

Memory (Agent)

Persistent context that an AI agent maintains across sessions or interactions. Memory enables continuity but creates security surfaces—memory can be poisoned, memory can leak information across trust boundaries, and memory creates persistence for attacks.

Model Provenance

The documented history of a model's origin, training, modifications, and deployment. Provenance enables supply chain security for models—knowing where a model came from, what it was trained on, and who modified it.

Model Registry

A system for storing, versioning, and managing model artifacts. Model registries are control plane infrastructure that enables provenance tracking, access control, and deployment governance.

Observability

The ability to understand a system's internal state from its external outputs—logs, metrics, and traces. In AI systems, observability must extend to reasoning chains, retrieval results, and tool invocations, not just operational metrics.

Orchestration Layer

The components that coordinate AI system behavior—routing requests, managing context, invoking tools, and assembling responses. The orchestration layer is often where security controls must be implemented, as it mediates between models and the systems they interact with.

Principal

See Identity.

Prompt Injection

An attack that manipulates AI behavior by including malicious instructions in input data. Direct prompt injection embeds instructions in user input; indirect prompt injection embeds instructions in content the model retrieves or processes.

RAG (Retrieval-Augmented Generation)

An architecture that supplements AI model responses with information retrieved from external data sources. RAG systems extend the attack surface to include retrieval systems and the data stores they access.

Reasoning Chain

The sequence of intermediate steps an AI system takes to produce an output. For agentic systems, the reasoning chain includes tool selection, parameter choices, and iterative refinement. Capturing reasoning chains is essential for auditing and incident investigation.

Shared Responsibility Model

The division of security obligations between service providers and customers. In AI, shared responsibility exists between model providers, platform providers, and organizations deploying AI systems. Misunderstanding shared responsibility creates security gaps.

Supply Chain (AI)

The dependencies that AI systems rely on—pre-trained models, training datasets, ML frameworks, inference infrastructure. AI supply chain security applies software supply chain principles to models, data, and ML-specific components.

Telemetry

Data collected about system operation—logs, metrics, traces, events. In AI systems, security-relevant telemetry includes not just operational data but reasoning context, retrieval results, and decision rationale.

Tool Use

The capability of AI agents to invoke external functions—querying databases, calling APIs, executing code, accessing files. Each tool represents a privilege grant, and tool combinations create emergent capabilities.

Trust Boundary

A point in a system where the level of trust changes—where data crosses from trusted to untrusted contexts, where privilege levels differ, or where different security policies apply. AI systems often obscure trust boundaries, treating all retrieved data as equally trustworthy.

Vector Database

A database optimized for storing and querying high-dimensional vectors, commonly used in RAG systems to enable semantic search. Vector databases are part of the AI attack surface—their contents can be poisoned, and their access controls determine what information AI systems can retrieve.