AI Cybersecurity Terms and Glossary

The intersection of artificial intelligence and cybersecurity has produced a distinct technical vocabulary that governs how practitioners, regulators, and service providers communicate about threats, defenses, and compliance obligations. This page defines the core terms structuring the AI cybersecurity service sector, establishes classification boundaries between related concepts, and maps the regulatory and standards frameworks where these definitions carry formal weight. Accurate terminology is not a stylistic preference — in procurement, incident response, and regulatory reporting, definitional precision determines liability.

Definition and scope

AI cybersecurity refers to the application of machine learning, large language models, automated reasoning systems, and related AI techniques to the detection, prevention, analysis, and response to cyber threats — and, reciprocally, to the security disciplines required to protect AI systems themselves from adversarial attack, manipulation, or unauthorized access.

The National Institute of Standards and Technology (NIST AI Risk Management Framework, NIST AI 100-1) distinguishes between AI for cybersecurity (AI used as a defensive or offensive security tool) and cybersecurity for AI (protecting AI models and pipelines from attack). Both directions are active service categories with distinct vendor markets and compliance requirements.

Core terms in scope:

  1. Adversarial machine learning (AML) — Techniques that manipulate AI model inputs to produce incorrect outputs; defined formally in NIST SP 1270 and further classified into evasion, poisoning, extraction, and inference attacks.
  2. AI red-teaming — Structured adversarial testing of AI systems to surface failure modes; the term was formalized for federal use in the White House Executive Order 14110 on Safe, Secure, and Trustworthy AI (2023).
  3. Model poisoning — An attack in which malicious data is injected into a training dataset to corrupt model behavior at inference time.
  4. Prompt injection — An input-manipulation attack specific to large language models in which adversarial instructions override system prompts or intended behavior.
  5. Explainability / interpretability — The degree to which a model's decision logic can be audited; a compliance-relevant concept under NIST SP 800-218A (Secure Software Development for AI).
  6. Zero-day AI vulnerability — A previously undisclosed flaw in an AI system's architecture or training pipeline with no available patch at time of discovery.
  7. AI governance — The organizational policies, technical controls, and audit processes governing the lifecycle of AI systems, addressed in the NIST AI RMF through its Govern, Map, Measure, and Manage functions.
  8. Threat intelligence augmentation — The use of AI to process and correlate threat feeds at scale beyond human analyst capacity, a capability category tracked by the Cybersecurity and Infrastructure Security Agency (CISA).

How it works

AI cybersecurity services operate across two structural modes that are often confused but require separate procurement and compliance frameworks:

Mode 1 — AI as defensive tool. Security operations centers deploy AI-driven platforms for anomaly detection, log correlation, and behavioral analysis. These systems ingest event data from endpoints, networks, and cloud environments, apply supervised or unsupervised learning models trained on known threat patterns, and generate alerts or automated responses. The MITRE ATT&CK framework provides the taxonomic vocabulary most commonly used to label model outputs in this mode, organizing attacker behaviors into 14 tactic categories and hundreds of technique identifiers.

Mode 2 — AI as attack surface. When organizations deploy AI models in production — for fraud detection, identity verification, customer interaction, or infrastructure automation — those models themselves become targets. Adversarial actors may submit crafted inputs to force misclassification, extract training data through membership inference attacks, or corrupt upstream training pipelines through supply-chain poisoning. NIST SP 1270 defines 4 primary attack categories (evasion, poisoning, privacy, and abuse) with subcategories relevant to procurement and vendor assessment.

The boundary between the two modes is not static. A model deployed defensively in Mode 1 is simultaneously an attack surface for Mode 2 techniques. Service providers addressing the AI cyber listings sector must specify which mode their capabilities address.

Common scenarios

Three deployment patterns account for the majority of AI cybersecurity service engagements:

Security Operations Center (SOC) augmentation. AI systems process high-volume telemetry — often millions of events per day per enterprise — to reduce analyst alert fatigue. The AI filters, prioritizes, and clusters events before human review.

AI model vulnerability assessment. Third-party assessors apply adversarial ML test suites against a client's deployed models to identify susceptibility to evasion or extraction attacks. This service category maps to the AI red-teaming mandates referenced in Executive Order 14110.

AI supply chain security. Organizations audit the training data provenance, third-party model components, and fine-tuning pipelines of AI systems they import. CISA's guidance on AI software supply chain security frames this as an extension of existing software bill of materials (SBOM) practices, extending SBOM concepts to model cards and dataset documentation.

For a structured view of service providers operating across these scenarios, see the AI Cyber Listings directory.

Decision boundaries

Terminological precision matters most when classification determines regulatory obligation. Three critical boundaries govern professional and regulatory usage:

Cybersecurity for AI vs. AI for cybersecurity. These are distinct compliance tracks. NIST AI RMF governs the former; NIST SP 800-53 Rev 5 controls apply to the latter when federal information systems are involved.

Adversarial ML vs. traditional malware. Adversarial ML attacks exploit model mathematics rather than software vulnerabilities. Standard endpoint detection tools do not detect model evasion attacks — separate tooling and assessment methodologies apply.

Automated decision vs. AI-assisted decision. Regulatory exposure differs between systems that autonomously act (automated) and those that produce recommendations for human review (assisted). The NIST AI RMF's Govern 6.2 function addresses accountability allocation across this boundary.

Professionals navigating service categories and vendor claims against these definitions should reference the AI Cyber Authority directory purpose and scope for sector classification conventions used across this reference property. Additional guidance on navigating structured terminology resources is available through how to use this AI cyber resource.

References

📜 1 regulatory citation referenced  ·  🔍 Monitored by ANA Regulatory Watch  ·  View update log

Explore This Site

Regulations & Safety Regulatory References Tools & Calculators Password Strength Calculator