Short Summary
An AI security assessment should verify what AI systems exist, what data and privileges they can access, how models and dependencies are sourced, whether AI-specific attacks are tested, and whether monitoring and incident response cover the complete AI lifecycle. It should combine conventional cybersecurity controls with AI-specific controls for prompt injection, sensitive information disclosure, poisoning, improper output handling, excessive agency, adversarial inputs and retrieval weaknesses.
Why AI Security Needs Its Own Assessment
AI systems inherit familiar application, cloud, identity, API and supply-chain risks, but they introduce additional attack paths. Training or retrieval data can be manipulated, prompts and external content can influence model behavior, model outputs can be unsafe when passed directly into applications, and agentic systems may call tools or perform actions with enterprise impact.
NIST’s 2025 adversarial machine learning taxonomy addresses evasion, poisoning, privacy attacks and misuse. The OWASP Top 10 for LLM Applications 2026, released in August 2026, covers prompt injection, sensitive information disclosure, excessive agency, supply-chain risks, data and model poisoning, unbounded consumption, misinformation, hidden context exposure, vector and embedding weaknesses, and improper output handling. For systems where AI agents plan, hold memory and call tools, the OWASP Top 10 for Agentic Applications 2026 adds agent-specific risks such as goal hijack, tool misuse, identity and privilege abuse, memory and context poisoning, and cascading failures. MITRE ATLAS provides a threat knowledge base for adversary tactics and techniques involving AI systems.
Enterprise AI Security Assessment Checklist
1. AI Inventory, Ownership and Governance
| Check | Assessment control |
|---|---|
| Maintain an inventory of internal models, third-party AI services, copilots, RAG systems, agents, APIs and experiments. | |
| Assign business, technical, data and security owners for material AI systems. | |
| Classify AI use cases by business impact, data sensitivity, external exposure, autonomy and privilege. | |
| Define approved providers, prohibited uses and permitted data classes. | |
| Require security and privacy assessment before production deployment. | |
| Discover shadow AI using proxy, DNS and CASB logs, browser-extension audits, expense records and reviews of AI features inside approved SaaS tools. | |
| Include MCP servers, agent skills and plugins, and AI coding assistants in the inventory. |
2. Data Security and Privacy
| Check | Assessment control |
|---|---|
| Map training, fine-tuning, prompt, retrieval, output, telemetry and feedback data flows. | |
| Apply data minimization and prevent unnecessary sensitive data from reaching AI services. | |
| Keep credentials, API keys, tokens and secrets out of prompts, datasets and model artifacts. | |
| For RAG, enforce authorization in the retrieval and downstream data layers, not in the LLM. | |
| Control who can modify training data, fine-tuning data, knowledge bases and embeddings. | |
| Define retention and deletion for prompts, outputs, uploads, conversation history and provider logs. |
3. Identity, Access and Agent Permissions
| Check | Assessment control |
|---|---|
| Apply least privilege to AI applications, service accounts and agents. | |
| Use dedicated machine identities rather than shared administrator credentials. | |
| Require every tool or downstream API to independently authorize an agent's requested action. | |
| Require deterministic policy checks or human approval for high-impact actions such as payments, deletion, access changes or production deployment. | |
| Remove unused plugins, functions and connectors to reduce excessive agency. | |
| Issue agents short-lived, narrowly scoped credentials. Do not pass a user's full access token through to tools. | |
| Record whether an agent is acting on behalf of a specific user or as its own service identity, and log which one was used for each action. | |
| Set limits on agent steps, spend and chained actions, with circuit breakers to stop failures from cascading across agents or systems. | |
| In multi-agent systems, authenticate and validate messages between agents rather than trusting them by default. |
4. Model, API and Supply-Chain Security
| Check | Assessment control |
|---|---|
| Record model name, version, provider, hosting location, approval status and provenance. | |
| Assess third-party providers for security, retention, training use, subprocessors and incident notification. | |
| Track AI libraries, model artifacts, containers, plugins and orchestration dependencies. | |
| Restrict access to model weights, fine-tuned models, prompts, adapters and configuration. | |
| Protect inference endpoints with authentication, authorization, rate limiting, network controls and abuse monitoring. | |
| Maintain an allowlist of approved MCP servers, agent skills and plugins. Pin versions and review provenance before installation. | |
| Review tool descriptions and schemas for embedded instructions, and re-review whenever they change. |
5. Prompt, Input and Output Security
| Check | Assessment control |
|---|---|
| Test direct prompt injection for instruction override, information disclosure and unauthorized actions. | |
| Test indirect prompt injection through documents, web content, emails, tickets, retrieved content, and images, audio or other multimodal inputs. | |
| Treat model output as untrusted before rendering, executing, querying or passing it to downstream tools. | |
| Separate instructions from data where possible, but do not treat prompt formatting as a security boundary. | |
| Treat all hidden context (system prompts, tool schemas, retrieved policy text and workflow rules) as extractable. Never place secrets or security-critical authorization logic there. | |
| Test for persistent injection that survives in agent memory or the knowledge base and affects later sessions or other users. | |
| For consequential outputs, require grounding in approved sources, visible citations and human review. Test for fabricated or unsupported answers. |
6. RAG and Vector Database Security
| Check | Assessment control |
|---|---|
| Use tenant-aware and document-level authorization filters during retrieval. | |
| Validate and scan content before it enters the knowledge base or vector index. | |
| Protect vector stores with authentication, encryption, backups, access logging and environment separation. | |
| Test whether malicious documents can dominate retrieval or redirect the model. | |
| Retain source provenance so grounded answers can be investigated. |
7. Adversarial ML and Abuse Testing
| Check | Assessment control |
|---|---|
| Test adversarial or crafted inputs that could cause misclassification or unsafe behavior. | |
| Assess data and model poisoning risks across training, fine-tuning, feedback and retrieval corpora. | |
| Consider model extraction, membership inference, memorization and training-data leakage where relevant. | |
| Test misuse of legitimate AI capabilities, including fraud support, social engineering and unsafe automation. | |
| Map red-team scenarios to relevant NIST adversarial ML categories and MITRE ATLAS techniques. |
8. Infrastructure and Runtime Security
| Check | Assessment control |
|---|---|
| Harden AI workloads with patching, isolation, secrets management and vulnerability management. | |
| Separate development, test and production environments and data stores. | |
| Encrypt sensitive datasets, vector stores, logs and model artifacts in transit and at rest. | |
| Restrict unnecessary outbound network access from AI workloads and agent runtimes. | |
| Protect AI administrative interfaces with MFA and privileged access controls. | |
| Run agent-generated or user-supplied code only in isolated sandboxes with no standing credentials and restricted network access. |
9. Logging, Detection and Continuous Monitoring
| Check | Assessment control |
|---|---|
| Log model/application identifiers, identities, tool calls, authorization decisions and security events. | |
| Detect unusual agent actions, abnormal data access, repeated failures and high-risk function calls. | |
| Use quotas, rate limits and cost alerts to reduce unbounded consumption and denial-of-wallet. | |
| Monitor material changes to models, retrieval sources, permissions, configurations, agent memory, tool descriptions and MCP server settings. | |
| Protect logs and avoid recording secrets or unnecessary sensitive prompt content. |
10. AI Incident Response and Resilience
| Check | Assessment control |
|---|---|
| Create playbooks for prompt injection, poisoning, model compromise, information disclosure and abusive agent actions. | |
| Maintain kill switches for models, tools, connectors, keys and autonomous actions. | |
| Preserve model versions, prompts, tool calls, retrieval sources, configuration and audit evidence. | |
| Test rollback to known-good models, prompts, indexes and configurations. | |
| Define escalation paths with AI and model service providers. |
11. Secure AI Development Lifecycle
| Check | Assessment control |
|---|---|
| Threat model assets, trust boundaries, data sources, dependencies, agent tools and abuse paths before build. | |
| Include SAST, dependency scanning, API testing, adversarial testing and AI red teaming according to risk. | |
| Use security acceptance criteria as a production release gate. | |
| Reassess after material changes to models, prompts, retrieval corpora, tools, permissions or providers. | |
| Retain test results, approvals, risk decisions and remediation evidence for auditability. | |
| Scan AI-generated code with the same SAST, dependency and secrets checks as human-written code, and keep secrets out of repository context shared with coding assistants. |
Technical Areas Enterprises Commonly Miss
Many enterprise assessments treat AI as a model endpoint. In practice, risk sits across the whole system: how content is ingested, how derived data inherits permissions, how retrieval is authorized, what agents can reach, and how incidents are contained. The following areas go deeper than the checklist and are frequently missed.
AI Architecture, Asset and Trust-Boundary Discovery
Document the complete AI system, including users, business owners, models, model providers, APIs, source repositories, RAG connectors, embedding models, vector databases, caches, agents, plugins, tools, downstream applications and administrative interfaces. Map data flows and identify trust, tenant and privilege boundaries before testing individual controls.
Why it matters: Security testing is incomplete when teams assess only the model endpoint. The security boundary includes every component that can influence model context, data access or downstream action.
Source Provenance, Ingestion and AI Data Integrity
For systems that ingest enterprise or external content, record source, uploader or connector, ingestion time, approval status and applicable version. Distinguish trusted internal sources from user-uploaded, external or dynamically ingested content. Restrict write privileges and define controls for stale, duplicate, revoked and deleted data.
Why it matters: Poisoned or unauthorized information can alter model behavior before a malicious user prompt is ever submitted.
Derived Data, Embeddings and Permission Inheritance
Treat chunks, embeddings, indexes, fine-tuning artifacts and other derived representations as security-relevant data. Preserve source identifiers, classification, tenant, ownership and authorization attributes when content is transformed. Verify that permission changes and deletion propagate to derived stores.
Why it matters: Document-level access control is ineffective if derived chunks, vectors or caches remain retrievable after the source is restricted or deleted.
Retrieval-Time Authorization and Tenant Isolation
Where AI retrieves enterprise data, enforce authorization before restricted content reaches the model. Apply identity, role, tenant, source and classification checks at retrieval time. Test cross-tenant queries, revoked permissions, repeated probing and attempts to bypass namespaces or metadata filters.
Why it matters: The LLM should never be the authorization decision-maker. Access decisions belong in deterministic identity, policy, retrieval and downstream application layers.
Traceability, Source Attribution and Reconstruction
For material use cases, record which model version, prompt configuration, source documents, chunks, retrieval decisions, tool calls and authorization decisions contributed to an output or action. Protect those records from unauthorized modification.
Why it matters: Security teams need to reconstruct how an unsafe answer or action occurred, identify affected users and sources, and determine whether poisoning, access-control failure or model behavior caused the event.
Caching, Retention and End-to-End Deletion
Scope caches to the appropriate user, tenant and permission context. Invalidate cached responses when relevant content or access changes. Test deletion across source data, chunks, embeddings, vector indexes, prompt stores, traces and applicable caches rather than assuming source deletion removes derived data.
Why it matters: AI systems create multiple derived copies and representations. Retention and deletion controls must cover the complete data path.
Fail-Safe Behavior and AI-Specific Containment
Define how the system behaves when authorization, integrity, retrieval, model or tool checks fail. Security-critical authorization and integrity controls should fail closed where appropriate. Incident procedures should support quarantining compromised content, disabling affected tools or connectors, invalidating indexes and caches, revoking credentials and restoring known-good configurations.
Why it matters: AI incidents may require containment at the data, model, retrieval, agent and integration layers at the same time.
Agent Tool Ecosystems and MCP Servers
Agents increasingly connect to tools through MCP servers, skills and plugin registries, many fetched from third parties at runtime. Inventory every connected server and skill, restrict installation to an approved list, pin versions, and review tool descriptions for hidden instructions. Treat a newly connected tool as a new third-party integration with its own risk review.
Why it matters: A compromised or malicious tool can change agent behavior or exfiltrate data without any change to the model or the user’s prompt.
A Practical Enterprise AI Security Assessment Flow
A repeatable assessment should move from system discovery to evidence-backed remediation rather than begin with isolated prompt testing.
| Stage | Activity |
|---|---|
| 1. Discover | Identify AI use cases, components, models, data sources, users, owners, agents, tools and data flows. |
| 2. Classify | Classify data sensitivity, business impact, source trust, autonomy, privilege and exposure. |
| 3. Assess | Evaluate governance, data protection, IAM, model supply chain, prompts, RAG, infrastructure, outputs and monitoring. |
| 4. Test | Run controlled abuse cases against model, retrieval, authorization, agent and downstream-action boundaries. |
| 5. Evidence | Capture configurations, logs, test evidence, approvals, ownership and risk decisions. |
| 6. Prioritize | Prioritize remediation using business impact, data sensitivity, exploitability, autonomy and control gaps. |
| 7. Retest | Verify remediation and reassess after material changes to models, data, prompts, connectors, tools or permissions. |
Assessment principle: protect trust, authorization, integrity, confidentiality and traceability across the complete AI lifecycle.
How to Prioritize Findings
Prioritize findings using data sensitivity, internet exposure, autonomy, privilege, production access, customer impact and regulatory context. A chatbot with no sensitive data and no tool access has a different risk profile from an autonomous agent that can query internal records and execute transactions.
Critical: Unauthorized privileged actions, exposure of regulated data, unauthenticated public inference, or compromised retrieval content that can trigger sensitive downstream actions.
High: RAG authorization bypass, unapproved sensitive-data transfer, uncontrolled model or knowledge-base changes, over-privileged AI identities, or unapproved MCP servers and tools connected to agents with production access.
Medium: Incomplete monitoring, weak provenance, inconsistent retention settings or gaps in repeatable security testing where compensating controls exist.
Low: Documentation or evidence-retention gaps with limited immediate exploitation potential.
Evidence to Collect
A mature assessment should produce evidence, not only Yes/No answers. Collect the AI inventory, architecture and data-flow diagrams, model and supplier records, threat models, IAM policies, retrieval authorization tests, prompt-injection and adversarial test results, dependency scans, model registry entries, agent permissions, SIEM detections, incident playbooks, change approvals and remediation records. Reassess after material changes to the model, provider, system prompt, retrieval corpus, connected tools, permissions, data classes or deployment architecture.
Frameworks to Use Alongside the Checklist
- NIST AI Risk Management Framework and Generative AI Profile (NIST AI 600-1): risk-management guidance for trustworthy AI and generative AI.
- NIST AI 100-2e2025: a taxonomy and terminology for adversarial machine learning attacks and mitigations.
- NIST Cyber AI Profile (NIST IR 8596, draft): a Cybersecurity Framework 2.0 community profile for securing AI systems, using AI for defense, and countering AI-enabled attacks. NIST is also developing SP 800-53 Control Overlays for Securing AI Systems (COSAiS).
- OWASP Top 10 for LLM Applications 2026: application-focused AI security risks and mitigations.
- OWASP Top 10 for Agentic Applications 2026: risks specific to AI agents that plan, use tools, hold memory and act with delegated authority.
- MITRE ATLAS: a threat knowledge base of adversary tactics and techniques involving AI systems.
- ISO/IEC 42001:2023: requirements for an organizational AI management system.
Regulatory timing note (EU AI Act): The Digital Omnibus on AI (Regulation (EU) 2026/1744) entered into force on 27 July 2026. Article 50 transparency obligations, including disclosure of AI interactions, have applied since 2 August 2026. Obligations for standalone high-risk systems under Annex III now apply from 2 December 2027, and for high-risk AI embedded in regulated products under Annex I from 2 August 2028. The delay is a planning window, not a removal of requirements.
How ServQual Can Help
ServQual’s AI Security Assessment is broader than model-level testing. It reviews AI systems and AI-enabled SaaS across AI inventory, ownership, data leakage, prompt injection, access control, vendor assurance, model governance, privacy and compliance readiness, human review, logging, monitoring and audit evidence. The assessment follows the same Discover, Classify, Assess, Test, Evidence, Prioritize and Retest approach described above, with technical testing expanded where LLM, RAG or agentic components create additional attack paths. Coverage includes cloud and SaaS AI usage, AI vendor and third-party risk, sensitive-data leakage through prompts and integrations, human review workflows for high-impact decisions, and EU AI Act readiness where applicable.
With SUSAN, teams can connect AI security findings to AI Risk Scoring, control mapping, remediation ownership, evidence tracking and continuous compliance visibility. This helps security, privacy and GRC teams connect technical AI findings with broader cybersecurity, third-party risk, regulatory readiness and continuous assurance workflows.
“AI security is not a single control or a one-time model test. It is a lifecycle discipline spanning governance, data, identity, software supply chain, model behavior, retrieval, agent permissions, infrastructure, monitoring and incident response.”
Sujal Patil
Head of Digital Marketing | ServQual
FAQ
Most frequent questions and answers
An AI security assessment is a structured review of security risks, controls and evidence across AI data, models, applications, infrastructure, identities, integrations and operations.
No. Traditional application and infrastructure testing remains important, but AI systems also require testing for Prompt Injection, poisoning, adversarial inputs, model privacy risks, RAG weaknesses and unsafe agent behavior.
AI security should be assessed before production deployment, after material changes and periodically for higher-risk systems. Continuous Monitoring should also cover security-relevant events and control changes.
No. Authorization should be enforced by the retrieval layer and downstream systems using authenticated identity and deterministic policy controls. The LLM should not make authorization decisions.
Major risks include excessive agency, over-privileged identities, unsafe tool use, weak authorization, poisoned context, compromised MCP servers and autonomous actions without appropriate control or approval.
An assessment should cover AI inventory and governance, Data Security and Privacy, IAM, model and supply-chain security, Prompt Injection, RAG Security, adversarial ML, infrastructure, logging, Incident Response and the Secure AI Development Lifecycle.
Evidence can include AI inventories, architecture diagrams, data flows, model and supplier records, threat models, IAM policies, retrieval authorization tests, Prompt Injection test results, dependency scans, agent permissions, SIEM detections, incident playbooks and remediation records.
Assess Your Enterprise AI Security Posture
Identify security gaps across AI models, applications, RAG systems, agents, data, identities, integrations and infrastructure.
ServQual helps organizations assess AI security risks, test AI-specific attack paths and connect findings to remediation, evidence and continuous assurance through SUSAN.
Disclaimer: This article provides general AI Security and Cybersecurity guidance and does not constitute legal, regulatory or certification advice. AI risks and control requirements depend on the system architecture, data sensitivity, deployment model, autonomy, connected tools, business impact and regulatory context.
