Table of Contents
What Is AI Security?
AI security is the discipline of protecting artificial intelligence systems – including large language models, machine learning pipelines, training datasets, model APIs, and AI agents – from attack, manipulation, misuse, and failure. It encompasses the security of the AI system itself (the models, data, and infrastructure), the governance of how AI is used within an organization, and compliance with AI-specific regulatory obligations such as the EU AI Act. Unlike traditional cybersecurity, which focuses on protecting networks, endpoints, and applications, AI security must also protect against threats that are unique to how AI systems learn, infer, and act.
Why AI Requires a Different Security Model
Traditional security tools were designed for a world where software behaves deterministically. A firewall either permits or blocks a connection. An application either has a SQL injection vulnerability or it does not. The threat model is relatively stable, and the controls are well understood.
AI systems break this model in three fundamental ways.
First, AI systems learn from data – which means the data itself becomes an attack surface. An adversary who can influence what an AI system is trained on can shape what it produces, long before any deployment security control is applied. This is data poisoning, and it has no equivalent in traditional application security.
Second, AI systems produce probabilistic outputs that can be manipulated through carefully crafted inputs. A prompt injection attack against an LLM does not exploit a code vulnerability – it exploits the model’s instruction-following behavior. Traditional web application firewalls and input validation controls offer little protection.
Third, AI systems are increasingly autonomous. Agentic AI frameworks – where an LLM can call external APIs, browse the web, write and execute code, or take actions in external systems – create an entirely new class of risk. An agent that can be manipulated to exfiltrate data, escalate privileges, or take destructive actions on behalf of an attacker represents a threat vector that did not exist two years ago.
Gartner projects that AI security incidents will double by 2027. Organizations that apply only traditional security frameworks to AI deployments will be systematically underprepared.
The Six AI Attack Surfaces Every CTO Must Understand
1. Prompt Injection
Prompt injection is the most widely exploited AI vulnerability in production environments today. It occurs when an attacker crafts inputs – either directly via a user interface or indirectly through content the model retrieves from external sources (websites, documents, emails) – that override or manipulate the model’s intended instructions. A direct prompt injection might instruct a customer-facing chatbot to ignore its safety guidelines. An indirect injection might embed malicious instructions in a document that a RAG-enabled assistant retrieves and acts on. The OWASP LLM Top 10 (2025) ranks prompt injection as the number one risk for LLM-based applications.
2. Data Poisoning
Training data is the foundation of every AI system. If an adversary can introduce malicious, biased, or subtly manipulated data into the training pipeline – through a compromised data source, an insider, or a supply chain attack – the resulting model’s behavior can be systematically corrupted. Data poisoning is particularly dangerous because the effects may be dormant until a specific trigger condition is met, making detection extremely difficult with standard monitoring.
3. Model Inversion and Extraction
Model inversion attacks attempt to reconstruct training data from a deployed model by querying it systematically. In cases where models are trained on sensitive data – patient records, financial transactions, personal communications – a successful inversion attack can expose that data without ever breaching the database it came from. Model extraction attacks go further: by querying a model’s API extensively, an adversary can reconstruct a functional copy of a proprietary model, stealing the intellectual property that represents months or years of training investment.
4. Adversarial Inputs
Adversarial inputs are carefully engineered perturbations to model inputs, often imperceptible to humans, that cause an AI system to produce systematically wrong outputs. In image classification, a few strategically placed pixels can cause a model to misclassify a stop sign as a speed limit sign. In text-based systems, subtle reformulations of inputs can cause models to produce harmful content they would otherwise refuse. This attack surface is particularly critical for AI systems used in security-sensitive decisions: credit scoring, fraud detection, identity verification, medical diagnosis.
5. Supply Chain Attacks
Most organizations do not build their AI systems from scratch. They use pre-trained foundation models from major providers (OpenAI, Anthropic, Google, Meta), fine-tune open-source models (from Hugging Face or similar repositories), or integrate third-party AI APIs. Each of these dependencies introduces supply chain risk. A compromised model weight file, a malicious fine-tuning dataset, or a vulnerability in an AI orchestration library (such as LangChain or LlamaIndex) can introduce backdoors or vulnerabilities into systems that have never been directly attacked. The AI supply chain is largely unaudited in most enterprise environments.
6. Agentic AI Risk
LLM-powered agents represent the newest and potentially most consequential attack surface. An agent that can browse the web, send emails, make API calls, write and execute code, and interact with internal systems on behalf of a user is, from a security perspective, a privileged executor with broad access and limited oversight. If such an agent can be manipulated through prompt injection – including indirect injection through content it retrieves, the consequences extend far beyond a misbehaving chatbot. They include unauthorized data access, unintended external communications, and actions in connected systems that may be difficult or impossible to reverse.
AI Security vs. Traditional Cybersecurity: Key Differences
| Dimension | Traditional Cybersecurity | AI Security |
|---|---|---|
| Primary attack surface | Networks, endpoints, applications, identity | Models, training data, prompts, inference APIs, agents |
| Nature of vulnerabilities | Deterministic code flaws (CVEs, misconfigurations) | Probabilistic behavior, semantic manipulation, data integrity |
| Detection approach | Signature-based, log analysis, anomaly detection | Output monitoring, behavioral baselining, prompt logging |
| Exploit mechanism | Exploit a vulnerability in code or configuration | Manipulate model behavior through inputs or training |
| Governance tools | SIEM, SOAR, vulnerability scanners, firewalls | AI asset registries, model cards, AI-specific observability |
| Regulatory framework | NIS2, ISO 27001, DORA, GDPR | EU AI Act, ISO 42001, NIST AI RMF (in addition to the above) |
| Established standards | OWASP Top 10, NIST CSF, CIS Controls | OWASP LLM Top 10, MITRE ATLAS, NIST AI RMF |
| Key test methodology | Penetration testing, vulnerability scanning | Red-teaming, adversarial probing, model evaluation |
| Speed of change | Mature, relatively stable threat model | Rapidly evolving — new attack classes emerging quarterly |
The OWASP LLM Top 10 (2025): What It Means for Your Organization
The OWASP LLM Top 10 is the closest thing the industry currently has to a standardized vulnerability classification for LLM-based applications. Updated in 2025, it identifies the ten most critical security risks for organizations deploying large language models:
- LLM01 – Prompt Injection: Attacker-controlled inputs override model instructions
- LLM02 – Sensitive Information Disclosure: Models leak training data or confidential context via outputs
- LLM03 – Supply Chain Vulnerabilities: Compromised model components, datasets, or AI libraries
- LLM04 – Data and Model Poisoning: Manipulation of training data corrupts model behavior
- LLM05 – Improper Output Handling: Model outputs passed to downstream systems without sanitization
- LLM06 – Excessive Agency: Agents granted permissions beyond what use cases require
- LLM07 – System Prompt Leakage: System prompts containing business logic or secrets exposed to users
- LLM08 – Vector and Embedding Weaknesses: Vulnerabilities in RAG pipeline data retrieval
- LLM09 – Misinformation: Models generating authoritative-sounding false information used in decisions
- LLM10 – Unbounded Consumption: Resource exhaustion attacks via excessive API usage
For a CTO, the OWASP LLM Top 10 is the most direct checklist for assessing whether your AI security controls are adequate. If your organization has not mapped your AI deployments against these categories, you have no documented baseline.
Shadow AI: The Unmanaged Risk Inside Every Enterprise
Shadow AI is the use of unsanctioned AI tools by employees, using personal or unlicensed accounts, to process work-related data. The scale of the problem in 2026 is significant. According to LayerX’s 2025 Enterprise AI Security Report, one in three organizations reviewed had sensitive corporate data exposed through employee use of public AI services.
The behavior is not malicious. An employee who uploads a client contract to ChatGPT to improve the language, pastes financial projections into Gemini to build a model, or uses an AI transcription service for a sensitive internal call is trying to do their job more efficiently. The data risk, however, is real and immediate: once data leaves the organization’s infrastructure and is processed by a third-party AI provider, the organization loses control of it. The data may be used for model training, stored in jurisdictions with different legal protections, or exposed in a breach of the AI provider.
For organizations subject to GDPR, the Israeli Privacy Protection Law, or NIS2, shadow AI creates direct regulatory exposure. Uploading personal data to an unvetted AI service may constitute an unauthorized transfer of personal data to a third party, triggering notification obligations.
The controls organizations need are: an AI use policy (what tools employees are permitted to use, for what purposes, and with what categories of data), visibility tools that detect AI traffic at the network or endpoint level, and an approved AI toolset that provides the capabilities employees need within a governed environment. Microsoft 365 Copilot, when properly configured with appropriate SharePoint permissions and data loss prevention policies, is the most practical enterprise answer for organizations already in the Microsoft ecosystem — but deployment without security configuration often makes the problem worse, not better.
EU AI Act Risk Tiers: What Applies to Your Business
The EU AI Act came into force on 1 August 2024 and applies extraterritorially – any organization whose AI systems are used in the EU is within scope, regardless of where the organization is headquartered. For Israeli companies with EU customers, EU distribution partners, or EU subsidiaries, this is a live compliance obligation.
The Act classifies AI systems into four risk tiers:
Prohibited (Unacceptable Risk) – Banned outright from February 2025. These include social scoring systems by public authorities, real-time remote biometric identification in public spaces (with narrow law enforcement exceptions), AI that exploits psychological vulnerabilities to manipulate behavior, and AI used to predict criminal activity based on profiling. Any organization operating systems in these categories must have ceased doing so.
High Risk – Permitted but subject to strict pre-deployment obligations effective August 2026. High-risk categories include AI used in critical infrastructure, employment decisions (hiring, performance evaluation, termination), educational assessment, access to essential financial services (credit scoring, insurance underwriting), law enforcement, and migration decision-making. Organizations with high-risk AI systems must implement risk management systems, maintain technical documentation, ensure human oversight mechanisms, implement logging for audit purposes, and register systems in the EU database before deployment.
Limited Risk – Primarily chatbots and AI-generated content. These systems must disclose their AI nature to users. Compliance is relatively low-burden but mandatory.
Minimal/No Risk – The majority of enterprise AI applications (spam filters, recommendation engines, productivity tools) fall here. No specific obligations beyond general Act provisions.
The financial consequences of non-compliance are severe: fines of up to €35 million or 7% of global annual turnover for prohibited practices, and €15 million or 3% of global annual turnover for non-compliance with high-risk requirements. For organizations approaching an August 2026 enforcement deadline for high-risk systems, the time to conduct a compliance assessment is now.
The NIST AI Risk Management Framework: Govern, Map, Measure, Manage
The NIST AI Risk Management Framework (AI RMF), published in January 2023, provides the most comprehensive voluntary governance structure for AI risk management currently available. It is organized around four core functions:
Govern – Establish the organizational policies, accountability structures, and culture needed to manage AI risk. This includes defining risk tolerance for AI systems, establishing ownership and accountability for AI risk, and ensuring that AI governance is integrated into enterprise risk management rather than treated as a separate technical function.
Map – Identify and categorize AI risks in context. This means understanding the specific deployment context of each AI system, the stakeholders who interact with it, the harms that could result if it fails or is manipulated, and the dependencies (data sources, third-party models, downstream systems) that create additional risk.
Measure – Evaluate AI risks using appropriate tools and methods. This includes testing for bias, robustness, and adversarial vulnerabilities; monitoring AI system behavior in production; and establishing metrics that allow risk levels to be tracked over time.
Manage – Prioritize and implement risk responses. This includes remediating identified vulnerabilities, implementing governance controls, managing third-party AI provider risk, and establishing incident response procedures specific to AI failures and attacks.
For organizations seeking a structured starting point for AI governance, the NIST AI RMF is the most credible and comprehensive framework available. It can be adopted alongside existing security frameworks (NIST CSF, ISO 27001) without conflict.
How IPV Security Approaches AI Architecture Security
IPV Security’s AI Architecture Security Review is a structured, three-to-five-week assessment that produces the first complete, security-focused map of an organization’s AI estate. It is designed specifically for organizations that have deployed AI systems faster than security governance could follow, which, in 2026, describes nearly every enterprise.
The engagement covers five domains: AI asset discovery and inventory (every model, pipeline, agent, and integration in the environment); attack surface analysis mapped against the OWASP LLM Top 10 and MITRE ATLAS threat matrix; access control and identity review specific to AI system permissions; data classification and privacy exposure assessment; and threat modelling using the STRIDE methodology adapted for AI systems.
The output is a Security Architecture Review Report with a prioritized remediation roadmap structured across 30, 60, and 90-day windows, plus an executive readout deck designed for board and leadership presentation. Findings are uploaded directly to CISOteria for clients using the platform, enabling continuous tracking rather than a point-in-time snapshot.
For organizations with EU-facing AI systems, the review can be combined with an EU AI Act Compliance Assessment (Service 04) to address both security and regulatory obligations in a single engagement at a combined discount.
Engagements are led by IPV Security’s AI Security practice, drawing on the OWASP LLM Top 10, MITRE ATLAS, NIST AI RMF, and IPV’s proprietary AI Security Operating Framework (AI-SOF) as the primary assessment references.
Explore IPV Security’s AI Security services →
Read our compliance guide for ISO 27001, NIS2, and DORA →
Related Articles
Frequently Asked Questions
What is AI security and why does it matter now?
AI security is the practice of protecting artificial intelligence systems – models, training data, inference APIs, and AI agents – from attack, manipulation, and misuse. It matters now because the deployment of AI in enterprise environments has created a new class of attack surface that traditional security controls do not address. Prompt injection attacks against LLM-based applications, data poisoning in training pipelines, and agentic AI systems with broad permissions represent genuinely new threat vectors. Gartner projects AI security incidents will double by 2027. Organizations that have deployed AI without a corresponding security assessment have an undocumented, unmanaged attack surface. The starting point for any organization is an AI Architecture Security Review – a structured inventory and risk assessment of every AI component in the environment.
How do you secure a large language model (LLM)?
Securing an LLM requires controls at four layers. At the input layer: prompt filtering, input validation, and rate limiting to detect and block injection attempts. At the model layer: access controls on model endpoints, output filtering to prevent sensitive data leakage, and monitoring of model inputs and outputs. At the infrastructure layer: proper IAM for model APIs, short-lived credentials rather than long-lived API keys, and network segmentation isolating model infrastructure. At the governance layer: an AI use policy, a model inventory and risk register, human-in-the-loop checkpoints for high-stakes decisions, and an incident response procedure specific to AI events. No single control is sufficient. Effective LLM security requires a layered defense aligned to the OWASP LLM Top 10 threat taxonomy.
What is the EU AI Act and does it apply to Israeli companies?
The EU AI Act is the world’s first comprehensive legal framework for artificial intelligence. It applies to any organization whose AI systems are used in the EU – regardless of where the organization is headquartered. Israeli companies with EU customers, EU distribution partners, EU subsidiaries, or EU investors whose AI systems interact with EU-based users are within scope. The Act classifies AI systems by risk tier. Prohibited systems have been banned since February 2025. High-risk AI system obligations – including technical documentation, human oversight, and pre-deployment registration – apply from August 2026. Fines for prohibited practices reach €35 million or 7% of global annual turnover. Israeli companies with EU exposure that have not completed a compliance assessment are operating with unquantified regulatory risk.
What is shadow AI and how do you manage it?
Shadow AI refers to the use of AI tools by employees without organizational authorization or visibility – typically consumer AI services such as ChatGPT, Gemini, or Grammarly that employees access using personal accounts to process work-related data. The data risk is concrete: sensitive company data, client information, or regulated personal data leaves the organization’s infrastructure and is processed by a third party under terms the organization has not reviewed. Managing shadow AI requires three things: an AI use policy that defines what tools are permitted for what purposes; network-level or endpoint-level visibility tools that can detect AI service traffic; and an approved AI toolset that gives employees capable alternatives within a governed environment. Discovery of what shadow AI tools employees are actually using is the essential first step and most organizations are surprised by the findings.
How long does an AI architecture security review take?
IPV Security’s AI Architecture Security Review takes three to five weeks, depending on the complexity of the organization’s AI environment. A smaller environment – one to three AI systems in a single-product company – runs three weeks: one week for AI asset discovery interviews with key stakeholders (engineering, data science, DevOps, security), one week for architecture analysis and access control review, and one week for report writing and executive readout preparation. A complex environment with eight or more AI systems across multiple product teams or geographies runs five weeks, with additional time for extended discovery and more comprehensive threat modelling. The engagement includes a live executive readout at completion, with the report and findings loaded into CISOteria for organizations using the platform.
What is the OWASP LLM Top 10?
The OWASP LLM Top 10 is a published list of the ten most critical security risks for applications built on large language models, maintained by the Open Worldwide Application Security Project and updated in 2025. It is the AI equivalent of the OWASP Web Application Top 10 – a standardized taxonomy used by security practitioners to classify and prioritize LLM vulnerabilities. The top risks include prompt injection (attackers manipulating model behavior through crafted inputs), sensitive information disclosure (models leaking training data or confidential context), supply chain vulnerabilities (compromised models or AI libraries), data poisoning (manipulation of training data to corrupt model outputs), and excessive agency (agents with permissions beyond what their function requires). Organizations with LLM-based applications should use the OWASP LLM Top 10 as the primary checklist for assessing their AI security posture.