Enterprise AI safety in practice: hallucination, prompt injection, permission-crossing data leaks, and the system designs that limit damage even when the AI is wrong - grounded in NIST AI RMF, OWASP and PDPA.
Picture a company using AI to answer customer questions. If the AI explains the refund policy incorrectly, the damage may start as a misunderstanding. But connect that AI to the finance system and allow it to execute refunds itself, and the same mistake becomes a real transaction.
That is why evaluating enterprise AI must not stop at "how well does it answer" - it must ask "what can it access, what can it do, and when it fails, how do we contain the damage?" OWASP identifies excessive capabilities, access rights or agency as a core risk of AI-agent systems.
The warning signs are not only external. In August 2026 several leading AI CEOs publicly agreed that "we should slow AI development down." The people building the world’s most advanced AI are saying it outpaces our control. That is why AI safety is not just for researchers - it matters to every organization using AI.
1. AI safety is not just hacker-proofing
In an enterprise context, treat AI safety as reducing the probability and impact of AI-caused harm - from errors, misuse in the wrong context, and inappropriate use. AI security focuses on preventing attacks and unauthorized access. Both must work alongside data protection, fairness and accountability rather than as isolated technical concerns. This aligns with the NIST AI Risk Management Framework, which views AI risk and trustworthiness across the entire system lifecycle.
For example, a system may never be hacked yet still cause harm by summarizing a contract wrongly or making staff believe unverified information. A well-hardened system is not automatically safe for every decision. Answer reliability and over-trust must be managed separately from system security.
2. Risks that have already happened
Not a distant future - already here:
- Deepfake fraud - a Hong Kong firm lost 200 million baht after an employee was deceived by a video meeting in which every "executive" was a deepfake
- AI voice scams - criminals clone the voice of a daughter or grandchild and call elderly relatives for money, indistinguishable from the real voice
- AI disclosing what it should not - staff paste company secrets into public chatbots and the data leaks without anyone noticing
- Hallucination in real work - lawyers using AI for case research received fabricated precedents and filed them with the court
3. The key risks enterprises must see whole
Answers that look credible but have no evidence behind them
AI can fabricate information - hallucination - delivered in confident language. Grounding answers in reference documents via RAG reduces the risk, but you must still verify answers match the actual documents, not merely that links are attached. Critical work needs checks against original sources by reviewers who understand the content.
Internal data leaking across access rights
A document-search chatbot becomes a disclosure channel if it searches a pooled corpus without checking the asker’s permissions. The rule: "whatever the user cannot read, the AI acting for that user must not receive either." Enforcement belongs in the retrieval layer before content reaches the model - not "send everything and hope the AI self-censors". OWASP lists this under unauthorized access and cross-tenant data leakage.
AI deceived through the data it reads
Prompt injection embeds instructions that alter AI behavior, hidden in websites, emails or ordinary-looking documents - for example a document instructing the AI to disclose other data in the system. Crucially, RAG or fine-tuning does not eliminate this risk. Treat retrieved content as "data to read", never as "an authority that issues commands".
Impacts on people that average accuracy scores hide
A system may perform well overall while failing disproportionately for certain groups. Work involving hiring or customer assessment should be audited by relevant segment, with a path to contest results - not judged by aggregate scores alone. NIST advises evaluating fairness in context with real impact; balanced numbers alone do not guarantee fairness.
4. Design so that "even if the AI errs, the system limits the damage"
The immediately applicable principle: let the AI propose actions, but let the system enforce permissions and conditions. Writing "never disclose data" or "never exceed the credit limit" in a prompt shapes behavior but must not replace authorization checks in the destination system. OWASP recommends verifying authorization for every request - never letting the model itself decide whether an action is permitted.
In practice: a refund agent may look up orders and prepare the record, but the finance system must re-check permissions and limits; high-impact entries require approval where the approver sees the recipient, amount and evidence - not just a "confirm" button. A read-only agent should never carry edit or delete rights. Least privilege and human approval are core OWASP-recommended measures.
Separately, organizations must get precise vendor answers: is data used for training, how long is it retained, who can access it, and how is it deleted? "Not used for training" does not answer retention or access. Define which data classes may be sent, minimize excess data, and verify use, retention and deletion terms before going live.
5. AI governance: from policy documents to real operations
AI governance is not "write a policy banning AI" - it is a working framework defining who may use AI for what, with which data classes, and how it is audited, covering four areas:
- AI usage policy - what is allowed, what is not, who approves, how it is logged
- Data management - which data classes may enter AI, which are forbidden, who holds access rights
- Audit - traceability of what the AI did, when, to which data, with what outcome
- Incident response - when AI errs, who owns it, how it is fixed, who is notified
At process level, the NIST AI RMF organizes risk management into four functions - Govern, Map, Measure and Manage - which organizations can adapt as assigning risk owners, understanding context and impact, testing with evidence, then controlling and monitoring after launch. All continuous, not a one-time pre-launch review.
A concrete starting point is an AI system registry: what each system does, who owns it, which data classes it uses, which tools it connects to, and what permissions it holds - plus policy on which tools and usages are permitted. NIST emphasizes this registry because it gives the organization an overview and traceability when things go wrong.
Before production, build a test suite resembling real work. For Thai organizations that includes Thai-language questions, incomplete documents, conflicting data and no-answer cases - plus permission-crossing tests and injected instructions. Metrics should cover harmful wrong answers, data disclosure and unauthorized actions, not just average accuracy. NIST likewise advises evaluation under conditions close to real operation.
That said, a clean test suite is not proof the system will not fail outside the suite. Monitor after launch; re-evaluate when models, prompts, data sources or connected tools change; and define who can halt the system, preserve evidence and switch to a fallback process when risk exceeds thresholds.
6. Connecting to PDPA
Thailand’s PDPA obliges organizations to protect personal data. Adopting AI without controls risks:
- Sending personal data outside the organization without consent
- Unknowingly collecting data beyond what is necessary
- Missing transparency in AI-informed decisions
The ETDA Generative AI governance guideline is a Thai-language starting point designed directly for Thai organizational contexts.
7. Start where impact is bounded, then expand autonomy
For organizations starting out: pick one clearly beneficial task that does not yet execute critical transactions - drafting answers from documents the user may already read, with human review - before adding the ability to send messages, edit data or transact.
Expansion should follow evidence that the system works well within defined boundaries and the organization can handle its failures - not because one demo looked impressive.
Ultimately, AI safety is not about stopping adoption - it is about never assuming AI is always correct. Know which failures are acceptable, which must be prevented, and who decides when the system steps out of bounds.
The right AI for an enterprise is not just a capable AI - it is one with boundaries, auditability, a stop button, and clear human ownership.
Standards and further reading
| Primary source | Best used for |
|---|---|
| NIST AI Risk Management Framework and AI RMF Playbook | Building risk management, governance, evaluation and monitoring processes across the AI lifecycle |
| OWASP LLM risks and Top 10 for Agentic Applications 2026 | Designing and testing defenses against prompt injection, tool use and AI-agent permissions |
| ISO/IEC 42001:2023 | Requirements for an organization-level AI management system - a management standard, not certification of individual answer correctness |
| ETDA: Generative AI Governance Guideline for Organizations | Thai-language guide for executives covering limitations, risks and governed adoption of generative AI |
The Devsign team designs AI governance and safety - from system registries and test suites to access control and incident response plans.
