Disclaimer. ISD Insights does not recommend any particular product, vendor, model or configuration. AI technology moves quickly and the variables (your data, volumes, regulation, skills and budget) differ widely, so each business needs its own assessment. Examples on this page are illustrative.
Note. Keeping AI on your own equipment gives you control over where data lives, who can use it and what is recorded. It also makes you responsible for the security that a cloud provider would otherwise handle. Both sides matter.
Security: residency, access and audit, and third-party logs
- Residency and sovereignty: prompts, documents and outputs stay on equipment you own, in a location you choose. That makes it easier to show clients and regulators where their data is, and to meet contract clauses that restrict data location or onward transfer.
- Access and audit: you decide who can use the model, which document collections each person can search, and how long logs are kept. Give each AI tool, and each person, only the minimum access they need. This principle of least privilege appears in most information security guidance, including the international standard ISO/IEC 27001. Keep an audit trail of who asked what, and review it regularly.
- Third-party logs: a model running locally sends no prompts or data to the lab that built it or to a hosted service. Nothing is retained in someone else's logs, used for someone else's training or exposed by someone else's breach. Licence terms still apply to the model itself.
More on securing AI use: see Security in ISD Insights.
Compliance: data privacy, data protection and sector rules
Many countries and regions have their own data privacy and data protection laws, and most apply to personal data used with AI wherever the model runs. The details differ, but the same core ideas appear again and again. UN Trade and Development (UNCTAD) keeps a worldwide tracker of data protection and privacy legislation showing how widely these laws now apply.
- Data privacy: people have rights over how their personal data is used. Common examples are collecting data only for a clear, stated purpose, telling people how it will be used, and letting them see and correct what is held about them. The OECD Privacy Guidelines are a long-standing international reference for these principles.
- Data protection: the organisation holding the data must keep it safe. Common examples are appropriate security controls, limits on who can access it and how long it is kept, and reporting serious breaches. ISO/IEC 27701 is the international standard for managing privacy information alongside information security.
- Sector rules: financial services, healthcare, legal and public-sector work often add their own expectations on outsourcing, confidentiality and record-keeping. Check those, and your client contracts, alongside general data protection law.
Check which laws apply in each place you operate and wherever your data subjects are. On-premises does not remove these obligations; it can make them easier to evidence.
The honest flip side: you own the risk
- Patching and updates: the operating system, drivers, runtime software and models all need keeping up to date.
- Physical security: the equipment needs a secure, cooled location, with controlled access.
- Backups and recovery: configurations, document indexes and logs need backing up and a tested recovery plan.
- Supply chain: download models only from trusted publishers, and avoid re-packaged versions of unknown origin.
- Monitoring: someone needs to notice when something goes wrong.
This is where ISD's support comes in: we help businesses plan, set up and look after on-premises systems, so the control you gain does not become a maintenance burden.
Respected frameworks to build on
The most widely respected guidance does not tell organisations to avoid AI. It sets out how to use it safely. It falls into a few broad types, and each has a recognised place to browse a wider list. These are starting points, not a complete catalogue:
- AI risk management frameworks: voluntary frameworks for identifying, measuring and managing AI risks across an organisation, such as the NIST AI Risk Management Framework. For a wider list, the OECD keeps a Catalogue of Tools and Metrics for Trustworthy AI.
- AI application security: the common weaknesses in applications built on language models, such as prompt injection and sensitive information disclosure, and how to defend against them. The OWASP Top 10 for LLM Applications is the best-known list; the OWASP GenAI Security Project and MITRE ATLAS, a knowledge base of real-world attack techniques against AI systems, go further.
- International management standards: certifiable standards for running AI responsibly, such as ISO/IEC 42001 for an AI management system. Related AI standards are published by the joint ISO and IEC committee on artificial intelligence, ISO/IEC JTC 1/SC 42, in the ISO standards catalogue.
- Government and regulator guidance: many national cyber security agencies and data protection regulators publish practical guidance on secure AI development and use, often jointly with international partners. The OECD.AI Policy Observatory tracks AI policies and guidance across many countries, so you can find what applies where you operate.
A middle path: AI never touches live systems
For businesses that want the benefits of AI analysis without connecting AI to the systems they run on, there is a safer middle path:
- The business exports selected data, on purpose, into an Isolated analysis environment (AI data clean room): A separate, controlled environment where selected data is analysed, away from live business systems. Glossary (sometimes called an AI data clean room or AI warehouse).
- The data is analysed there, away from live systems.
- Results come back through controlled, read-only views.
AI tools in the environment never connect to live systems, and nothing flows back automatically. Read more on our isolated analysis environment page.