The public vs private AI question comes up in almost every AI governance conversation we have, and most organisations answer it before they have actually worked out what they are trying to protect. Public AI usually means tools like ChatGPT or Gemini, run by someone else, on infrastructure you don’t control. Private AI means the model runs inside your own environment, or at least under a contract that gives you real control over where data goes. The mistake organisations make is treating this as a simple upgrade path, public for now and private later, rather than a decision that depends on what the AI actually touches.
Public vs Private AI Is Not a Straight Trade-Off
The instinct is to assume private is always safer and public is always riskier, so the sensible plan is to start public and migrate to private once budget allows. That logic misses the point. A privately hosted model with sloppy access controls can leak data just as easily as a public tool, sometimes more easily, because everyone assumes it is already secure and stops checking. Public tools, used with the right contractual terms and staff training, can be perfectly appropriate for low-sensitivity work. The real question is not which one sounds safer. It is which one matches the sensitivity of what you are feeding into it.
What Public AI Actually Means for Your Data
When staff use a public AI tool through a personal account, or even a business account without the right settings, prompts and uploaded documents can end up stored on a third party’s servers, sometimes used to improve their models, sometimes retained for far longer than anyone in your organisation realises. Terms of service change. Regions handling that data change. None of this is malicious, it is just how these services are built to work at scale, and it is why relying on default settings is rarely a safe assumption for anything involving client data, personal information, or commercial detail.
What Private AI Actually Buys You
Running a model on your own infrastructure, or under a tightly negotiated private contract, gives you control over where data sits, who can access it, and how long it is kept. That control is real and worth having for sensitive workloads. But private AI is not automatically governed AI. Plenty of organisations set up a private model, feel reassured by the word “private,” and never build the access controls, logging, or usage policy that would make it actually safer than the public alternative. The infrastructure choice buys you the option of good governance. It does not deliver it automatically.
Matching the Tool to the Task
In practice, most organisations need both. Public tools are often fine for drafting a first version of a blog post or summarising a public document. Private tools earn their cost when the AI is handling contracts, patient records, financial data, or anything covered by a duty of confidentiality. The organisations that get this right do not pick one model and force everything through it. They classify their data first, work out what is sensitive and what isn’t, and then route work to the appropriate tool based on that classification, not on which option was easiest to roll out first.
Running a Local Model as a Middle Ground
For organisations that want the control of private AI without the cost of a full enterprise deployment, running a model locally is worth serious consideration. It keeps data inside your own network, avoids sending anything to a third-party provider, and can be set up on infrastructure you already have. Our guide on how to run a local AI model in your organisation walks through what that setup actually involves, including the hardware and staffing implications organisations tend to underestimate before they start.
Governance Comes Before the Public vs Private AI Decision
None of this works without a policy that tells staff what can go where. If people don’t know which tool to use for which task, they will default to whatever is fastest, usually a public tool on their personal device. Getting the public vs private AI split right depends far more on clear rules and enforcement than on which infrastructure you have bought. A written policy, backed by actual checks rather than good intentions, is what turns a technology choice into something that holds up under real use.
If you’re still weighing up which tools belong where, our breakdown of public AI vs private AI and what organisations get wrong is a useful next step.
