The Illusion of Private AI: Rethinking Data Privacy in Isolated Contextand Private Knowledge Base Systems

Abstract

Private AI is used in this paper as a deployment-based umbrella term for AI systems whose data and computation are constrained within a defined trust boundary. That boundary may be local or on-device, on-premises or private-network infrastructure, a private or dedicated cloud, or a managed cloud service that provides logical and contractual isolation. These models are not equivalent. This paper is concerned mainly with the final category: cloud-based isolated-context AI services that allow users to upload a bounded collection of documents and query them through one or more foundation models. NotebookLM is used as the main illustrative case, while Microsoft 365 Copilot and ChatGPT Business/Enterprise are comparison points. The central argument is straightforward: source isolation and exclusion from foundation-model training are useful privacy controls, yet neither is equivalent to local processing. The study uses a structured conceptual review of peer-reviewed security and privacy research, authoritative standards and regulator guidance, and current provider documentation. It separates privacy, cybersecurity and governance concerns across five layers: transmission and account boundary; inference; stored and derived representations; training and model improvement; and governance and human access. The paper then converts those layers into a practical zero-trust decision framework for deciding what may be uploaded, what requires an approved enterprise environment, and what should be minimised, anonymised or kept local. Its practical value is that it gives educators, managers and technical staff a way to make a data decision without pretending that a simple label such as private or no training answers every privacy question. The framework is illustrated through the design of an AI Security, Ethics and Management module at the University of Leicester. No empirical validation is claimed at this phase; the educational application is a design case that motivates a planned evaluation.