Private AI and AI agents – data security in the enterprise

First category: sector regulations. DORA (financial institutions), NIS2 (essential service operators), MDR (medical devices), KNF/PFSA regulations (banks, insurers, asset managers) – each imposes requirements on data localisation and control. For some operational scenarios, Microsoft 365 with private endpoints and an Azure Poland region is sufficient. For others, full infrastructure control is required.

Second category: special-category data under GDPR art. 9 – health, biometric, genetic, sexual orientation, religious beliefs. For scenarios where an AI agent processes such data (e.g. a clinical assistant, an HR agent handling sick leave), private AI is often a mandatory standard.

Third category: highest organisational data classes – M&A, pre-publication results, intelligence data, military and national-security data, intellectual property (code, know-how). For these classes no SLA with a cloud provider changes the risk calculation.

Fourth category: residency and sovereignty. Organisations with a policy of 'data does not leave Poland' / 'data does not leave the EU'. Cloud providers offer EU and Poland regions, but for some customers (especially the public sector) this is insufficient – they require full physical control over the infrastructure.

Fifth category: edge and offline. AI agents in environments with limited connectivity (ships, drilling platforms, factories, uniformed services). There, private AI is not a choice but the only technical possibility.

Over the last two years the answer to the question boards ask most often – 'will AI in our own data centre give us the same business value as ChatGPT or Copilot in the cloud?' – has shifted. For most business uses today the answer is yes. Families of AI models have emerged that can run inside the organisation and match the leading cloud models for typical analytical, document and team-support tasks.

In practice, picking a specific model is now a technical detail, not a strategic one. The solution architect matches the model to the scenario – one for an assistant in medical documentation, another for contract analysis, another for employee questions. From the board's perspective the relevant fact is that the ecosystem of 'run-it-yourself' models is mature and still developing – being locked into a single cloud vendor has stopped being a necessity.

What this means for the business: an organisation that a year ago had to choose between 'use the cloud and accept the risk' or 'give up on AI' now has a third option – its own AI platform running on the company's infrastructure with full control over where the data sits.

Practical consequence: have the conversation with the partner around business scenarios ('assistant for procurement', 'project documentation analysis', 'handling employee enquiries'), not around specific model names. An experienced partner will pick the model that fits – this is not a decision the board has to make itself.

From a board and PMO perspective it is worth understanding that private AI is not one product but a set of building blocks that must work together. It is closer to your in-house data centre or ERP than to a single tool – an ecosystem of infrastructure, software and processes.

The first block is infrastructure – servers with the right compute power (specialised machines for running AI models), networking and storage. It can sit in your own data centre, in a private one or in a sealed-off slice of the cloud. The second block is the platform that runs and maintains the AI models – the operating system for AI. The third block is the assistant's decision layer – the part that turns an AI model into a 'teammate': understands the question, picks the right knowledge sources, plans the actions and executes them in the company's systems.

The fourth block is the organisation's knowledge base – organised documents (policies, procedures, historical data) that the assistant uses. This is often the decisive quality element – good AI on poorly organised knowledge produces weak answers. The fifth block is integration with company systems (ERP, CRM, document workflows, Active Directory). The sixth – the oversight layer: monitoring, audit, access and security policies.

Main difference vs cloud solutions such as Microsoft 365 Copilot: there most blocks come bundled and only need to be 'switched on'. In private AI the organisation builds (or commissions) the whole stack. That raises the initial cost but gives full control over every element.

Microsoft 365 with Copilot Studio agents: deployment time 4–8 weeks for the first agent. Cost: ~PLN 30–80/user/month (Copilot licence) + project cost. Plus: ready stack, inherited governance, out-of-the-box Microsoft 365 integrations. Minus: data leaves to Azure OpenAI (even on private endpoints and the Poland region – contract with Microsoft).

Private AI with agents: deployment time 4–6 months for the first agent (more infrastructure and custom development). Cost: PLN 0.6–2m CAPEX + PLN 200–400k annual OPEX (depending on scale). Plus: full control, data never leaves the organisation, ability to fine-tune on organisational data. Minus: much higher initial investment, no ready connectors, requires an internal team or a partner.

Hybrid architecture: Microsoft 365 for 'mainstream' scenarios (HR, IT, finance in the operational dimension), private AI for sensitive-data scenarios (M&A, medical, intelligence). This is today the most common path for large organisations – combining M365's deployment speed with private AI's data security.

A practical observation: organisations rarely choose 'private AI only'. Much more often they pick a hybrid mix where 70–80% of scenarios go to M365 and 20–30% (the most sensitive) to private AI. This captures benefits of both. We design this decision under our advisory and strategy practice.

Compliance and the board need a clean answer from the private-AI vendor to six questions. The same questions you ask a data-warehouse provider, an ERP vendor or an outside law firm. If the answers are not there, the project should not move forward – no matter how good the demo looks.

First question: do our data leave our environment at all? In a well-designed private AI the answer is 'no' – the platform runs in an isolated internal network, with no internet contact beyond a strictly controlled software-update channel.

Second question: is the data encrypted at every stage (on disk, in transit between components, in the knowledge base)? Market standard is full encryption with keys managed in a way that not even the platform team can access. Third question: does every user see only the information they are entitled to in the company? The AI assistant must operate 'on behalf of' a specific employee and inherit their permissions from the company's systems (Active Directory, access policies) – never see more than the employee.

Fourth question: does every assistant action leave an audit trail? The compliance officer should be able at any time to check who used the assistant, when, in what context, with what questions and what actions. Logs retained for at least 7 years (DORA, SOX), in an immutable format.

Fifth question: are there safeguards against accidental disclosure of sensitive data (e.g. the assistant should never answer with an employee's national ID number, even if asked directly)? Sixth: how does the platform defend against attempts to 'trick' the assistant with malicious questions (so-called prompt injection)? These questions should be part of every AI procurement process.

Deploying AI in regulated organisations meets several layers of regulation that are easy to overlook in planning and can stop a project at the finish line. It is worth ordering them from a business perspective – what obligation each imposes and what the consequence of non-compliance is.

First layer: general AI rules. The EU AI Act classifies AI systems by risk level and imposes concrete obligations – from documentation through testing to a user's right to an explanation. GDPR covers every process touching personal data. Breaches are heavy financially and reputationally – we are talking about fines counted in a percentage of annual revenue.

Second layer: sector rules. DORA for banks and insurers requires rigorous control of the digital IT supply chain, including AI vendors. NIS2 applies to operators of essential services (energy, transport, healthcare, public sector). MDR for medical device makers requires validation of AI systems used in clinical processes. Each of these can determine whether cloud AI is even permissible.

Third layer: standards and certifications expected by business partners and customers. ISO 27001 (information security), ISO 27701 (privacy), ISO 42001 (AI management), ISO 13485 (medtech). Lack of these often disqualifies vendors from enterprise tenders.

Fourth layer: the national regulator. KNF for financial services, UODO for personal data, sector-specific (aviation, energy, defence). Each has its own interpretive line. Practical consequence for the board: compliance and security are not a 'post-deployment' layer – they are foundations that have to be designed in the first week. Fixing them later costs many times more and sometimes is no longer possible.

The most common board concern before a private-AI rollout is: 'fine, but will our assistant know our company, our contracts, our products, our industry – or will it answer generic platitudes from the internet?'. A fair question, and the answer comes down to two elements.

The first element is well-organised organisational knowledge. An AI assistant does not 'learn' on its own – it uses documents, policies, procedures and data the organisation provides. If SharePoint is tidy, policies are current and procedures are alive, the assistant has something to work with. If knowledge is scattered, stale and inconsistent, the assistant will reflect those weaknesses. Almost every AI project surfaces debts in knowledge management that need to be paid down along the way.

The second element is optional 'fine-tuning' of the model to the organisation's specifics. We apply it only when the organisation uses highly specialised language (medical, legal, industry-specific), its own communication style or specific decision patterns that an off-the-shelf model does not know. For most typical uses (HR assistant, IT assistant, operational finance assistant) such tuning is not required – a well-set knowledge layer is enough.

Practical recommendation: in year one focus on tidying up knowledge. This usually delivers 80% of the value at 30% of the cost. Tuning the model is worth considering only after a year in production – when the organisation knows where answer quality is actually weak. Starting with 'we will build our own AI model from scratch' is a classic trap for organisations that mistake AI for a research project.

Boards often ask: 'if we go with private AI, will we pay more than for the cloud?'. The answer depends on organisational size and the horizon we measure. For a mid-sized organisation with 500 users and a few AI assistants, the picture usually looks as follows.

Cloud variant (Microsoft 365 with Copilot): mostly subscription cost – per-user licences plus a one-off deployment project. Over 3 years the typical total is ~PLN 5m, most of it a recurring fee. Main upside: fast start (4–8 weeks to the first assistant), low upfront cost, no infrastructure to maintain.

Private AI variant: higher upfront cost (hardware, platform, project) and lower recurring cost. Over 3 years the typical total is PLN 2.5–4m plus integration with corporate systems. Main financial risk: under-estimating maintenance (updates, team, monitoring). Longer time to first assistant (4–6 months vs 4–8 weeks).

On pure maths private AI starts winning on cost at larger scale and longer horizons (4+ years). But in practice TCO is rarely the main driver. The main arguments are compliance, control of data, vendor independence. If those are important to the board and to compliance, the TCO comparison is a bonus, not the reason for the choice.

A private-AI rollout is not an IT project but a transformation programme touching strategy, risk, infrastructure and the way people work. From the board's perspective it is worth thinking of it in four phases, of which the first and the last matter most.

Phase 1 (6–8 weeks) is discovery and strategic decisions. Identify the processes for which private AI really makes sense (usually 2–4 in the organisation). Decide whether we go 'fully in-house' or 'partly in-house, partly cloud'. Align with compliance, the board and IT. Outcome: a business case with concrete ROI and a risk map – not technology, but a board decision.

Phase 2 (3–4 months) is preparing the infrastructure and platform. IT works mostly with the implementation partner here – servers, networks, software, baseline security. From a business view this is the 'patience phase' – visible results come in phase 3.

Phase 3 (3–4 months) is launching the first AI assistant for a specific business process. Usually the most risk-sensitive one (compliance assistant, medical-data assistant, M&A assistant). We measure the real business effect in the first production quarter.

Phase 4 (4–6 months) is extending to further processes. Each next assistant deploys faster because the platform already exists. The whole programme – from board decision to a mature platform with 3–5 assistants – typically takes 16–22 months. Investment in a mid-sized organisation: PLN 2–5m, payback usually visible in months 24–36, long-term ROI 200–500% annually.

First mistake, most common: choosing private AI for emotional rather than business reasons. The board decides 'we want our own AI because no one should see our data', and the analysis shows that in 80% of cases compliance does not actually require it. The result: a multiple-times higher cost without proportional value. Better: an honest count of how many processes actually need private AI – usually fewer than first thought.

Second mistake: under-estimating maintenance cost. Organisations focus on hardware and project costs and forget the recurring ones – model updates every few months, monitoring, security patching, team rotation. The real 3-year total is typically 30–50% above the initial figure. The board should demand a concrete annual OPEX table from the vendor.

Third mistake: wanting to build everything in-house. Organisations that hear 'this is just open-source software' commission an internal AI team. After a year: no working product, high team cost, no operational maturity. Better: use mature market components and an implementation partner, and build management competence in-house, not technical competence.

Fourth mistake: assuming Polish-language quality is a given. Off-the-shelf models are optimised mainly for English; without deliberate work on the language layer the assistant can answer imprecisely on Polish industry terms. A trap that is especially costly in language-dense industries (medicine, law, insurance).

Fifth mistake: assuming 'private AI defends itself'. Sitting in our own data centre does not eliminate risk – without audit, access policies, monitoring and procedures, private AI is just as risky as cloud. Governance and compliance must be there from day one, not added after the rollout.

The decision on private AI is rarely black and white. Experienced strategy advisors propose a five-step process to reach a defensible choice – one you can defend before the supervisory board and compliance.

Question 1: do our data really require private AI? Inventory the processes and data categories. Usually private AI is required only for 2–4 processes (e.g. M&A, medical data, patent documentation) and the rest run perfectly well on Microsoft 365.

Question 2: is there a risk the supervisory board will not accept, even if formally compliance allows the cloud? Sometimes the answer is 'yes' – psychological and reputational risk is greater for the board than formal contractual safeguards with a vendor.

Question 3: does the organisation have the competence (in-house or via a proven partner) to run private AI in production for 5–7 years? Without it the deployment ends as a costly unused asset.

Question 4: is private AI needed for the whole organisation, or only for a slice of processes? Practically always the answer is 'a slice' – which means the right answer is a hybrid architecture, not pure private AI.

Question 5: does the 3–5-year total cost fit the board's investment appetite? Demand a concrete calculation from the vendor, not a generic offer. The output of this process is not 'yes/no', but a clear map: which processes → which model. A decision you can defend before any oversight body.

The private-AI market is mature enough today that organisations with the most sensitive data – banks, insurers, healthcare, public sector, IP-intensive companies – have a real alternative to the cloud. This is no longer a 'cloud or nothing' choice but a deliberate one: which processes can use the cloud, and which must stay inside our infrastructure.

The key conclusion for the board: 'private AI vs cloud' is not a technical decision. It is a board-level decision on risk, compliance and control. Most organisations choose a hybrid model – most processes (HR, IT, operational finance) on Microsoft 365, and the most sensitive (M&A, medical, pre-publication financial) on private AI. Only organisations with a dominant share of sensitive data go fully private.

The investment is significant: PLN 2–5m in a mid-sized organisation and 16–22 months to a mature platform. But the payoff, if the decision was well-grounded, is twofold – measurable operational savings (typically 200–500% annually after year three) and a step-change reduction in regulatory and reputational risk. At Algorcomp we support clients through the full cycle: from the board-level decision, through architecture design, to deployment and ongoing operation.

The other articles in the series cover the topic from four complementary angles: AI agents in business – how organisations automate processes, how to implement AI agents in an organisation, AI agents in finance and AI agents in Microsoft Teams.