Select Page

It’s Tuesday afternoon at a 150-person manufacturer in the Merrimack Valley. A contracts manager has a 40-page supply agreement to review before a 4 p.m. call. She pastes it into a free AI chatbot, asks for the payment terms and penalty clauses, and has a clean summary in 30 seconds.

Good for her. Now a question for the owner: where is that contract?

It’s on someone else’s servers, under someone else’s terms of service. Depending on the tool and its settings, it may be stored for weeks, seen by the vendor’s reviewers, or used to improve the vendor’s product. Nobody approved that. Nobody even knows it happened. Multiply that by every employee who has discovered that AI makes the job easier, and your company now has a data policy written one prompt at a time.

What private AI actually is

Private AI means running AI on infrastructure your company controls: servers in your building, or a dedicated environment in a cloud account you own. Your documents, questions and answers stay inside that boundary. No outside company sees them, stores them or sets the rules for them.

Think of the difference between renting a desk in a shared coworking space and owning your office. The coworking desk is cheap, fast and fine for a lot of work. But you don’t control who has a key, the landlord can change the lease, and you probably wouldn’t keep your trade secrets in the drawer.

This has only recently become practical. A few years ago, capable AI lived inside a handful of large technology companies. Today, models a business can download, run and own outright are good enough for most everyday work: summarizing, drafting, searching, and answering questions about company documents. The hardware to run them has come down in price as well.

Three reasons to want it

Your data stays yours

The large AI vendors offer business contracts with real protections, including commitments not to train on your data. Those protections are worth having. But they are still promises, governed by terms the vendor can revise and enforced by systems you can’t inspect. If you hold export-controlled drawings, patient information or contracts with confidentiality clauses, “we trust their terms” is a hard answer to give an auditor, a customer or a regulator. With private AI the answer is simpler: the data never left.

Your costs become predictable

Public AI services charge by usage, measured in “tokens,” which are roughly fragments of words. Every question in and every answer out is metered. That looks cheap in a pilot. It looks different once AI is connected to thousands of documents and used by hundreds of employees all day, and different again once automated “agents” start running tasks on their own, generating far more traffic than any person typing. Many companies learn what their AI costs only when the bill arrives. A private system has a different cost shape: you pay for the hardware or hosting and the people to run it, and that cost stays roughly flat whether employees ask ten questions a day or ten thousand.

Your costs become predictable


At low volume, paying by the token is cheaper. The crossover comes at roughly 220 million tokens a month; past that, a dedicated GPU wins, and the gap keeps widening. At 3.3 billion tokens, the private system costs about a fifteenth as much.

The AI actually knows your business

A public chatbot knows the internet. It doesn’t know your part numbers, your pricing history, your quality procedures or what you promised a customer in 2019. Getting that knowledge into AI means connecting it to your own files, databases and systems, which is exactly where companies get nervous about handing data to an outside service. Inside a private environment you can connect AI to everything, with the same permissions your employees already have. The shop floor sees what it should, and payroll stays with payroll. That is where the real productivity gains are.

What it costs you

Private AI isn’t free, and it isn’t right for everyone yet. You need somewhere to run it, either your own servers or dedicated cloud capacity. You need someone to set it up, keep it current and secure it, on staff or through a partner. And on the hardest reasoning tasks, the very largest public models may outperform what most companies will run privately, though the gap has narrowed and rarely matters for routine business work.

A 10-person firm using AI to polish emails probably doesn’t need any of this. A business-tier subscription from a major vendor, with sensible rules about what goes into it, is a reasonable choice. The case for private AI gets strong once any of these is true: you handle regulated or confidential data, you want AI connected to your core systems, or your usage is growing fast enough to worry your CFO. Many companies end up with both, keeping sensitive work private and using public tools for everything else.

Three questions before you sign anything

Whichever way you go, ask these before your company commits to an AI platform:

  1. Where does our data go, and who can see it?
    Get the answer in writing: what is stored, for how long, and whether it is used to train anything.
  2. What will this cost at full scale?
    Ask for a projection at the usage you expect in two years, with agents running, not the usage in today’s pilot.
  3. Can we leave?
    If you build your processes on one vendor’s platform and its prices or policies change, how hard is it to move? Owning your AI infrastructure is the strongest answer to that question.

AI is going to run through every part of your business. The question worth asking now is whose infrastructure it runs on.

About the author

Keith Gutfreund is Head of Engineering & Systems Architecture at Kirk Tech Solutions, which builds private AI and data platforms for enterprise clients in fields ranging from manufacturing and energy management to health care. Kirk Tech’s FlatClaw platform is a finalist for the NH Tech Alliance’s 20th annual Product of the Year award. Keith took his first AI course in 1982 and holds patents in internet communications and search.


Related Posts