For most small practices, running AI on your own machine costs more and performs worse than a business cloud account with the right contract. A capable box is about $4,700. Five cloud seats are about $1,500 a year. And the best model you can run on hardware you own scores in the twenties on a public intelligence index, against a frontier cloud model in the low sixties. There is a narrow set of situations where owning the box is still right, and this post is about them.
All of this assumes you have worked out that some part of your work cannot leave the building. If you have not, start with whether you need this at all: most practices that ask turn out not to need it. Nothing here is legal advice.
Open weights and runs on your computer are two different things
The open-weight models near the top of the public charts in 2026 run from 284 billion to 2.8 trillion parameters, and those need a datacenter. Only two vendors publish a claim about smaller hardware in plain enough language to hold them to: OpenAI's gpt-oss-20b card says it will "run within 16GB of memory", and Mistral says Small 3.1 "can run on a single RTX 4090 or a Mac with 32GB RAM". Everything else about what runs on a laptop is somebody's estimate. Insist on an Apache 2.0 or MIT licence: both allow commercial use outright, which removes a legal review.
The software is free and easy. The system is not.
LM Studio has been free to use at work since 8 July 2025, and it is graphical: download, click, pick a model. Ollama is free locally but command line first, which a practice manager will not enjoy without help. Installing either takes about twenty minutes. What does not: choosing the model, wiring it into a document workflow, keeping it patched, backing it up, and locking down the network. The installer is easy. The system around it is a small IT project with no end date.
2026 is a bad year to buy the hardware
NVIDIA raised its DGX Spark from $3,999 to $4,699 effective the week of 25 February 2026, giving the reason on its own developer forum: memory supply constraints. Contract DRAM prices rose 172% through 2025, memory went from 15 to 18 percent of HP's PC build materials to 35 percent in a single quarter, and Micron's chief executive said in June 2026 he expects the shortage to run through 2027. Memory is what local AI leans on hardest. Every price here carries last-verified: 2026-08-13, and we re-check this section every six months rather than yearly, because it moves that fast.
The capability gap, plainly
On a public intelligence index read on 2026-08-13, frontier cloud models sit in the high fifties to low sixties. The best open-weight model is close behind, at 2.8 trillion parameters nobody puts in an office. The tier you can put on a desk, at around 31 billion parameters, scores in the twenties. That is not a ten percent difference but a different class of tool, and the copy you run locally is usually compressed further than the one that got scored, which costs a little more accuracy again.
The other half: for a narrow, repetitive, sensitive job like searching your own files, a model in the twenties is often good enough. That is why anyone buys the box. What it will not do is invent less: privacy and accuracy are separate properties, and a model you own makes things up as readily as one in a datacenter.
Exhibit 1
The choice is three ways, not two, and the middle one is where most practices belong.
The arithmetic
Five Claude Team seats at $25 each per month is $1,500 a year, at the published price read on 2026-08-13. A DGX Spark is $4,699. Electricity barely registers: 250 watts for ten hours a day over 250 working days is 625 kilowatt hours, about $85 a year at the EIA commercial average of 13.54 cents per kilowatt hour for May 2026.
So $4,699 against $1,500 a year is roughly 3.1 years before the hardware pays for itself, and that is hardware alone. It leaves out the person who sets it up, patches it and replaces it, and the box depreciates while the cloud models improve underneath the same subscription. That is our own arithmetic from published prices, not a study; we found no comparison whose method we could check. Cost is not a good reason to go local. Control of the data is the only good reason.
Exhibit 2
The lines cross around year three, and crossing is not the same as saving.
What actually goes wrong
Local is not automatically compliant. If you hold health information the Security Rule follows the data onto your own hardware: 45 CFR 164.306(a) requires you to ensure the confidentiality, integrity and availability of what you hold. 164.312 asks for access control with a unique user identification per person, audit controls that record and examine activity, and encryption as an addressable specification, meaning a decision you assess and document rather than skip. The same rule has good news for a two-person practice: what counts as reasonable takes into account "the size, complexity, and capabilities of the covered entity".
Then there is the machine, which is a server whether or not anyone calls it one. Ollama binds to 127.0.0.1 by default, and gets exposed the moment someone rebinds it so the front desk can reach it. In January 2026 a scan reported 175,000 publicly exposed Ollama hosts across 130 countries. CVE-2026-7482, disclosed that May against a separate count of roughly 300,000 exposed servers, let unauthenticated attackers read API keys, system prompts and other users' conversation data out of Ollama's memory. It is rated critical at 9.1 and is patched in Ollama 0.17.1.
The likeliest failure is quieter. Local tools sell cloud tiers and every local interface takes an API key, so the first time someone hits a quality wall and pastes one in, the architecture you paid for is gone, with nothing signed behind what just left the building.
The middle path, where most readers should land
All five major providers now publish, in writing, that they do not train on your data, that they keep it for a defined window or not at all, and that they will sign a business associate agreement for healthcare. Those protections sit on the business or API product rather than the consumer subscription, most need an approval step, and none are on by default.
The wording is specific enough to be useful. Microsoft, on the models it sells in Foundry: "The models are stateless: no prompts or completions are stored in the model." AWS explains Bedrock's mechanism rather than promising an outcome: model providers "don't have access to Amazon Bedrock logs or to customer prompts and completions". For practices handling patient records and firms with privilege obligations, this is usually where the answer lands, at a fraction of the price of a machine.
Where local genuinely wins, and what to do first
You want at least one of four things to be true. A contract or client that forbids sending the work to an outside processor. Data sensitive enough that residual risk is unacceptable whatever paperwork exists. Someone on the payroll who already runs servers. A task narrow enough that a smaller model does it well.
If one of those fits, the first step still is not a purchase. Put a free graphical tool on a machine you already own, load the smallest model that might do the job, lock the network down, and run two weeks of real work through it. If the quality holds, price hardware then. If it does not, you found out for nothing, and we can scope this before you buy hardware.
Can I run a local LLM completely offline?
Yes. These tools run models on hardware you own, and once the weights are downloaded the machine never needs the internet. Offline is a setting you enforce and check, though: the same interface accepts an API key, and the paid tiers are cloud.
Do I need a GPU for local AI?
Not to find out whether this works for you. A business laptop with 16GB of memory runs a model in the gpt-oss-20b class, going by OpenAI's own "within 16GB of memory" claim, and it will be slow. That class also scores lower than a desk-class box: about 15 on the public intelligence index, against the twenties above. A graphics card buys speed and a bigger model.
What does self-hosted AI cost against per-seat SaaS?
$4,699 of hardware against $1,500 a year for five seats crosses over at about three years, plus roughly $85 a year of electricity locally. Neither includes the person maintaining the machine, and only one of them gets better on its own.