Disclaimer. ISD Insights does not recommend any particular product, vendor, model or configuration. AI technology moves quickly and the variables (your data, volumes, regulation, skills and budget) differ widely, so each business needs its own assessment. Examples on this page are illustrative.
Executive summary
Running AI on your own premises is a business decision before it is a technical one. Two questions are worth asking first:
- Is your IP in the cloud? If staff paste research notes, contract drafts or client files into hosted AI services, that material leaves your control, however good the provider's terms.
- Are you overspending on routine queries? Much day-to-day AI use is drafting, summarising and searching. Paying premium per-seat or per-Tokens: The small chunks of text a model reads and writes, often a word or part of a word. Glossary prices for all of it may not be necessary.
| What you gain | What it costs you | Example |
|---|---|---|
| Confidential IP stays on your own equipment | You buy, house and secure the equipment | A biotech research team analysing unpublished results and compound notes |
| Routine queries no longer add to per-seat or per-token bills | Upfront spend, and a capability ceiling: closed models still score higher on the hardest tasks | A legal practice running first drafts, clause comparisons and bundle summaries |
| Predictable costs and no surprise changes from a vendor | Someone has to maintain, patch and update it | Any firm that wants AI spend to be a planned budget line |
| Data residency you can demonstrate to clients and regulators | You carry the compliance evidence yourself | Firms working under client data-location clauses |
Many businesses end up with a mix: a model they control for routine and confidential work, and a hosted frontier service for the hardest tasks, with clear rules on what may go where.
Sizing rule of thumb
As a rule of thumb, a model needs roughly its file size in combined RAM and VRAM: The memory on a graphics card (GPU). Models run fastest when they fit in it. Glossary (graphics card memory) to run at usable speed. A 500 GB model needs roughly 500 GB. Download sizes are from the Open-weight model tracker (Hugging Face listings, 29 Oct 2026); the starting tier is our own grouping, explained below.
| Model | 4-bit download | Starting tier (our grouping) |
|---|---|---|
| Qwen3.8-27B | 17.6 GB | Small |
| Gemma 4 31B | 18.8 GB | Small |
| Recursive Flint | 25.1 GB | Small |
| Nemotron 3.5 Bolt 74B-A9B | 50.2 GB (native 2-bit 22.0 GB) | Medium (small at 2-bit) |
| Nemotron 3.5 Super VL | 86.9 GB | Medium |
| Reflection Beam 1.0 | 306.5 GB | Large |
| DeepSeek V4.1-Flash-1017 | 500.7 GB | Large |
Small, medium and large starting points
These tiers are our own plain-English guidance, not a specification or a quotation. They describe where to start, not where you must end up. Not sure which fits? Try the self-check: is on-prem right for you?
Small: one workstation, a few users
- Suits: a team trying AI on confidential documents, or cutting routine spend, with a handful of users.
- Runs: compact Open-weight model: A model whose weights the lab publishes, so you can download it and run it on your own hardware. Glossary models whose Quantisation: Storing a model's numbers at lower precision (for example 4-bit) so it needs less memory. Glossary downloads are roughly 18-25 GB (for example Qwen3.8-27B, Gemma 4 31B, Recursive Flint).
- Takes: a single workstation-class machine with enough graphics memory or unified memory to hold the model, set up with sensible access controls. Lowest cost and the easiest way to test value before committing further.
Medium: a shared machine for one or more teams
- Suits: regular use across one or more teams on confidential material, often with document search over your own files.
- Runs: mid-sized models in roughly the 50-90 GB range at 4-bit (for example Nemotron 3.5 Bolt or Super VL), or several smaller models at once.
- Takes: a high-memory workstation or a small server, shared over your network, with user accounts, logging and backups set up from the start.
Large: server-class, governed like any core system
- Suits: high volumes, many users, the strictest data rules, or a need for the largest open models.
- Runs: large models in the 300-500 GB range at 4-bit (for example Reflection Beam 1.0 or DeepSeek V4.1-Flash-1017).
- Takes: multi-GPU or large-memory servers, proper power, cooling and space, and ongoing support, managed as part of your wider IT estate.