Disclaimer. ISD Insights does not recommend any particular product, vendor, model or configuration. AI technology moves quickly and the variables (your data, volumes, regulation, skills and budget) differ widely, so each business needs its own assessment. Examples on this page are illustrative.
Power, cooling and space considerations:
Data-centre GPUs draw 700 watts or more each, with the newest above 1,000; workstation cards such as the RTX 5090 draw around 450 to 600, so plan for PSU headroom, circuit limits and UPS sizing. Cooling scales with that heat, so airflow and ambient temperature decide whether air or liquid cooling works. Space here means rack footprint and weight, not storage — VRAM: The memory on a graphics card (GPU). Models run fastest when they fit in it. Glossary affects how many cards you need, while disk only holds the model weights. Contact ISD at [email protected] for further information.
Buy, lease or rent considerations:
Buying ties up a lot of cash upfront but you own the asset and can depreciate it. Leasing spreads the cost monthly and often includes maintenance, but you pay a premium over the life of the contract. Renting is the most flexible for short-term or burst needs, yet the priciest per month if you hold it long. GPU generations move fast, so a three-year lease can leave you stuck with outdated silicon while a competitor upgrades. If the hardware sits idle most of the time, renting beats owning. Contact ISD at [email protected] for further information.
Further reading: independent testing
We link to independent testers rather than reproduce their charts: the figures and charts belong to their publishers. Each takes a different approach, so compare like with like (same model, same Quantisation: Storing a model's numbers at lower precision (for example 4-bit) so it needs less memory. Glossary, same software).
- Artificial Analysis: hardware benchmarks: independent benchmarking firm testing datacenter systems, laptops and workstations, and phones.
- MLCommons: MLPerf Inference (datacenter), edge and MLPerf Client: industry-consortium benchmarks with published rules; results are submitted by vendors and reviewed.
- llama.cpp: performance on Apple Silicon: community-maintained results for the same small model across Mac chips, from the llama.cpp project.
- Puget Systems Labs: detailed workstation GPU testing with published methods. Puget also sells workstations.
- ServeTheHome: in-depth reviews of servers and compact AI systems, often of vendor-supplied review units.
- llm-speed open dataset: speed results from its own and community runs, published under an open (CC BY 4.0) licence.
- OpenBenchmarking.org: llama.cpp: crowd-sourced results from the Phoronix Test Suite across many CPUs and GPUs.
- Alex Ziskind on YouTube: hands-on video testing of local AI on Macs, PCs and compact AI systems. Many videos are sponsored; check each video's disclosures.
Benchmark speeds depend heavily on software versions and settings, and change often. Treat any single number as a snapshot.