DGX Spark, Cloud, or a GPU Inference Node in 2026

Inference has two hard limits, and neither is the one marketing talks about. Capacity: weights plus key-value cache must fit in one pool of memory, or the model does not run. Bandwidth: generating a token means reading essentially the whole active weight set out of memory, so the memory bus sets the speed, not the arithmetic units.

NVIDIA says the second part plainly: “The speed at which the data… is transferred to the GPU from memory dominates the latency, not how fast the computation actually happens… this is a memory-bound operation” (NVIDIA Technical Blog, November 17, 2023). Everything below follows from taking that literally.

Capacity and bandwidth: NVIDIA datasheets and product pages for the P100, Quadro RTX 6000, A100, H100 and DGX Spark, read September 20, 2026. The RTX 3070’s 448 GB/s is not on NVIDIA’s specification page: it is the third-party tracker GPUDojo’s figure, flagged as such, as is the used A100 asking price — a lowest current offer across marketplaces, not a completed sale. Our prices and unit counts: PCSP live catalog, same day. Dollars per GB are our division of price by memory.

Both rankings are true and they disagree. The Spark buys memory at $36.71 a gigabyte against the Quadro’s $41.67; the P100 buys bandwidth 98× cheaper. Which one matters depends on the model.

Read the table twice. Down the bandwidth column the ten-year-old HBM2 card wins outright, 732 against 273 GB/s. Down the dollars-per-gigabyte column the ranking inverts, and the Spark is the cheapest memory except the P100 — which only wins there in 16 GB slices a model cannot join.

That clause is the difficulty. Four P100s are $514.60 and 64 GB, still $8.04 per GB, but four separate 16 GB pools. A framework can split a model across them; a model needing 40 GB contiguous cannot have it. The Spark’s 128 GB is one coherent pool, and that, not the token rate, is what $4,699 buys.

“An 8B model needs 8 GB” is the sentence that wastes the most money here. Two things go into memory, and the second is usually the bigger. Weights first, measured rather than estimated: below are the published byte sizes of the GGUF files, pulled from the Hugging Face model API on September 20, 2026 and re-pullable by anyone.

Published file sizes from the Hugging Face model API, September 20, 2026, shards summed where a quantization is split. All rows are the unsloth GGUF repositories, the most-downloaded mirrors but not the model authors, so another publisher’s Q4_K_M will differ by a percent or two. Note the gpt-oss rows: those weights already ship in MXFP4, so Q8_0 is barely larger than Q4_K_M and “quantize it and it halves” is false for that family. The fit column is ours and counts weights only.

Now what that column leaves out. Every token of context occupies the key-value cache, and NVIDIA publishes the formula in the same guide: bytes per token = 2 × layers × (heads × head_dim) × bytes_per_value. Applied to the models’ own configuration files, with grouped-query attention accounted for:

• Llama 3.1 8B — 32 layers, 8 key-value heads, head dimension 128: 128 KiB per token, so 4.00 GiB at 32,768 tokens and 16.00 GiB at its full 131,072.
• Qwen3.5-9B — 32 layers, 4 key-value heads, head dimension 256: also 128 KiB per token, and its native 262,144-token context is 32 GiB of cache.
• Llama 3.3 70B — 80 layers, 8 key-value heads, head dimension 128: 320 KiB per token, 10.00 GiB at 32,768.

Put the halves together. A Llama 3.1 8B at Q4_K_M is 4.92 GB of weights and up to 16 GiB of cache — three and a half times the weights. On a 16 GB P100 that leaves roughly 80,000 tokens of context; on an 8 GB RTX 3070, roughly 20,000. Both are before activations, CUDA context and runtime overhead, so treat them as ceilings and redo the arithmetic for your own model.

Sort any used-GPU listing by gigabytes per dollar and the same board comes out on top: an NVIDIA Tesla M10, 32 GB GDDR5, $94.99, twelve on our shelf on September 20, 2026. At $2.97 per gigabyte it undercuts every other figure here — the next cheapest is the P100 at $8.04 — and it is unusable for this job. The correction is the manufacturer’s.

NVIDIA’s Tesla M10 Product Brief (PB-08118-001, revision v04, February 13, 2017) opens: it is “a dual-slot 10.5 inch PCI Express Gen3 graphics card with four mid-range NVIDIA Maxwell graphics processing units (GPUs)… 32 GB GDDR5 memory (8 GB per GPU) and a 225 W maximum power limit.” Three more rows settle it:

• The memory table prints bandwidth as “83 GB/s ×4” and the interface as “128-bit ×4”. The multiplication sign is NVIDIA’s: four independent memory systems, not one 32 GB pool.
• The software table gives the compatibility mode as “Graphics” and only Graphics. The Tesla M60 brief lists “Graphics (default) and Compute”. The M10 has no compute mode at all.
• NVIDIA states what it is for: “designed to accelerate graphics in virtual desktop and application environments… up to 64 users per Tesla M10.”

So a model on an M10 sees 8 GB, four times over, on four Maxwell GM107 dies at 83 GB/s each — roughly a ninth of one P100’s bandwidth, two generations below the standard serving engine’s floor. The honest figure is not $2.97 per gigabyte but $23.75 per usable 8 GB GPU. The $59.99 Tesla M60 beside it is the same story with two GM204 dies at “160 GB/s ×2”.

We sell the M10, and we rank for the exact question: in our Search Console for the 28 days to September 17, 2026, nvidia tesla m10 32gb put us at position 5.0 and tesla m10 32gb at 6.0 — 348 impressions across five M10 queries, zero clicks. It is not a 32 GB card, and it remains a good buy for the virtual-desktop job NVIDIA built it for.

The most expensive mistake in a used inference build is not too little memory. It is a card your serving software will not compile for, and the two most popular engines answer differently.

vLLM has a published floor and enforces it in the build. Its installation page states the CUDA requirement as “GPU: compute capability 7.5 or higher (e.g., T4, RTX20xx, A100, L4, H100, B200, etc.)”. Not documentation drift: on main its CMake configuration carries a supported-architecture list per CUDA version, every one of them beginning at 7.5, and 6.0 and 6.1 appear in none. With a current toolkit, even a V100 at 7.0 is off the list.

llama.cpp has no such floor. Its build documentation says it compiles “for the hardware that is connected to the system at that time,” and its CUDA header carries a named constant for Pascal, GGML_CUDA_CC_PASCAL 600. A P100 running llama.cpp is a working, sensible thing. What it loses is specific: the same header defines GGML_CUDA_CC_DP4A 610, “minimum compute capability for __dp4a, an intrinsic for byte-wise dot products.” At 6.0 the P100 sits a tenth of a version below that line, so integer-quantized kernels lose their fast path. The Tesla P40, at 6.1, has it — and 347 GB/s instead of 732. Pascal split its own generation down the middle.

What that means for the shape of your deployment

This is the decision the price tags hide. vLLM is what several people or applications share: continuous batching, paged attention, an OpenAI-compatible endpoint. llama.cpp serves one or a few requests at a time. So a P100 node is a capable small-team box, not a multi-tenant serving stack. Going multi-card, vLLM’s parallelism documentation contradicts the folklore: NVLink is not required. “If the GPUs on the node do not have NVLINK interconnect (e.g. L40S), leverage pipeline parallelism instead of tensor parallelism.” One hard constraint is asserted in its source: attention-head count must divide by the tensor-parallel size, so a 32-head model runs on one, two, four or eight cards and not three.

The expiry date, which nobody selling this hardware prints

CUDA 13.0 dropped Maxwell, Pascal and Volta. The current CUDA Toolkit release notes record it under both cuFFT and cuSPARSE: support “removed… for Maxwell, Pascal, and Volta GPUs, corresponding to compute capabilities earlier than Turing.” NVIDIA’s developer blog dates the end. In “Navigating GPU Architecture Support” of August 4, 2025, Jonathan Bentz and Tony Scudiero write that the 580 branch “will be the final branch that supports GPUs before CC 7.5… an LTS driver that will be supported for three years from release and supported until mid-year 2028.”

The same post carries the reprieve: “Applications built with older versions of the CUDA toolkit will continue to run as before on all hardware supported by the driver.” A P100 bought today is not a brick in 2028; it is a card whose software stack stops moving. A pinned container running a known llama.cpp build is an acceptable trade for $128.65; a plan resting on the 2029 ecosystem is not. Our used graphics card guide maps the same cliff across the used market.

The Spark is a 150 × 150 × 50.5 mm, 1.2 kg desktop box on the GB10 Grace Blackwell superchip: a 20-core Arm CPU, 128 GB of coherent unified LPDDR5x on a 256-bit interface at 273 GB/s, 4 TB of NVMe, a 240 W supply and a 140 W GB10 TDP. It went on sale October 15, 2025, two days after NVIDIA’s launch announcement.

Two things about the headline number. NVIDIA’s page says “Up to 1 PFLOP” of tensor performance, and its own footnote reads “Theoretical FP4 TOPS using the sparsity feature” — a theoretical four-bit figure assuming structured sparsity, never to be set against a dense FP16 number. And NVIDIA’s framing is developmental: its three workload headings are Prototyping, Fine-tuning and Inference, the copy offers “test, validate, and inference with AI models up to 200 billion parameters”, fine-tuning to 70 billion, and 700 billion across four linked units. NVIDIA never claims production.

The price has already moved once. On February 25, 2026 an NVIDIA moderator posted a price-change announcement on the developer forums: “The MSRP for DGX Spark (Founders Edition) has been adjusted from $3,999 to $4,699 due to memory supply constraints… There are no hardware or configuration changes tied to this adjustment.” That is $700, or 17.5% (our arithmetic), from the same DRAM squeeze running through every server bill of materials this year. Availability we could not confirm on September 20, 2026, so treat $4,699 as a price, not a ship date.

Measured throughput, and the rule it reveals

The best first-party numbers live in llama.cpp’s own repository, in the results file behind maintainer Georgi Gerganov’s DGX Spark benchmark discussion of October 14, 2025, last re-run February 5, 2026. For gpt-oss 120B MXFP4 at 59.02 GiB on the CUDA backend: 2,443.91 tokens/s of prompt processing, 58.72 of generation. Fill the context and both fall — at depth 32,768, to 1,567.08 and 42.76. Contributors posted the rest in the thread:

llama.cpp benchmark discussion #16578, figures posted by contributors in November 2025 and March 2026, read September 20, 2026; tg32 generation, error bars in the thread. The first four rows are one Spark; the 124.91 GiB file in the last was run across two linked over RPC. The active-parameter column is read out of each model’s own label — the A3B to A22B suffixes — not estimated by us.

Active parameters, not file size, set the rate against a fixed 273 GB/s. llama.cpp benchmark file, re-read September 20, 2026.

Size and speed do not track each other. What orders the table is the active-parameter count printed in each model’s own name. A 35-billion file with 3 billion active runs at 59.98 tokens/s and an 80-billion file with the same 3 billion active at 49.33, while raising the active set to 10 and then 12 billion drops it to 21.33 and 16.58 — and 22 billion active takes two linked boxes to manage 12.98. Set that beside 58.72 tokens/s on a 116.83-billion-parameter gpt-oss file and the rule is plain: total size decides whether it loads, active parameters decide how fast it answers. So a 128 GB unified-memory box bought for a dense 70B is the wrong purchase; bought for a sparse 120B it is the point of the product.

Every rent-versus-buy article picks one cloud rate and builds on it; here is why that is worthless. On September 20, 2026, from each provider’s own price page or API, an H100 80 GB rented at $1.99 an hour on RunPod’s Community Cloud tier and at an effective $12.29 on Azure — our division of the Standard_ND96isr_H100_v5 rate of $98.320 by the eight GPUs its naming implies. A 6.2× spread on the same chip on the same day, both published list prices.

List prices captured September 20, 2026 from each provider’s own page: RunPod (whose page states “Updated September 13, 2026”), Lambda, CoreWeave North America, Google Cloud us-central1, AWS us-east-1 Linux from the feed its on-demand page consumes, and Azure East US from its retail prices API. Per-GPU figures on multi-GPU SKUs are our division of the instance rate by the GPU count implied by the provider’s own series naming. Annual figures are rate × 8,760, our arithmetic, excluding storage, egress and tax. Spot, pre-emptible and committed-use rates are lower and none appear here. No provider publishes current capacity, so these are list prices, not a promise of an instance.

Three details there beat any average. Google’s newest generation has no on-demand price at all: the B200-based a4-highgpu-8g lists only Dynamic Workload Scheduler, spot and committed-use rates, so the frontier is not rentable by the hour at list price even from a hyperscaler. AWS GPU list prices have not moved in three weeks: g6e.xlarge at $1.861 and the p5 and p4d per-GPU figures of $6.88 and $2.74 match those we published on August 28, 2026. And the last row is the cleanest comparison here: Lambda rents, for $0.69 an hour, the exact card that left our shelf on September 22.

Electricity below is the U.S. average commercial retail price of 14.19 ¢/kWh for June 2026, from the EIA’s Electric Power Monthly, Table 5.6.A, released August 26, 2026. Annual cost at 24/7 is kW × 8,760 × 0.1419; your own utility rate is the one that matters.

Hardware: PCSP live catalog, September 20, 2026 — four Tesla P100s at $128.65 plus the cheapest complete 2U on our priced rack shelf, an HPE ProLiant DL380 Gen9 with two E5-2640 v3, 64 GB and eight 600 GB drives at $750.67. DGX Spark MSRP from NVIDIA’s February 25, 2026 forum post. Wattages are NVIDIA board TDPs plus an assumed ~300 W host, our estimate and the softest number here. Electricity: EIA, June 2026, 14.19 ¢/kWh. Cloud rows: 8,760 hours at the list rates above. Our arithmetic throughout.

Solid is the first year, outline the third. Renting passes every owned option inside year one. Electricity at the EIA commercial average for June 2026.

Against the identical rented card the sums are short. A Quadro RTX 6000 at our September 20 price of $999.99 divided by Lambda’s $0.69 an hour is 1,449 hours — 60 days of continuous running. In the $750.67 host, capex becomes $1,750.66; subtract the node’s electricity at roughly 510 W, $0.0724 an hour, and the saving is $0.6176 an hour, so 2,835 hours, 118 days. Nothing else here is that clean, because nothing else is the same card on both sides — and the owned side is the scarce one: our unit sold two days after we priced it.

Four P100s against a rented A100 look cleaner still and are not: $1,265.27 of capex and $0.1845 an hour of electricity at 1,300 W against RunPod’s $1.19 A100 give 1,258 hours, 52 days. The caveat belongs in the same breath: that A100 is one 80 GB pool at 1,935 GB/s with tensor cores and vLLM support; four P100s are four 16 GB pools at 732 GB/s each, with no vLLM. The hours are real; the work is not.

The crossover that goes against us

Now the comparison we would rather not run. In year one the four-P100 node costs $1,265.27 of hardware plus $1,615.96 of electricity: $2,881.23. A DGX Spark costs $4,699 plus $174.03: $4,873.03. The node wins year one by $1,991.80, then stops: from year two it costs $1,615.96 a year against the Spark’s $174.03, a gap of $1,441.93 every year. Divide the $3,433.73 difference in entry price by that gap and the Spark is ahead on electricity alone after 2.4 years of round-the-clock operation, cheaper to own thereafter.

Two asterisks. It assumes a genuinely 24/7 duty cycle; at 20% utilisation the electricity line shrinks by four fifths and the used node keeps its lead far longer. And below roughly 15–20% utilisation, renting beats owning of any kind — the crossover our AI server for business guide works through.

A Tesla P100 is $128.65. The parts that make it work are not, and most are not on our shelf at any price. Every requirement below is a vendor’s own.

NVIDIA sets the airflow, in numbers. The P100 product brief says the card “uses a passive heat sink for cooling, which requires system air flow to properly operate the card within its thermal limits”, then prints minimum CFM per inlet temperature: 12 CFM at 30 °C, 15 at 35 °C, 19 at 40 °C, 26 at 45 °C, 37 at 50 °C, assuming “a fully ducted airflow through the product’s heat sink”. It also takes a CPU 8-pin, not a PCIe 8-pin. That ends the “drop it in a tower” plan, in NVIDIA’s words, not ours.

Dell sets the rest, for an R730. Its GPU card installation guidelines require redundant 1,100 W supplies, a processor of 135 W or less with the GPU-kit low-profile heat sink, an optional third riser for two double-wide cards, an ambient inlet restricted to 30 °C, and the rule that catches mixed used buys: “All GPU cards must be of the same type and model.”

HPE sets it for a DL380 Gen9, and contradicts our own shelf. Its QuickSpecs (DA-15034, version 48, December 2, 2019) require the Graphics Enablement Kit 719082-B21, a GPU-ready riser, “at least 1x1400W Power Supply… per card for cards over 150W”, and for passive boards “the High Performance Fan Kit (719079-B21)”. Mixing GPUs is not supported. Most listed accelerators carry the note “supported on the E5-2600v4 series processors only” — and the cheapest complete 2U on our priced rack shelf runs two E5-2640 v3 chips, a generation below what HPE’s own document asks. HPE added the P100 to this chassis’s options in November 2016; by the last QuickSpecs revision it was no longer orderable there.

What that means in a shopping list. We sell the card. We list no GPU enablement kit, GPU-ready riser, high-performance fan kit or CPU-8-pin-to-PCIe cable anywhere on our components shelf, and nothing we price is sold as a GPU platform. Mind the floor trap on the rack server and 2U categories: the cheapest listing is a $104.99 barebones R730 with no CPU, memory or disks, the cheapest complete machine $750.67. Never build a budget on the first number.

Two limits close it. The cheapest 1U on our priced shelf, a SuperMicro CSE-113-7, cannot take this card at all: the P100 is dual-slot, full-height, full-length, and no riser changes that. And a GeForce card in a rack is a licensing question before a cooling one: NVIDIA’s GeForce software licence of February 25, 2025 states at §2.8 that the software “is not licensed for datacenter deployment.” Which is why the 201-unit RTX 3070 is a workstation answer; the used GPU server guide makes the full case.

We sell refurbished hardware, and on this topic the honest answer points away from us more often than towards. Here is the case, with our own catalog as evidence.

The shelf is thinner than the category page suggests. Our graphics card category holds 51 priced listings and 654 units on September 24, 2026, and 365 of those units — 56% — carry 4 GB or less. Only 64 sit at 12 GB or above, twelve of them the M10 that cannot pool its memory, and since our one 24 GB card sold, nothing holds more than 16 GB on one die.

Nothing modern carries a price, and the list deserves saying out loud. The priced shelf has no H100, H200, A100, L40S, L4, A6000, RTX 6000 Ada, RTX PRO 6000, RTX 4090 or 5090; no NVLink bridge or SXM module; nothing above 16 GB on one die. The newest architecture we price is Ampere, as a consumer RTX 3070. Our GPU Servers category is fourteen build-to-order platforms — Dell R760xa and R7525, HPE DL380a and the like — carrying no prices at all. An L40S- or A100-class accelerator there is configured and quoted, never picked off a shelf, and any dealer’s “in stock” claim about those parts deserves a sceptical read, ours included.

Electricity beats us, and the table above proves it. Four P100s and a host burn $1,615.96 a year against a Spark’s $174.03, so run around the clock, the small new machine is cheaper in total inside three years — quieter and smaller with it.

The software floor rules out our best card for the commonest deployment. Serving concurrent users through an OpenAI-compatible endpoint means vLLM, and vLLM will not build for a P100: the 7.5 floor puts our whole Pascal inventory outside it.

Buy new, or rent, when any of these is true. Your model needs more than 16 GB in one pool and you will not manage a multi-card split. Your workload is bursty or seasonal. You need a supported stack with a vendor to call. You are fine-tuning rather than serving, which takes several times the memory. Or it is going in an office, where nobody tolerates a 1,300 W 2U on high-performance fans.

And the price spread is public. The tracker GPUDojo, which reports the lowest current offer across eBay, Amazon and Newegg and earns affiliate commissions, listed a used Tesla P100 at $99 and a new one at $94 on September 20, 2026. Ours is $128.65, $29.65 above the cheapest asking price. That buys a tested card, a 90-day parts warranty and a return path; our used graphics card guide sets out how we test. Whether it is worth $30 is your judgement.

Where we are genuinely right is narrow and real: a small-team inference node, 16 GB of fast memory per card, llama.cpp rather than vLLM, a chassis you own or can equip. For that, $128.65 for 732 GB/s is a bargain nothing else here matches. For every other shape one of the other two columns wins, and better to read it here than after the freight.

What AI server should my company buy?

Three questions, in this order. What is the largest model you will actually serve, measured as its published file size plus a key-value cache at your real context length? How many people hit it at once, which decides whether you need vLLM and therefore compute capability 7.5 or higher? And what is the duty cycle, because below roughly 15 to 20 per cent utilisation renting wins? Only then does a price list matter.

Is a DGX Spark worth buying for local inference?

It depends on the model shape. Its 128 GB of coherent unified memory at $36.71 per gigabyte is the cheapest large single pool discussed here, and at 140 W it costs $174.03 a year at 24/7 on the EIA's June 2026 commercial rate. But 273 GB/s is less bandwidth than a ten-year-old Tesla P100's 732 GB/s, so generation tracks active parameters: in llama.cpp's benchmark thread, raising the active parameter count from 10 to 12 billion dropped generation from 21.33 to 16.58 tokens per second. Buy it for a sparse mixture-of-experts model, not a dense 70B.

Can I run vLLM on a Tesla P100?

No. vLLM's installation documentation requires "compute capability 7.5 or higher", and its CMake configuration lists no supported architecture below 7.5 under any current CUDA version. NVIDIA lists the P100 at 6.0. llama.cpp does run on Pascal and carries a named constant for it, so a P100 makes a working single-user or small-team node; what it lacks is the fast byte-wise integer path that begins at 6.1.

Is the NVIDIA Tesla M10 really a 32 GB card?

Not in any way that helps a language model. NVIDIA's own product brief describes "four mid-range NVIDIA Maxwell graphics processing units" with "32 GB GDDR5 memory (8 GB per GPU)", prints the bandwidth as "83 GB/s x4", and gives the compatibility mode as Graphics with no compute mode at all. It is a virtual-desktop board for up to 64 users per card. A model sees 8 GB, four times over, so the honest figure is $23.75 per usable 8 GB GPU.

Do you sell H100, A100 or L40S cards?

They are not on our priced shelf. Our GPU Servers category is fourteen build-to-order platforms - Dell R760xa and R7525, HPE DL380a and similar - configured and quoted rather than stocked, and an L40S- or A100-class accelerator is specified as part of that quote. The priced graphics-card shelf has topped out at 16 GB on one die since our single 24 GB Quadro RTX 6000 sold on September 22, and nothing we price is newer than Ampere.

How long will an old NVIDIA card keep getting drivers?

NVIDIA has published the date: its developer blog of August 4, 2025 states that the 580 branch "will be the final branch that supports GPUs before CC 7.5", that it is an LTS driver "supported until mid-year 2028", and that CUDA 13.0 removed offline compilation for Maxwell, Pascal and Volta. Applications built with older CUDA toolkits keep running on all hardware the driver supports, so a pinned container is fine; a plan resting on the 2029 serving ecosystem is not.

Is the cloud or an owned inference node cheaper?

Utilisation decides, and so does which tier you compare with. On September 20, 2026 an H100 80 GB listed at $1.99 an hour on RunPod's Community tier and at an effective $12.29 on Azure, a 6.2 times spread on the same chip. Against the one identical pairing available - Lambda's $0.69 an hour for a Quadro RTX 6000 - the $999.99 we asked for ours before it sold pays for itself in 1,449 hours, or 60 days; 118 days once a host and its electricity are counted.