What fits in a GB10's 110 GB

An NVIDIA GB10 (the DGX Spark) has one memory for everything: 128 GB, shared by the GPU and the system. About 110 GB is usable for models. This page is what fits in that, how to add it up, and why a model that stays under the line serves better than one that fills the machine to the brim. It holds for a rented GB10 and for a bare metal GB10 alike.

1. One memory for everything

• There is no separate GPU memory. The model's weights, its cache, the runtime, your other programs and the operating system all come out of the same 128 GB.
• Linux shows about 121 GiB in total (free -h on an idle rented GB10, 2026-11-05). The system and a safety margin need their part of it, which leaves about 110 GB for your models.
• nvidia-smi shows the memory as Not Supported: the GPU has none of its own. Read it with free -h, in the available column.
• The memory moves 273 GB/s (NVIDIA's figure). That, not the size, sets how fast one request gets its tokens: every token reads the weights it needs once.

2. Add it up

What a model needs is weights + cache + runtime:

Our rule of thumb, calculated from the weights alone:

• Fits: the weights take up to about 80 GB, so 30 GB or more is left for the cache and the runtime.
• Tight: 80 to 100 GB of weights. It loads, with a short context and few requests at once.
• No: more than about 100 GB of weights leaves no room for a useful cache, or does not load at all.

Calculated: parameters × bytes per parameter, rounded. Real files differ a little: some layers stay at a higher precision. vLLM's log counts in GiB (1 GiB = 1.074 GB).

What we run on our own GB10s, for scale:

To start from a model instead: the catalogue shows each model's builds with their size, and lists the language models that fit a DGX Spark (also image and video).

3. Why stay under the line

It is tempting to give the model every last GB. On a GB10 that is how we have seen machines stop. What we have measured on our own machines:

• When the memory runs out, the whole machine stops. SSH, your server and everything else stop answering; the machine still answers a ping and nothing more. The kernel's out-of-memory killer does not count what a program holds for the GPU, so it may not stop it, and a container's memory limit does not hold it either: on a GB10, a container limited to 12 GB took 24 GB for the GPU.
• vLLM takes a share of everything, not what the model needs. Its share (--gpu-memory-utilization) is a share of the whole machine's memory, whatever the model's size. In our test (2026-10-08), vllm serve with a 1 GB model at its default share took 110 GB and left the machine 4.3 GB. At 0.75 it served the same model and the machine kept 22.6 GB.
• It happens in seconds. A model too big for the line (a 72B model in BF16, 145 GB) took the machine from 119 GB free to 4.4 GB in 4 seconds. In another test, 8.6 GB was still free one second before the memory was gone. Looking at free -h now and then does not catch it.
• The last GB buy little. Serving Qwen3.8 Flash Next on one GB10 (2026-10-30), a cache sized by vLLM from a 0.80 share gave 16 users 141 tokens a second together; a fixed 10 GiB cache gave 131.5, 7% less. But with the bigger cache, 16 users with 8,000-token prompts used up the memory and the kernel stopped the server. The fixed cache kept at least 13.6 GB free through every test. Measured on our own serving build, not the rental's ready vLLM: your numbers depend on your build and settings.
• Starting needs room too. As the weights finish loading, free memory dips for a moment: our GLM servers dip to about 1.1 GB free at every start (2026-10-30). Set up for twice the requests at once (16 in place of 8, with its picture part on), the dip went to 0.4 to 0.5 GB and the server was stopped at its start, twice; a lower share barely moved it. Room at rest is not room at the start, nor at the heaviest moment.

What happens past the line depends on what you have:

4. How to stay under it

1. Pick a build that leaves room: 4-bit NVFP4 or FP8. The GB10's GPU (Blackwell) computes both in hardware.
2. Give the server a share that leaves the system its part. On a rented GB10, vllm serve starts at --gpu-memory-utilization 0.75. A fixed cache size is steadier still: the bare metal guide shows ours.
3. Allocate the context and the users you need, not the model's maximum: --max-model-len and --max-num-seqs in vLLM, -c in llama.cpp. An FP8 cache (--kv-cache-dtype fp8) holds twice the tokens.
4. Test at your heaviest load (the longest prompts, every user at once) with free -h open in a second terminal, not only at rest.

# the available column, every second, while you test
watch -n 1 free -h