# Your bare metal machine


# Your bare metal machine

A whole NVIDIA GB10 for you alone, with root over SSH. Nobody else uses its GPU, its memory or its
    disk. This page is the practical part. The price, the payment and the rules are on the
    bare metal page.

## 1. Connect

      - After you pay, the **Bare metal** page in the console asks for your SSH key: your saved key, or
      paste one.

      - The machine is installed fresh for you. We email you when it is ready. The email and the machine
      page give your exact line:

```
# HOST and PORT are your machine's, from its page in the console
ssh dev@HOST -p PORT
```

      - The first time, say yes to the machine's host key.

      - You log in with your key only: passwords are off. Keep it that way.

      - ⚠️ **Keep a second key.** Add it to `~/.ssh/authorized_keys` on the machine now: the
      console takes no new key once the machine is yours. If you lock yourself out, write to us with your new
      public key: we can reinstall the machine fresh, and everything on it is lost.

## 2. The machine

      - An NVIDIA GB10 (ARM64): **128 GB** of memory that the GPU and the system share, and a 4 TB disk
      (Linux shows about 3.7 TB).

      - NVIDIA's DGX OS (Ubuntu 24.04) as NVIDIA ships it, installed fresh: the NVIDIA driver, CUDA, Docker
      and the NVIDIA container toolkit. Anything more, you install yourself.

      - You are `dev`. `sudo` needs no password, and Docker runs without it.

      - Your software has to run on ARM64. `exec format error` means an image built for x86
      only: pull an arm64 or multi-platform one.

The GPU, from a container:

```
docker run --rm --gpus all nvidia/cuda:12.6.3-base-ubuntu24.04 nvidia-smi
# memory shows as Not Supported: the GPU shares it with the system (free -h shows it)
```

A few things of ours are on it: the login message with the rules, the AxForge agent (it tells us the
    machine is alive and reads nothing of yours) and a service that keeps the boot order. Leave them
    running. The rules in the login message, and under **The rules** on your machine's page in the
    console, say what else not to touch.

## 3. The network

      - The machine reaches the internet: `apt`, `pip` and `docker pull`
      work.

      - Nothing reaches it from outside except your SSH. It has no address of its own.

      - Keep SSH on port 22, open to every address: that is how you get in. Your connection does not arrive
      from your own address, so a firewall rule for your IP, or a tool that bans addresses after failed logins,
      locks you out. If you turn on a firewall, also keep port 9977 (the AxForge agent) open.

The apps run on the machine, not on the internet. To use one in your browser, forward its port:

```
# a second terminal: the app's port on your own computer (here 8888)
ssh -L 8888:localhost:8888 dev@HOST -p PORT
# then open http://localhost:8888
```

To serve something to the internet, run your own outbound tunnel from the machine.

## 4. Memory

      - Plan for about **110 GB** for your models. More can make the machine unstable or hang it.

      - Which models fit: models for a DGX Spark. Each
      model page shows the smallest card that runs it.

vLLM takes 90% of all the memory by default, whatever the model. Give it less:

```
# MODEL is the model you serve; 0.75 or lower leaves room for the system
vllm serve MODEL --gpu-memory-utilization 0.75
```

## 5. Reboot

      - While the machine answers, use `sudo reboot`. Not a shutdown.

      - If it stops answering, even over SSH, press **Reboot** on the machine page in the console. It
      cuts the power for a moment, like pulling the plug, and the machine takes a few minutes to come back.
      Work not saved to disk is lost.

      - One Reboot at a time: a second press within a few minutes does nothing.

      - A reboot never reinstalls the machine. Only the end of the subscription does.

## 6. Common snags

        | What you see | What it means | What to do |  |

        | `Permission denied (publickey)` | The key you connect with is not on the machine | Use the key you gave in the console (first block below) |  |

        | `REMOTE HOST IDENTIFICATION HAS CHANGED` | The machine was installed fresh | Forget its old host key (second block below), then connect again |  |

        | `Address already in use` on `ssh -L` | That port is taken on your computer | Pick another one on your side, for example `8889:localhost:8888` |  |

        | `exec format error` | An image built for x86 only | Pull an arm64 or multi-platform image |  |

        | The machine stops answering | Usually the memory | Press Reboot, and stay under about 110 GB |  |

```
# connect with a given key
ssh -i ~/.ssh/your_key dev@HOST -p PORT
# forget the old host key of a machine installed fresh
ssh-keygen -R "[HOST]:PORT"
```

## 7. Copy your files

Down from the machine, and up to it:

```
# the folder results on the machine, into this folder (again later: it is copied over)
scp -P PORT -r dev@HOST:results .
# the folder data here, into your home folder on the machine
scp -P PORT -r ./data dev@HOST:
# or rsync, which copies only what changed
rsync -av -e "ssh -p PORT" dev@HOST:results/ ./results/
```

```
# the folder results on the machine, into this folder (again later: it is copied over)
scp -P PORT -r dev@HOST:results .
# the folder data here, into your home folder on the machine
scp -P PORT -r .\data dev@HOST:
```

```
rem the folder results on the machine, into this folder (again later: it is copied over)
scp -P PORT -r dev@HOST:results .
rem the folder data here, into your home folder on the machine
scp -P PORT -r .\data dev@HOST:
```

`scp` takes the port as a capital `-P`; `ssh` takes a small
    `-p`.

## 8. The end

      - Renewing, cancelling and your card are in the console, under **Billing & Usage** ›
      **Bare metal**: **Cancel renewal**, **Turn renewal back on**, **Change card** and your
      invoices.

      - When you cancel, the machine stays yours to the end of the period you paid for.

      - ⚠️ **At the end the power is cut and the whole disk is wiped.** It cannot be recovered, and there
      are no backups. Copy out what you want to keep before the last day.

      - The full rules for paying, renewing and the end are on the bare metal
      page. Questions: hello@axforge.ai reaches a person.

      ← How GPU rentals work
      Bare metal →



Source: https://dev.axforge.ai/docs/bare-metal/