Jupyter GPU server: JupyterLab on a dedicated GPU, reached over SSH

Sizing the machine

Size card memory first, system memory second, disk third. A model or batch that does not fit in card memory does not run slower, it fails to allocate. System memory has to hold whatever your notebook loads into a dataframe, and for a shared hub the TLJH project publishes a rule per resource.

TLJH suggests measuring one typical session yourself and adding 20-40% headroom per user; a user who exceeds the memory allotted to them gets a restarted kernel. For a single user the same logic applies with a user count of one.

Why the whole machine

A GPU dedicated server is one tenant's machine, and its card is handed to your own kernel by PCIe passthrough. Nothing virtualises the card or queues other jobs onto it. The OS, the NVIDIA module and the CUDA release are yours to choose and hold at the versions your notebooks were tested with.

• Root and a console. The GPU machines come with their own iLO or iDRAC console on a separate management network, so a driver install that takes the network down still leaves you the screen, virtual media and power control.
• Unmetered port. Datasets in and checkpoints out are not counted; there is no allowance and no egress line.
• Monthly, no contract. The GPU is its own line on the invoice; swap it later and the bill moves by the difference between the two cards.

Which machines fit

The GPU page sorts cards into three shelves. Compute accelerators carry large memory and ECC, and suit fine-tuning and runs that have to finish. Workstation cards carry certified drivers and, on most of the line, ECC. GeForce cards give the most compute per dollar with no ECC, which is acceptable for experiments where a flipped bit costs a retry.

Each host platform gives the slot a different PCIe generation and lane count, and the page prints both. The Xeon E5-2600 v3/v4 platform goes to 1 TB of memory, which matters more for pandas work than the card does. Disks, RAID level and memory beyond the entry build are chosen in the configurator; a racked machine is live about 30 minutes after the order is approved, and a build to order takes 4-24 hours. The five cities are listed on the data centers page.

Driver, PyTorch and JupyterLab

1. Update and reboot, then let Ubuntu pick the server (ERD) driver with ubuntu-drivers install --gpgpu. Canonical's guide also installs the nvidia-utils-…-server package for the same branch, which provides nvidia-smi; take the branch number from ubuntu-drivers list --gpgpu.
2. Create a virtual environment and install PyTorch with the command the pytorch.org selector prints for Linux, pip and your CUDA version. That page asks for no separate CUDA toolkit; the toolkit appears only in its build-from-source prerequisites.
3. Turing and newer cards (RTX 20 series and later, T4, Quadro RTX, A-series, L4, L40S, H100, Blackwell) take the default pip install torch torchvision, a CUDA 13 build that needs a driver from the 580 branch or later. Maxwell, Pascal and Volta cards (Tesla M6/M60, GTX 1070/1080/1080 Ti, P4/P40/P100, V100, TITAN V) need the pinned CUDA 12.6 build: PyTorch's CUDA 13 wheels leave those architectures out, and its notice says 2.14 is the last release with 12.6 wheels. Kepler cards such as the K3100M are not covered by current PyTorch builds at all.
4. Install JupyterLab in the same environment and confirm the card is visible from Python.

sudo apt update && sudo apt upgrade
sudo reboot
sudo ubuntu-drivers list --gpgpu
sudo ubuntu-drivers install --gpgpu
sudo apt install nvidia-utils-<branch>-server   # e.g. 580
sudo reboot
nvidia-smi

sudo apt install python3-venv
python3 -m venv ~/jl
source ~/jl/bin/activate
# Turing and newer: default CUDA 13 build, driver 580+
pip install torch torchvision
# Maxwell, Pascal, Volta instead:
# pip install torch==2.14.0 torchvision==0.29.0 --index-url https://download.pytorch.org/whl/cu126
pip install jupyterlab
python -c "import torch; print(torch.cuda.is_available(), torch.cuda.get_arch_list())"

torch.cuda.get_arch_list() returns the architectures the wheel was compiled for. Compare that list with your card's compute capability rather than trusting is_available() alone.

Access: localhost, token, tunnel

Jupyter Server listens on localhost by default and protects itself with a token that it prints at start-up. Leave both alone. Start JupyterLab inside tmux so it survives your SSH session, then forward the port from your laptop with ssh -N -L: -L maps a local port to one on the server, -N opens no remote shell.

# on the server, inside tmux
source ~/jl/bin/activate
jupyter lab --no-browser --ip=127.0.0.1 --port=8888

# on the laptop
ssh -N -L 8888:127.0.0.1:8888 you@your-server
# open the http://127.0.0.1:8888/lab?token=... URL the server printed

Port 8888 never appears on the public interface, so the firewall needs only SSH open. If a local Jupyter already uses 8888 on the laptop, pick another local port, for example 8889:127.0.0.1:8888. Run jupyter server password if you prefer a password to the token; the hash goes into jupyter_server_config.json.

Setting the token and password to empty strings disables authentication. The Jupyter docs mark that NOT RECOMMENDED unless another layer handles access. A notebook server on a public IP with no auth is a shell for whoever finds it.

More than one user: JupyterHub

Jupyter's public-server guide says it is for a single user; several people on one Jupyter Server will clobber each other's work. JupyterHub is the multi-user answer: each user signs in and gets their own single-user server. The Littlest JupyterHub (TLJH) packages it for one machine on a current Debian or Ubuntu LTS, needs root and at least 1 GB of RAM, and installs into a conda environment that every user shares.

sudo apt install python3 python3-dev git curl
curl -L https://tljh.jupyter.org/bootstrap.py | sudo -E python3 - --admin yourname

# once a DNS name points at the server
sudo tljh-config set https.enabled true
sudo tljh-config set https.letsencrypt.email you@example.com
sudo tljh-config add-item https.letsencrypt.domains hub.example.com
sudo tljh-config reload proxy

# GPU PyTorch for every user, from a hub terminal as an admin
sudo -E pip install torch torchvision
# Maxwell, Pascal, Volta instead:
# sudo -E pip install torch==2.14.0 torchvision==0.29.0 --index-url https://download.pytorch.org/whl/cu126

TLJH ships the FirstUseAuthenticator: whatever password is typed at an account's first login becomes its password, the admin's included. Keep ports 80 and 443 closed in the firewall while you install. Claim the admin account through a tunnel, ssh -N -L 8000:127.0.0.1:80 you@your-server, then http://127.0.0.1:8000 in your browser, and set a strong password. Open 80 and 443 only when you switch HTTPS on, add users after that, and tell each one to log in as soon as you create the account.

Every user's kernels allocate from the same card, and nothing in this setup divides its memory between them. PyTorch's caching allocator keeps freed memory reserved, so nvidia-smi shows it as used; torch.cuda.empty_cache() hands back the unused cache, and shutting an idle kernel down frees the rest.

Running it well, and when not to

• Keep notebooks in git and copy checkpoints off the machine. A RAID 1 mirror survives a disk, not a deleted directory.
• Pin the driver branch once PyTorch works, and upgrade it deliberately, with the console open.
• Do not apt upgrade the NVIDIA packages under a running training job.

This is the wrong place for a GPU you need a few hours a month; a monthly machine sits idle the rest of the time. Past one card the build is quoted by hand on multi-GPU servers. To serve a trained model behind an API rather than explore in notebooks, the vLLM server page covers that job.