GPU dedicated servers · NVIDIA Ada Lovelace · by the month
Our RTX 4090 dedicated server runs the GeForce RTX 4090D, not the full RTX 4090. It keeps the 4090's 24 GB of GDDR6X, its 1,008 GB/s of memory bandwidth and its two AV1 encoders, and has 14,592 CUDA cores where the 4090 has 16,384. A complete server with one starts at $372.47 a month.
With our RTX 4090 server hosting you rent the whole machine. The GPU is passed straight to your own operating system, with root access and the server's out-of-band console. The price shown is the complete server: processor, memory, disks, the port and one IPv4 address, plus the GPU's own monthly line.
RTX 4090D vs RTX 4090
NVIDIA released the RTX 4090D on 28 December 2023, after a United States export rule stopped sales of the RTX 4090 in China. It is the same AD102 chip with fewer streaming multiprocessors switched on: 114 instead of 128. The memory, the boost clock and the video engines are the same; the board power limit is a little lower.
| Specification | RTX 4090 | RTX 4090Dwhat we fit |
|---|---|---|
| CUDA cores | 16,384 | 14,592 lower |
| Memory | 24 GB GDDR6X | 24 GB GDDR6X same |
| Memory bus | 384-bit | 384-bit same |
| Memory bandwidth | 1,008 GB/s | 1,008 GB/s same |
| L2 cache | 72 MB | 72 MB same |
| Boost clock | 2.52 GHz | 2.52 GHz same |
| Video encoders | 2 NVENC, AV1 | 2 NVENC, AV1 same |
| GPU power | 450 W | 425 W lower |
Anything limited by memory size or memory speed. A model that fits in 24 GB on a 4090 fits on a 4090D, and generating tokens from it is bound by the same 1,008 GB/s. Video encoding uses the same pair of NVENC engines. The driver, CUDA, PyTorch, Blender and every other tool treat it as an Ada Lovelace GeForce GPU.
Work bound by shader throughput: large training batches, Cycles and OptiX rendering, prompt processing on long inputs. The 4090D has 11% fewer CUDA cores at the same 2.52 GHz boost clock, so expect it to land behind a 4090 by up to that margin on those jobs. Board power is 425 W against 450 W.
The builds
These are the two RTX 40 GeForce GPUs our order catalogue fits to rack servers. The RTX 4090D is the 24 GB choice and stacks in one machine. The RTX 4070 has 12 GB and the same 8th-generation AV1 encoder, for jobs that do not need more memory.
24 GB GDDR6X · the GPU this page is about
$372.47/mo for the complete server
GPU $299/mo, in this machine:
Intel Xeon E5-2620 v4, 8 cores, 2.10 GHz · 32 GB DDR4 · 2 × SATA 500 GB (RAID 1) · 1 Gbps
12 GB GDDR6X · the 12 GB step
$262.67/mo for the complete server
GPU $179/mo, in this machine:
Intel Xeon Silver 4110, 8 cores, 2.10 GHz · 16 GB DDR4 · 2 × SATA-SSD 240 GB (RAID 1) · 1 Gbps
Each price is the cheapest machine that takes that GPU, as the configurator opens it, and the button opens that machine with the GPU already selected. Other processors, more memory, larger disks, a faster port and more GPUs are priced there. Bucharest builds are billed in euros; the configurator switches currency when you pick the city.
Sizing
Size inference by memory first. Weights take parameters times bytes per parameter, and the GPUs need a quarter to a half more for the KV cache and the runtime. The table applies that rule to the RTX 4070, to one RTX 4090D, and to several in one machine, which llama.cpp and vLLM split a model across.
| Model size (weights) | RTX 407012 GB | One RTX 4090D24 GB | Two RTX 4090D48 GB | Four RTX 4090D96 GB |
|---|---|---|---|---|
| 7-8 billion parameters | 4-bit: Fits8-bit: Fits | 4-bit: Fits8-bit: Fits | 4-bit: Fits8-bit: Fits | 4-bit: Fits8-bit: Fits |
| 13-14 billion parameters | 4-bit: Fits8-bit: No | 4-bit: Fits8-bit: Fits | 4-bit: Fits8-bit: Fits | 4-bit: Fits8-bit: Fits |
| 30-34 billion parameters | 4-bit: No8-bit: No | 4-bit: Tight8-bit: No | 4-bit: Fits8-bit: Tight | 4-bit: Fits8-bit: Fits |
| 70 billion parameters | 4-bit: No8-bit: No | 4-bit: No8-bit: No | 4-bit: Tight8-bit: No | 4-bit: Fits8-bit: Tight |
GeForce RTX 40 GPUs have no NVLink connector, so several 4090Ds in one server talk over PCIe. Splitting a model across them for inference works over that link. Training that exchanges gradients every step gains less from each extra GPU than it would on GPUs joined by NVLink.
A model that needs one large memory space, or ECC memory, belongs on a compute or workstation GPU with 48 or 80 GB. Those are on every GPU we fit. For a plan model by model, see LLM servers.
Platforms
Both GPUs have a PCIe 4.0 x16 interface. The Intel Xeon machines run it at PCIe 3.0, which halves the bandwidth to the host. The AMD EPYC machine gives each GPU a full PCIe 4.0 link. Once weights or a scene sit in GPU memory the link hardly matters; it does for training loops that stream data from system memory, and for splitting a model across GPUs.
| Machine | PCIe link | RTX 4090D | RTX 4070 |
|---|---|---|---|
| Intel Xeon E5-2600 v3/v4from $73.47/mo without a GPU | PCIe 3.0 | up to 4 | not offered |
| Intel Xeon Silver / Goldfrom $83.67/mo without a GPU | PCIe 3.0 | up to 3 | up to 4 |
| AMD EPYCfrom $217.59/mo without a GPU | PCIe 4.0 | up to 2 | not offered |
Machine prices are the cheapest build of each line without a GPU. RTX 40 servers can be ordered in New York, Miami, San Francisco and Bucharest. Amsterdam machines are not offered with these GPUs. Each building is described on data centers.
Workloads
One RTX 4090D serves a 13B model at 8-bit with room for context, or a 30B model at 4-bit with a short one. Run Ollama, llama.cpp or vLLM behind your own API; prompts and outputs stay on your machine.
3rd-generation RT cores accelerate Blender Cycles through OptiX, as well as Unreal Engine and V-Ray. A ray tracing GPU server on a flat monthly price renders overnight without a per-hour meter running.
The 4090D carries two 8th-generation NVENC engines and one NVDEC. Both GPUs encode AV1 in hardware, which the RTX 30 series cannot, so a live pipeline can serve AV1 next to H.264.
The GeForce driver runs DirectX, Vulkan and OpenGL as a desktop does. Install Windows or Linux and run a cloud gaming host, a streamed workstation or a game server with a dedicated GPU.
Delivery and terms
It is installed and handed over. Instant servers shows what is racked today.
A technician installs the GPU in the machine you configured, builds the disk array and installs the operating system. Added memory or drives go in on the same visit.
We order the part, then build as above. You are told which case applies before you pay.
Each timeline starts once payment and fraud review clear. Most orders clear within a couple of hours; a first order on a new account can take longer.
Owners list RTX 40 machines on our marketplace too, each at a monthly price the owner sets. A listing from our sister company, Primcast, is rented on primcast.com.
Specifications
| GPU | Memory | Bus | Bandwidth | CUDA cores | Boost | L2 | Encoders | Power |
|---|---|---|---|---|---|---|---|---|
| RTX 4090D | 24 GB GDDR6X | 384-bit | 1,008 GB/s | 14,592 | 2.52 GHz | 72 MB | 2 NVENC, AV1 | 425 W |
| RTX 4070 | 12 GB GDDR6X | 192-bit | 504 GB/s | 5,888 | 2.48 GHz | 36 MB | 1 NVENC, AV1 | 200 W |
GPU figures from NVIDIA's GeForce RTX 40 series comparison, its video encode and decode support matrix, and the published RTX 4090D launch specification. Prices, machines, cities and per-machine counts are read from our order catalogue, the one the checkout bills from.
Answers
A complete server with one RTX 4090D starts at $372.47 a month. That is the GPU at $299 plus the cheapest machine that takes it: Intel Xeon E5-2620 v4, 8 cores, 2.10 GHz · 32 GB DDR4 · 2 × SATA 500 GB (RAID 1) · 1 Gbps. Each extra GPU adds its own monthly line, and a GPU fitted to order adds a one-time setup of $29.
It is the RTX 4090D. It has the RTX 4090's 24 GB of GDDR6X, 1,008 GB/s of bandwidth, 2.52 GHz boost clock and two AV1 encoders, with 14,592 CUDA cores instead of 16,384. We say so here so that nobody finds out after paying.
At most about 11% on work bound by shader throughput, such as rendering and training, because that is the share of CUDA cores it lacks at the same clock. On work bound by memory size or bandwidth, such as LLM token generation, the two should perform alike.
Yes. One machine takes up to 4 of them, 96 GB of GPU memory in total, and the configurator shows the most each machine takes. The GPUs connect over PCIe, since GeForce RTX 40 GPUs have no NVLink.
Bare metal. The machine is rented to one account and the GPU is passed straight through to your operating system: no hypervisor, no vGPU slice, no time-sharing. You install the NVIDIA driver and CUDA version you choose.
Hourly billing applies to a machine already racked and rented as it stands, so it depends on what the instant servers page lists that day. A GPU fitted to a machine you configured is billed by the month, with no contract.
Yes. Its 3rd-generation RT cores accelerate ray tracing in Blender Cycles with OptiX, Unreal Engine, V-Ray and other renderers, and 24 GB holds large scenes. On a flat monthly price, long renders cost the same as short ones.
About 30 minutes after approval when a machine with the GPU is already racked. When we fit a GPU from stock, 4 to 24 hours; when the GPU has to be bought in, 5 to 10 working days, and you are told which before you pay.
In New York, Miami, San Francisco and Bucharest, on Server Room's own network. Amsterdam machines are not offered with these GPUs.
The RTX 4070 server starts at $262.67 a month complete, GPU $179 of that. It has 12 GB of GDDR6X and one AV1 encoder, and one machine takes up to 4.
Choose how many GPUs, then the processor, memory, disks and city. The configurator reprices each change as you make it.
Every GPU we fit · RTX 30 servers · RTX 50 servers · Racked right now