NVIDIA data-centre GPUs

In practice, the compute capability that CUDA software queries tells the generations apart: 6.1 on the P40, 7.0 on the V100, 7.5 on the T4, 8.0 on the A100, 8.6 on the A40 and RTX A6000, 8.9 on the L4. Many AI frameworks require a minimum version, and older architectures gradually drop out of support in new CUDA releases. So check what your software needs before you buy.

Data-centre GPU or GeForce in a server?

CUDA runs on both. In a server, though, four differences decide:

• Passive cooling: cards such as the P40, V100, A100 or A40 have no fan. They rely on the front-to-back airflow the server's fans create. Consumer cards blow their heat into the chassis with their own fans and are often taller than a PCIe slot allows for – in a 2U server with risers they frequently don't fit.
• ECC memory: HBM2 on the V100 and A100 is error-corrected out of the box; on GDDR cards such as the T4, A40 or L4, ECC can be toggled with nvidia-smi. Over days of training or simulation, that prevents silent errors from flipped bits.
• Drivers and licensing: data-centre cards use the datacenter driver branch and, depending on model, support vGPU (licensed separately) and MIG. The GeForce driver licence restricts data-centre use; check the current terms with NVIDIA.
• Power connector: many passive cards take an 8-pin CPU (EPS) connector rather than a PCIe plug. Servers built for GPUs come with the right cables.

Which card for which job?

• Entry-level LLM inference: Tesla P40 with 24 GB, but slow at FP16. Our guide AI servers for local LLMs compares it with the V100 and L4.
• Inference and video in a 1U server: T4 or L4, both low profile and without auxiliary power cables.
• Training and HPC: V100 or A100. The 80 GB A100 also holds large models and can be split into up to seven instances with MIG.
• VDI and rendering: A40 with vGPU in the server, RTX A6000 in the workstation.

Servers with the space, power and airflow for several cards are under GPU servers, such as the Dell R740 with 2× Tesla V100, or systems from Supermicro with room for up to eight cards. We take decommissioned cards through our buyback service.