Which GPU is suitable for local LLMs with Ollama or vLLM?

GPU memory is what counts. 24 GB (P40, L4) is enough for models up to about 30 billion parameters in 4-bit quantization, 48 GB (A40, RTX A6000) for 70B models. vLLM requires compute capability 7.0 or higher, so it runs from the V100 upwards but not on the P40; llama.cpp and Ollama also support the P40.

Can I run a passive server GPU in a workstation?

Only with additional cooling, such as a fan adapter that pushes air straight through the heatsink. Without directed airflow, passive cards throttle quickly or shut down. The actively cooled RTX A6000, on the other hand, is designed for workstations.

Which power cable do I need?

The Tesla P40, V100, A100, A40 and RTX A6000 use an 8-pin CPU (EPS-12V) connector. In servers, use the server vendor's GPU cable; in workstations, an adapter from two PCIe 8-pin plugs to one 8-pin EPS. The T4 and L4 are powered entirely through the PCIe slot.

+49 1522 3245373 contact@servertech24.eu For businesses

Stock items leave our warehouse in Germany within 24 hours

GrossNet

DE EN

servertech24

Server GPUs & AI Accelerators – NVIDIA Tesla, A100, L4

Server GPUs & AI Accelerators

Server GPUs and AI accelerators from NVIDIA speed up inference and training of AI models, scientific computing, rendering, virtual desktops and video transcoding. The range goes from the affordable Tesla P40 with 24 GB through the low-power, low-profile T4 and L4 to the A100 with 80 GB HBM2e. Important before you buy: most data center cards are passively cooled and need the directed airflow of a server; workstations need additional cooling. Many models are powered through an 8-pin CPU (EPS-12V) connector rather than PCIe 8-pin, and dual-slot cards need a free full-height, full-length (FHFL) slot. Refurbished cards are cleaned, checked and tested. Warranty is 12 to 36 months.

Prices from: €0.00 gross
Warranty up to: 36 months
Dispatch from stock: within 24 h

Phiarc, CC BY-SA 4.0 · Wikimedia Commons

0 products

GrossNet

No products found

Adjust the filters or ask us directly. We can source most hardware within a few days.

View all products

Buying server GPUs: NVIDIA A100, Tesla V100, L4 and P40 compared

From the NVIDIA A100 with 80 GB of HBM2e to the Tesla P40 as an affordable 24 GB entry point: server GPUs differ less in raw compute than in memory size, bandwidth, form factor and power draw. The overview and the sections below show which card fits which server and what each one is good at.

Every card at a glance

GPU	Generation	Memory	Bandwidth	TDP	Form factor
Tesla P40	Pascal	24 GB GDDR5	346 GB/s	250 W	dual-slot FHFL, passive
Tesla T4	Turing	16 GB GDDR6	320 GB/s	70 W	single-slot low profile, passive
Tesla V100	Volta	16 GB HBM2	900 GB/s	250 W	dual-slot FHFL, passive
L4	Ada Lovelace	24 GB GDDR6	300 GB/s	72 W	single-slot low profile, passive
A40	Ampere	48 GB GDDR6 with ECC	696 GB/s	300 W	dual-slot FHFL, passive
RTX A6000	Ampere	48 GB GDDR6 with ECC

Questions about Server GPUs & AI Accelerators

Answers to the questions we hear most often before a first order.

Question not listed?

Server GPUs & AI Accelerators

0 products

Buying server GPUs: NVIDIA A100, Tesla V100, L4 and P40 compared

Every card at a glance

Questions about Server GPUs & AI Accelerators

NVIDIA A100: when the big card pays off

Tesla V100, P40 and T4: proven and affordable

NVIDIA L4 and RTX A6000: the current alternatives

AI servers for local LLMs: Tesla P40, V100, L4 or A100?