The B300 only has 144GB VRAM. Llama 3.1 405B needs ~246.5GB. Time to rent or upgrade.
Need more VRAM?
Rent cloud GPUs with 246.5GB+ VRAM to run Llama 3.1 405B, or upgrade hardware.
This GPU is short by ~102.5GB VRAM for Llama 3.1 405B at Q4.
We may earn a commission from cloud and hardware partners at no extra cost to you.