The RX 470 8GB only has 8GB VRAM. Llama 3.1 70B needs ~42.6GB (estimate). Time to rent or upgrade.
Estimate · Q4_K_M · 4k context · batch 1. For dense 7B–70B this sits ~5–8% above the Q4_K_M GGUF file. Not peak runtime VRAM and not a measured bench.
Need more VRAM?
Rent cloud GPUs with 42.6GB+ VRAM to run Llama 3.1 70B, or upgrade hardware.
This GPU is short by ~34.6GB VRAM for Llama 3.1 70B at Q4.
We may earn a commission from cloud and hardware partners at no extra cost to you.
MSRP: $179