Specifications
Parameters
70B
VRAM (Q4)
38.5 GB
VRAM (FP16)
140 GB
Context Window
16,384
Architecture
transformer
License
Llama-2
Finding cloud alternatives...
Run Code Llama 70B on NVIDIA A100 40GB PCIe
~38.5GB VRAM needed at Q4. NVIDIA A100 40GB PCIe has 40GB — buy hardware or rent cloud.
We may earn a commission from cloud and hardware partners at no extra cost to you.
Find the cheapest GPU that can run Code Llama 70B
Compatibility Lab — check every GPU × quantization combination