Inference Calculator

Estimate tokens/second, latency, and cost for LLM inference on any GPU. Compare self-hosted vs API pricing with break-even analysis.

Limited data: tok/s and cost are order-of-magnitude heuristics from GPU bandwidth and listed hourly rates, not measured benches.

Model Parameters (B)

Context Length

Requests/Minute

GPU Type

VRAM Required

3.9of 24GB (16%)

Throughput

114tokens/sec

Cost per 1K Tokens

N/ANo cloud price available

Monthly Cost

N/ANo cloud price available

Performance Metrics

9ms

Latency per token

122.9K

Tokens/minute

86,400

Daily requests

Recommendations

Good Fit

Model fits well in GPU memory with room for batching.

Normal Load

Current load is manageable for single GPU deployment.

Cost Analysis

Tokens per Hour411,429

Cloud pricing not available for Select a GPU. Check the Cloud Compute Tracker for live rental prices.

HardwareHQ

Inference Calculator

Estimate tokens/second, latency, and cost for LLM inference on any GPU. Compare self-hosted vs API pricing with break-even analysis.

Limited data: tok/s and cost are order-of-magnitude heuristics from GPU bandwidth and listed hourly rates, not measured benches.

Model Parameters (B)

Context Length

Requests/Minute

GPU Type

VRAM Required

3.9of 24GB (16%)

Throughput

114tokens/sec

Cost per 1K Tokens

N/ANo cloud price available

Monthly Cost

N/ANo cloud price available

Performance Metrics

9ms

Latency per token

122.9K

Tokens/minute

86,400

Daily requests

Recommendations

Good Fit

Model fits well in GPU memory with room for batching.

Normal Load

Current load is manageable for single GPU deployment.

Cost Analysis

Tokens per Hour411,429

Cloud pricing not available for Select a GPU. Check the Cloud Compute Tracker for live rental prices.

Also

Live cloud pricesInstance tape from the live feed GPU catalogSpecs for the unique catalog Model catalogParams and estimated VRAM GPU comparisonSide-by-side catalog specs