RTX 4090 for local LLMs — what fits in 24 GB & the real cost

Rent a RTX 4090 from $0.37/hr ($0 markup, as of 2026-07-02). 308 of the 421 models in our catalog fit its 24 GB — plus the honest own-vs-rent-vs-API math, electricity and amortization included.

Three honest ways to run AI on a RTX 4090

• Own it: buy the card and pay electricity + amortization — local is real money, not "$0".
• Rent it: from $0.37/GPU-hr on your own vendor account (as of 2026-07-02, $0 markup — we never resell compute).
• Skip it: run a comparable open model through your own API key, paying per million tokens (prices as of 2026-06-29).

Rent a RTX 4090 — real vendor rates, $0 markup

• TensorDock: $0.37/GPU-hr — vendor's own price as of 2026-07-02; you pay them directly.
• Vast.ai: $0.40/GPU-hr — vendor's own price as of 2026-07-02; you pay them directly.

What fits in 24 GB

• 308 of the 421 models in the Spanvero catalog fit on one RTX 4090 at a sensible quant (context capped at 16k for the estimate). Most capable first:
• Laguna XS.2 (poolside, 33.4B) — ~24 GB at Q4_K_M; own-key API alternative ≈ $0.15/1M tokens.
• sarvam 30b (sarvamai, 32.2B) — ~22 GB at Q4_K_M; own-key API alternative ≈ $0.36/1M tokens (size estimate).
• llm jp 4 32b a3b thinking (llm-jp, 32.1B) — ~23 GB at Q4_K_M; own-key API alternative ≈ $0.36/1M tokens (size estimate).
• NVIDIA Nemotron 3 Nano 30B A3B BF16 (nvidia, 31.6B) — ~22 GB at Q4_K_M; own-key API alternative ≈ $0.35/1M tokens (size estimate).
• Nemotron Cascade 2 30B A3B (nvidia, 31.6B) — ~22 GB at Q4_K_M; own-key API alternative ≈ $0.35/1M tokens (size estimate).
• Qwen3 30B A3B (Qwen, 30.5B) — ~22 GB at Q4_K_M; own-key API alternative ≈ $0.31/1M tokens.
• Qwen3 Coder 30B A3B Instruct (Qwen, 30.5B) — ~22 GB at Q4_K_M; own-key API alternative ≈ $0.17/1M tokens.
• Qwen3 30B A3B Instruct 2507 (Qwen, 30.5B) — ~22 GB at Q4_K_M; own-key API alternative ≈ $0.12/1M tokens.
• Qwen3 30B A3B Thinking 2507 (Qwen, 30.5B) — ~22 GB at Q4_K_M; own-key API alternative ≈ $0.24/1M tokens.
• Tongyi DeepResearch 30B A3B (Alibaba-NLP, 30.5B) — ~22 GB at Q4_K_M; own-key API alternative ≈ $0.34/1M tokens (size estimate).
• lynx instruct 30b (bineric, 30.5B) — ~22 GB at Q4_K_M; own-key API alternative ≈ $0.34/1M tokens (size estimate).
• North Mini Code 1.0 (CohereLabs, 30.5B) — ~22 GB at Q4_K_M; own-key API alternative ≈ $0.34/1M tokens (size estimate).
• granite 4.1 30b (ibm-granite, 28.9B) — ~24 GB at Q4_K_M; own-key API alternative ≈ $0.33/1M tokens (size estimate).
• Gemma 2 27B Instruct (Google, 27B) — ~22 GB at Q4_K_M; own-key API alternative ≈ $0.65/1M tokens.
• Trinity Mini (arcee-ai, 26.1B) — ~19 GB at Q4_K_M; own-key API alternative ≈ $0.10/1M tokens.

• Laguna XS.2 — full cost breakdown
• sarvam 30b — full cost breakdown
• llm jp 4 32b a3b thinking — full cost breakdown
• NVIDIA Nemotron 3 Nano 30B A3B BF16 — full cost breakdown
• Nemotron Cascade 2 30B A3B — full cost breakdown
• Qwen3 30B A3B — full cost breakdown
• Qwen3 Coder 30B A3B Instruct — full cost breakdown
• Qwen3 30B A3B Instruct 2507 — full cost breakdown
• Qwen3 30B A3B Thinking 2507 — full cost breakdown
• Tongyi DeepResearch 30B A3B — full cost breakdown
• lynx instruct 30b — full cost breakdown
• North Mini Code 1.0 — full cost breakdown
• granite 4.1 30b — full cost breakdown
• Gemma 2 27B Instruct — full cost breakdown
• Trinity Mini — full cost breakdown

Own a RTX 4090 vs rent it — the honest math

• NVIDIA RTX 4090 24GB: street price $2,450.00 (as of 2026-07-02; GPU Poet US price tracker, Jun 2026 — lowest-average new listing $2,452 (discontinued card; launch MSRP was $1,599)) → $2.2374/day amortized over 3 years.
• Electricity ≈ $0.1356/1M tokens at 350 W and $0.19/kWh (EIA Electric Power Monthly Table 5.6.A — U.S. residential average, Apr 2026, as of 2026-07-02), assuming ~135 tok/s (8B-class model at Q4).
• vs renting: owning wins above ~7.4 active hours/day at the cheapest logged rate ($0.37/hr).
• vs API: Llama 3.1 8B Instruct at $0.03/1M (as of 2026-06-29) is cheaper than the electricity alone — owning never breaks even at any volume.

Own a RTX 4090 vs rent it — the honest math

• Used NVIDIA RTX 4090 24GB (used market): street price $2,270.00 (as of 2026-07-02; BestValueGPU eBay used-price tracker, Jun 2026 (~$2,268)) → $2.0731/day amortized over 3 years.
• Electricity ≈ $0.1356/1M tokens at 350 W and $0.19/kWh (EIA Electric Power Monthly Table 5.6.A — U.S. residential average, Apr 2026, as of 2026-07-02), assuming ~135 tok/s (8B-class model at Q4).
• vs renting: owning wins above ~6.8 active hours/day at the cheapest logged rate ($0.37/hr).
• vs API: Llama 3.1 8B Instruct at $0.03/1M (as of 2026-06-29) is cheaper than the electricity alone — owning never breaks even at any volume.

Rent-price history (append-only daily log)

• Vast.ai: $0.40/hr on 2026-06-20 → $0.40/hr on 2026-07-02 (+0.0%). Logged daily; never back-filled.

Keep exploring

• All GPUs — rental rates compared
• Can I run it? Check any model against your hardware
• Free cost calculator — local vs rent vs API
• All models that run on 24 GB of VRAM
• How we stay honest

Open the free Spanvero advisor → · GPU rates as of 2026-07-02. Honest, $0-markup — your own accounts, we never resell compute. © 2026 Cynosure LLC.