RTX 5090 for local LLMs — what fits in 32 GB & the real cost

Rent a RTX 5090 from $0.56/hr ($0 markup, as of 2026-07-02). 346 of the 445 models in our catalog fit its 32 GB — plus the honest own-vs-rent-vs-API math, electricity and amortization included.

Three honest ways to run AI on a RTX 5090

• Own it: buy the card and pay electricity + amortization — local is real money, not "$0".
• Rent it: from $0.56/GPU-hr on your own vendor account (as of 2026-07-02, $0 markup — we never resell compute).
• Skip it: run a comparable open model through your own API key, paying per million tokens (prices as of 2026-07-06).

Rent a RTX 5090 — real vendor rates, $0 markup

• Vast.ai: $0.56/GPU-hr — vendor's own price as of 2026-07-02; you pay them directly.

What fits in 32 GB

• 346 of the 445 models in the Spanvero catalog fit on one RTX 5090 at a sensible quant (context capped at 16k for the estimate). Most capable first:
• Phi 3.5 MoE instruct (microsoft, 41.9B) — ~31 GB at Q4_K_M; own-key API alternative ≈ $0.44/1M tokens (size estimate).
• Karnak 40B v1.0 (Applied-Innovation-Center, 40.7B) — ~29 GB at Q4_K_M; own-key API alternative ≈ $0.43/1M tokens (size estimate).
• Seed OSS 36B Instruct (ByteDance-Seed, 36.2B) — ~27 GB at Q4_K_M; own-key API alternative ≈ $0.39/1M tokens (size estimate).
• Hermes 4.3 36B (NousResearch, 36.2B) — ~27 GB at Q4_K_M; own-key API alternative ≈ $0.39/1M tokens (size estimate).
• Yi-1.5-34B-Chat (01.AI, 34.4B) — ~25 GB at Q4_K_M; own-key API alternative ≈ $0.38/1M tokens (size estimate).
• Laguna XS.2 (poolside, 33.4B) — ~24 GB at Q4_K_M; own-key API alternative ≈ $0.15/1M tokens.
• Qwen3-32B (Alibaba, 32.8B) — ~25 GB at Q4_K_M; own-key API alternative ≈ $0.18/1M tokens.
• Qwen2.5 32B Instruct (Qwen, 32.8B) — ~27 GB at Q4_K_M; own-key API alternative ≈ $0.36/1M tokens (size estimate).
• cogito v1 preview qwen 32B (deepcogito, 32.8B) — ~27 GB at Q4_K_M; own-key API alternative ≈ $0.36/1M tokens (size estimate).
• QwQ 32B (Qwen, 32.8B) — ~27 GB at Q4_K_M; own-key API alternative ≈ $0.36/1M tokens (size estimate).
• DeepSeek-R1-Distill-Qwen-32B (DeepSeek, 32.5B) — ~27 GB at Q4_K_M; own-key API alternative ≈ $0.36/1M tokens (size estimate).
• granite 4.0 h small (ibm-granite, 32.2B) — ~25 GB at Q4_K_M; own-key API alternative ≈ $0.36/1M tokens (size estimate).
• sarvam 30b (sarvamai, 32.2B) — ~22 GB at Q4_K_M; own-key API alternative ≈ $0.36/1M tokens (size estimate).
• llm jp 4 32b a3b thinking (llm-jp, 32.1B) — ~23 GB at Q4_K_M; own-key API alternative ≈ $0.36/1M tokens (size estimate).
• Qwen2.5-Coder 32B Instruct (Alibaba, 32B) — ~25 GB at Q4_K_M; own-key API alternative ≈ $0.83/1M tokens.

• Phi 3.5 MoE instruct — full cost breakdown
• Karnak 40B v1.0 — full cost breakdown
• Seed OSS 36B Instruct — full cost breakdown
• Hermes 4.3 36B — full cost breakdown
• Yi-1.5-34B-Chat — full cost breakdown
• Laguna XS.2 — full cost breakdown
• Qwen3-32B — full cost breakdown
• Qwen2.5 32B Instruct — full cost breakdown
• cogito v1 preview qwen 32B — full cost breakdown
• QwQ 32B — full cost breakdown
• DeepSeek-R1-Distill-Qwen-32B — full cost breakdown
• granite 4.0 h small — full cost breakdown
• sarvam 30b — full cost breakdown
• llm jp 4 32b a3b thinking — full cost breakdown
• Qwen2.5-Coder 32B Instruct — full cost breakdown

Own a RTX 5090 vs rent it — the honest math

• NVIDIA RTX 5090 32GB: street price $3,680.00 (as of 2026-07-02; GPU Poet US price tracker, Jun 2026 — lowest-average listing $3,680 (launch MSRP $1,999; GDDR7 supply crunch)) → $3.3607/day amortized over 3 years.
• Electricity ≈ $0.2118/1M tokens at 575 W and $0.19/kWh (EIA Electric Power Monthly Table 5.6.A — U.S. residential average, Apr 2026, as of 2026-07-02), assuming ~142 tok/s (8B-class model at Q4).
• vs renting: owning wins above ~7.4 active hours/day at the cheapest logged rate ($0.56/hr).
• vs API: Llama 3.1 8B Instruct at $0.03/1M (as of 2026-07-06) is cheaper than the electricity alone — owning never breaks even at any volume.

Own a RTX 5090 vs rent it — the honest math

• Used NVIDIA RTX 5090 32GB (used market): street price $3,700.00 (as of 2026-07-06; videocardprices.com eBay used tracker, retrieved 2026-07-06 (~$3,700; used trades near new because of the GDDR7 supply crunch)) → $3.379/day amortized over 3 years.
• Electricity ≈ $0.2118/1M tokens at 575 W and $0.19/kWh (EIA Electric Power Monthly Table 5.6.A — U.S. residential average, Apr 2026, as of 2026-07-02), assuming ~142 tok/s (8B-class model at Q4).
• vs renting: owning wins above ~7.5 active hours/day at the cheapest logged rate ($0.56/hr).
• vs API: Llama 3.1 8B Instruct at $0.03/1M (as of 2026-07-06) is cheaper than the electricity alone — owning never breaks even at any volume.

Keep exploring

• Can a RTX 5090 run it? Check any model
• All GPUs — rental rates compared
• Can I run it? Check any model against your hardware
• Free cost calculator — local vs rent vs API
• How we stay honest

Building a local rig? Hardware that matches these numbers

• 1000 W+ 80+ Gold PSU (ATX 3.0) — Shop on Amazon
• Electricity usage monitor (watt meter) — Shop on Amazon

As an Amazon Associate we earn from qualifying purchases. Each link opens an Amazon search for the category — not an endorsement of any specific product — and Cynosure LLC may earn a commission at no extra cost to you.

Open the free Spanvero advisor → · GPU rates as of 2026-07-02. Honest, $0-markup — your own accounts, we never resell compute. © 2026 Cynosure LLC.