A100 40GB for local LLMs — what fits in 40 GB & the real cost

Rent a A100 40GB from $1.29/hr ($0 markup, as of 2026-07-02). 339 of the 421 models in our catalog fit its 40 GB — plus the honest own-vs-rent-vs-API math, electricity and amortization included.

Three honest ways to run AI on a A100 40GB

• Own it: buy the card and pay electricity + amortization — local is real money, not "$0".
• Rent it: from $1.29/GPU-hr on your own vendor account (as of 2026-07-02, $0 markup — we never resell compute).
• Skip it: run a comparable open model through your own API key, paying per million tokens (prices as of 2026-06-29).

Rent a A100 40GB — real vendor rates, $0 markup

• Verda: $1.29/GPU-hr — vendor's own price as of 2026-07-02; you pay them directly.
• Lambda: $1.99/GPU-hr — vendor's own price as of 2026-07-02; you pay them directly.

What fits in 40 GB

• 339 of the 421 models in the Spanvero catalog fit on one A100 40GB at a sensible quant (context capped at 16k for the estimate). Most capable first:
• Kimi Linear 48B A3B Instruct (moonshotai, 49.1B) — ~36 GB at Q4_K_M; own-key API alternative ≈ $0.49/1M tokens (size estimate).
• Mixtral 8x7B Instruct (Mistral AI, 46.7B) — ~34 GB at Q4_K_M; own-key API alternative ≈ $0.24/1M tokens (last-known).
• Phi 3.5 MoE instruct (microsoft, 41.9B) — ~31 GB at Q4_K_M; own-key API alternative ≈ $0.44/1M tokens (size estimate).
• Karnak 40B v1.0 (Applied-Innovation-Center, 40.7B) — ~29 GB at Q4_K_M; own-key API alternative ≈ $0.43/1M tokens (size estimate).
• Seed OSS 36B Instruct (ByteDance-Seed, 36.2B) — ~27 GB at Q4_K_M; own-key API alternative ≈ $0.39/1M tokens (size estimate).
• Hermes 4.3 36B (NousResearch, 36.2B) — ~27 GB at Q4_K_M; own-key API alternative ≈ $0.39/1M tokens (size estimate).
• Qwen AgentWorld 35B A3B (Qwen, 34.7B) — ~40 GB at Q4_K_M; own-key API alternative ≈ $0.38/1M tokens (size estimate).
• Yi-1.5-34B-Chat (01.AI, 34.4B) — ~25 GB at Q4_K_M; own-key API alternative ≈ $0.38/1M tokens (size estimate).
• Laguna XS.2 (poolside, 33.4B) — ~24 GB at Q4_K_M; own-key API alternative ≈ $0.15/1M tokens.
• HyperCLOVAX SEED Think 32B (naver-hyperclovax, 33.3B) — ~39 GB at Q4_K_M; own-key API alternative ≈ $0.37/1M tokens (size estimate).
• Qwen3-32B (Alibaba, 32.8B) — ~25 GB at Q4_K_M; own-key API alternative ≈ $0.18/1M tokens.
• Qwen2.5 32B Instruct (Qwen, 32.8B) — ~27 GB at Q4_K_M; own-key API alternative ≈ $0.36/1M tokens (size estimate).
• cogito v1 preview qwen 32B (deepcogito, 32.8B) — ~27 GB at Q4_K_M; own-key API alternative ≈ $0.36/1M tokens (size estimate).
• QwQ 32B (Qwen, 32.8B) — ~27 GB at Q4_K_M; own-key API alternative ≈ $0.36/1M tokens (size estimate).
• DeepSeek-R1-Distill-Qwen-32B (DeepSeek, 32.5B) — ~27 GB at Q4_K_M; own-key API alternative ≈ $0.36/1M tokens (size estimate).

• Kimi Linear 48B A3B Instruct — full cost breakdown
• Mixtral 8x7B Instruct — full cost breakdown
• Phi 3.5 MoE instruct — full cost breakdown
• Karnak 40B v1.0 — full cost breakdown
• Seed OSS 36B Instruct — full cost breakdown
• Hermes 4.3 36B — full cost breakdown
• Qwen AgentWorld 35B A3B — full cost breakdown
• Yi-1.5-34B-Chat — full cost breakdown
• Laguna XS.2 — full cost breakdown
• HyperCLOVAX SEED Think 32B — full cost breakdown
• Qwen3-32B — full cost breakdown
• Qwen2.5 32B Instruct — full cost breakdown
• cogito v1 preview qwen 32B — full cost breakdown
• QwQ 32B — full cost breakdown
• DeepSeek-R1-Distill-Qwen-32B — full cost breakdown

Why no own-it math for the A100 40GB?

• We only publish own-it economics where we have a sourced, dated street price — none for this card yet, and we don't invent numbers.

Rent-price history (append-only daily log)

• Lambda: $1.10/hr on 2026-06-20 → $1.10/hr on 2026-07-02 (+0.0%). Logged daily; never back-filled.

Keep exploring

• All GPUs — rental rates compared
• Can I run it? Check any model against your hardware
• Free cost calculator — local vs rent vs API
• How we stay honest

Open the free Spanvero advisor → · GPU rates as of 2026-07-02. Honest, $0-markup — your own accounts, we never resell compute. © 2026 Cynosure LLC.