| # RunPod Multi-GPU Options for Qwen3.6-35B-A3B Full SFT |
|
|
| **Requirement:** 160GB+ VRAM, 500GB+ system RAM, 560GB+ total memory. |
|
|
| ## Best Single-Pod Options (ranked) |
|
|
| | Config | VRAM | Rate (Community) | Rate (Secure) | Notes | |
| |--------|------|-------------------|----------------|-------| |
| | 4x A100 80GB SXM | 320GB | ~$5.56/hr | ~$6.80/hr | Best fit. VRAM headroom reduces CPU offload. NVLink 600GB/s. Often "Unavailable" -- rotates in/out. | |
| | 2x H100 80GB SXM | 160GB | ~$5.38/hr | ~$6.58/hr | Minimum viable VRAM. NVLink 900GB/s. Better availability in US regions. | |
| | 2x A100 80GB SXM | 160GB | ~$2.78/hr | ~$3.40/hr | Cheapest viable. Tight on VRAM -- heavy CPU offload via DeepSpeed ZeRO-3 required. | |
| | 4x H100 80GB SXM | 320GB | ~$10.76/hr | ~$13.16/hr | Overkill but fast. Rarely unavailable in Secure US. | |
|
|
| System RAM is configurable per pod. Request 502GB+ when creating the pod -- RunPod allows custom RAM allocation up to host limits. SXM variants are mandatory for NVLink (PCIe has no inter-GPU fabric). |
|
|
| ## Instant Clusters (Multi-Node) |
|
|
| RunPod Instant Clusters link 2-8 nodes (up to 64 GPUs) with 800Gbps-3.2Tbps inter-node bandwidth. H100 SXM focused. Pre-configured with distributed PyTorch env vars and Slurm support. Setup takes minutes. |
|
|
| - **2-node cluster (2x 8xH100 SXM):** ~$43/hr but guaranteed availability and 128x80GB = 10.24TB aggregate memory |
| - **Use case:** If single-pod 4xA100 stays unavailable, a 2-node cluster with fewer GPUs per node may be obtainable faster |
|
|
| Docs: https://docs.runpod.io/instant-clusters |
|
|
| ## Grabbing Scarce Pods |
|
|
| RunPod has **no built-in waitlist or availability alerts**. Strategies: |
|
|
| 1. **Poll the API** -- `runpodctl` or REST API to check GPU type availability programmatically; script a notification |
| 2. **Try off-peak** -- US availability best outside 9am-5pm PT weekdays |
| 3. **Secure Cloud > Community Cloud** -- US Secure regions stay available "almost always" for A100/H100; Community is cheaper but host-dependent |
| 4. **Region hop** -- 30+ regions; EU-CENTRAL H100 SXM scarce during business hours, US-TX/US-OR better |
|
|
| ## Recommendation |
|
|
| Start with **2x A100 80GB SXM Secure Cloud** (~$2.78/hr) + DeepSpeed ZeRO-3 offload. If VRAM is too tight, upgrade to **4x A100 SXM** when it appears. Write a 20-line polling script against RunPod API to alert on 4xA100 availability. Instant Clusters are the fallback if single-pod multi-GPU stays scarce. |
|
|
