FSDP or DDP
#1
by PhysiQuanty - opened
Hello, I greatly appreciate your work. I see you're using an RTX 4090 from vast.ai. Personally, I use clore.ai, which specializes in peer-to-peer rentals. We can rent 8 x V100 32GB cards for $5 USD per day (4 RTX 4090 for 4.5 usd per day) . With FSDP or DDP from Accelerate , you can increase your number of tokens per second ;)
PhysiQuanty changed discussion title from FSDP or DDP on clore.ai gpu to FSDP or DDP
Could be useful. Right now I’ve got one box mainly for training and another doing scraping/other workload, so cost and stability matter more to me than raw GPU count. What kind of throughput and uptime are you getting?