Stop distributed inference on EOS

Add AGILLM3.5 round 299 inference-slim checkpoint

Reduce distributed inference checkpoint memory

Add KV-cache distributed inference

Add distributed inference harness