aws-neuron
/

optimum-neuron-cache

dacorvo HF Staff commited on 6 days ago

Commit

412a86d

verified ·

1 Parent(s): aaba15e

Update inference-cache-config/trn1/llama3.json

Files changed (1) hide show

inference-cache-config/trn1/llama3.json CHANGED Viewed

@@ -125,7 +125,7 @@
     {
       "batch_size": 1,
       "sequence_length": 16384,
-      "tensor_parallel_size": 2,
       "instance_type" : "trn1"
     },
     {

     {
       "batch_size": 1,
       "sequence_length": 16384,
+      "tensor_parallel_size": 24,
       "instance_type" : "trn1"
     },
     {