Update README.md

README.md

@@ -172,25 +172,28 @@ The following performance benchmarks were conducted with [vLLM](https://docs.vll
       <th></th>
       <th></th>
       <th></th>
+      <th></th>
       <th style="text-align: center;" colspan="2" >Document Visual Question Answering<br>1680W x 2240H<br>64/128</th>
       <th style="text-align: center;" colspan="2" >Visual Reasoning <br>640W x 480H<br>128/128</th>
       <th style="text-align: center;" colspan="2" >Image Captioning<br>480W x 360H<br>0/128</th>
     </tr>
     <tr>
       <th>Hardware</th>
+      <th>Number of GPUs</th>
       <th>Model</th>
       <th>Average Cost Reduction</th>
       <th>Latency (s)</th>
-      <th>QPD</th>
+      <th>Queries Per Dollar</th>
       <th>Latency (s)th>
-      <th>QPD</th>
+      <th>Queries Per Dollar</th>
       <th>Latency (s)</th>
-      <th>QPD</th>
+      <th>Queries Per Dollar</th>
     </tr>
   </thead>
   <tbody style="text-align: center">
     <tr>
-      <td>A100x4</td>
+      <th rowspan="3" valign="top">A100</th>
+      <td>4</td>
       <td>nm-testing/Pixtral-Large-Instruct-2411-hf</td>
       <td></td>
       <td>7.5</td>
@@ -201,7 +204,7 @@ The following performance benchmarks were conducted with [vLLM](https://docs.vll
       <td>79</td>
     </tr>
     <tr>
-      <td>A100x2</td>
+      <td>2</td>
       <td>nm-testing/Pixtral-Large-Instruct-2411-hf-quantized.w8a8</td>
       <td>1.86</td>
       <td>8.1</td>
@@ -212,7 +215,7 @@ The following performance benchmarks were conducted with [vLLM](https://docs.vll
       <td>148</td>
     </tr>
     <tr>
-      <td>A100x2</td>
+      <td>2</td>
       <td>nm-testing/Pixtral-Large-Instruct-2411-hf-quantized.w4a16</td>
       <td>2.52</td>
       <td>6.9</td>
@@ -223,7 +226,8 @@ The following performance benchmarks were conducted with [vLLM](https://docs.vll
       <td>221</td>
     </tr>
     <tr>
-      <td>H100x4</td>
+      <th rowspan="3" valign="top">H100</th>
+      <td>4</td>
       <td>nm-testing/Pixtral-Large-Instruct-2411-hf</td>
       <td></td>
       <td>4.4</td>
@@ -234,7 +238,7 @@ The following performance benchmarks were conducted with [vLLM](https://docs.vll
       <td>79</td>
     </tr>
     <tr>
-      <td>H100x2</td>
+      <td>2</td>
       <td>nm-testing/Pixtral-Large-Instruct-2411-hf-FP8-Dynamic</td>
       <td>1.82</td>
       <td>4.7</td>
@@ -245,7 +249,7 @@ The following performance benchmarks were conducted with [vLLM](https://docs.vll
       <td>145</td>
     </tr>
     <tr>
-      <td>H100x2</td>
+      <td>2</td>
       <td>nm-testing/Pixtral-Large-Instruct-2411-hf-quantized.w4a16</td>
       <td>1.87</td>
       <td>4.7</td>
@@ -258,7 +262,9 @@ The following performance benchmarks were conducted with [vLLM](https://docs.vll
   </tbody>
 </table>
  
+**Use case profiles: Image Size (WxH) / prompt tokens / generation tokens
 
+**QPD: Queries per dollar, based on on-demand cost at [Lambda Labs](https://lambdalabs.com/service/gpu-cloud) (observed on 2/18/2025).
 
 ### Multi-stream asynchronous performance (measured with vLLM version 0.7.2)
 
@@ -277,16 +283,16 @@ The following performance benchmarks were conducted with [vLLM](https://docs.vll
       <th>Model</th>
       <th>Average Cost Reduction</th>
       <th>Maximum throughput (QPS)</th>
-      <th>QPD</th>
+      <th>Queries Per Dollar</th>
       <th>Maximum throughput (QPS)</th>
-      <th>QPD</th>
+      <th>Queries Per Dollar</th>
       <th>Maximum throughput (QPS)</th>
-      <th>QPD</th>
+      <th>Queries Per Dollar</th>
     </tr>
   </thead>
   <tbody style="text-align: center">
    <tr>
-      <td>A100x4</td>
+      <th rowspan="3" valign="top">A100x4</th>
       <td>nm-testing/Pixtral-Large-Instruct-2411-hf</td>
       <td></td>
       <td>0.4</td>
@@ -297,10 +303,9 @@ The following performance benchmarks were conducted with [vLLM](https://docs.vll
       <td>399</td>
     </tr>
     <tr>
-      <td>A100x2</td>
       <td>nm-testing/Pixtral-Large-Instruct-2411-hf-quantized.w8a8</td>
       <td>1.70</td>
-      <td>0.8</td>
+      <td>1.6</td>
       <td>383</td>
       <td>1.1</td>
       <td>571</td>
@@ -308,10 +313,9 @@ The following performance benchmarks were conducted with [vLLM](https://docs.vll
       <td>674</td>
     </tr>
     <tr>
-      <td>A100x2</td>
       <td>nm-testing/Pixtral-Large-Instruct-2411-hf-quantized.w4a16</td>
       <td>1.48</td>
-      <td>0.5</td>
+      <td>1.0</td>
       <td>276</td>
       <td>1.0</td>
       <td>505</td>
@@ -319,7 +323,7 @@ The following performance benchmarks were conducted with [vLLM](https://docs.vll
       <td>680</td>
     </tr>
     <tr>
-      <td>H100x4</td>
+      <<th rowspan="3" valign="top">H100x4</th>
       <td>nm-testing/Pixtral-Large-Instruct-2411-hf</td>
       <td></td>
       <td>1.0</td>
@@ -330,10 +334,9 @@ The following performance benchmarks were conducted with [vLLM](https://docs.vll
       <td>511</td>
     </tr>
     <tr>
-      <td>H100x2</td>
       <td>nm-testing/Pixtral-Large-Instruct-2411-hf-FP8-Dynamic</td>
       <td>1.61</td>
-      <td>1.7</td>
+      <td>3.4</td>
       <td>467</td>
       <td>2.6</td>
       <td>726</td>
@@ -341,10 +344,9 @@ The following performance benchmarks were conducted with [vLLM](https://docs.vll
       <td>908</td>
     </tr>
     <tr>
-      <td>H100x2</td>
       <td>nm-testing/Pixtral-Large-Instruct-2411-hf-quantized.w4a16</td>
       <td>1.33</td>
-      <td>1.4</td>
+      <td>2.8</td>
       <td>393</td>
       <td>2.2</td>
       <td>634</td>
@@ -353,3 +355,9 @@ The following performance benchmarks were conducted with [vLLM](https://docs.vll
     </tr>
   </tbody>
 </table>
+
+**Use case profiles: Image Size (WxH) / prompt tokens / generation tokens
+
+**QPS: Queries per second.
+
+**QPD: Queries per dollar, based on on-demand cost at [Lambda Labs](https://lambdalabs.com/service/gpu-cloud) (observed on 2/18/2025).