Spaces:
Running
Running
Update README.md
Browse files
README.md
CHANGED
|
@@ -13,7 +13,7 @@ pinned: false
|
|
| 13 |
Welcome to CARROT-LLM-Routing! For a given desired trade off between performance and cost,
|
| 14 |
CARROT makes it easy to pick the best model among a set of 13 LLMs for any query. Below you may read the CARROT paper, replicate the training process of CARROT, or see how to utilize CARROT out of the box for routing.
|
| 15 |
</p>
|
| 16 |
-
<a href="https://arxiv.org/" class="block overflow-hidden group">
|
| 17 |
<div
|
| 18 |
class="w-40 h-39 object-cover mb-2 rounded-lg flex items-center justify-center bg-[#ECFAFF]"
|
| 19 |
>
|
|
@@ -22,7 +22,7 @@ pinned: false
|
|
| 22 |
<div class="underline">Read the paper</div>
|
| 23 |
</a>
|
| 24 |
<a
|
| 25 |
-
href="https://github.com/somerstep"
|
| 26 |
class="block overflow-hidden"
|
| 27 |
>
|
| 28 |
<div
|
|
@@ -30,11 +30,11 @@ pinned: false
|
|
| 30 |
>
|
| 31 |
<img alt="" src="logo.png" class="w-40" />
|
| 32 |
</div>
|
| 33 |
-
<div class="underline">
|
| 34 |
</a>
|
| 35 |
|
| 36 |
<p class="lg:col-span-3">
|
| 37 |
-
As is, CARROT supports routing to the following collection of large language models. Instantiating the CarrotRouter class automatically loads the trained predictors for ouput token count and performance that are provided below.
|
| 38 |
|
| 39 |
| | claude-3-5-sonnet-v1 | titan-text-premier-v1 | openai-gpt-4o | openai-gpt-4o-mini | granite-3-2b-instruct | granite-3-8b-instruct | llama-3-1-70b-instruct | llama-3-1-8b-instruct | llama-3-2-1b-instruct | llama-3-2-3b-instruct | llama-3-3-70b-instruct | mixtral-8x7b-instruct | llama-3-405b-instruct |
|
| 40 |
|----------------------|---------------------|----------------------|---------------|--------------------|----------------------|----------------------|----------------------|----------------------|----------------------|----------------------|----------------------|----------------------|----------------------|
|
|
|
|
| 13 |
Welcome to CARROT-LLM-Routing! For a given desired trade off between performance and cost,
|
| 14 |
CARROT makes it easy to pick the best model among a set of 13 LLMs for any query. Below you may read the CARROT paper, replicate the training process of CARROT, or see how to utilize CARROT out of the box for routing.
|
| 15 |
</p>
|
| 16 |
+
<a href="https://arxiv.org/abs/2502.03261" class="block overflow-hidden group">
|
| 17 |
<div
|
| 18 |
class="w-40 h-39 object-cover mb-2 rounded-lg flex items-center justify-center bg-[#ECFAFF]"
|
| 19 |
>
|
|
|
|
| 22 |
<div class="underline">Read the paper</div>
|
| 23 |
</a>
|
| 24 |
<a
|
| 25 |
+
href="https://github.com/somerstep/CARROT"
|
| 26 |
class="block overflow-hidden"
|
| 27 |
>
|
| 28 |
<div
|
|
|
|
| 30 |
>
|
| 31 |
<img alt="" src="logo.png" class="w-40" />
|
| 32 |
</div>
|
| 33 |
+
<div class="underline">Access code for CARROT</div>
|
| 34 |
</a>
|
| 35 |
|
| 36 |
<p class="lg:col-span-3">
|
| 37 |
+
As is, CARROT supports routing to the following collection of large language models. Instantiating the CarrotRouter class automatically loads the trained predictors for ouput token count and performance that are provided below. Note that you ust provide a hugging face token with access to the Llama-3 herd of models. mu takes a value between 0 and 1, this controls the cost performance trade off. A smaller mu will prioritize perofrmance!
|
| 38 |
|
| 39 |
| | claude-3-5-sonnet-v1 | titan-text-premier-v1 | openai-gpt-4o | openai-gpt-4o-mini | granite-3-2b-instruct | granite-3-8b-instruct | llama-3-1-70b-instruct | llama-3-1-8b-instruct | llama-3-2-1b-instruct | llama-3-2-3b-instruct | llama-3-3-70b-instruct | mixtral-8x7b-instruct | llama-3-405b-instruct |
|
| 40 |
|----------------------|---------------------|----------------------|---------------|--------------------|----------------------|----------------------|----------------------|----------------------|----------------------|----------------------|----------------------|----------------------|----------------------|
|