Add pipeline tag and library name
#2
by
nielsr
HF Staff
- opened
README.md
CHANGED
|
@@ -1,15 +1,15 @@
|
|
| 1 |
---
|
| 2 |
-
|
|
|
|
| 3 |
language:
|
| 4 |
- en
|
|
|
|
| 5 |
metrics:
|
| 6 |
- accuracy
|
| 7 |
-
|
| 8 |
-
|
| 9 |
---
|
| 10 |
|
| 11 |
-
|
| 12 |
-
|
| 13 |
# CoALM-8B: Conversational Agentic Language Model
|
| 14 |
|
| 15 |
[](https://github.com/oumi-ai/oumi)
|
|
@@ -51,31 +51,31 @@ CoALM-8B is trained on a **multi-task dataset** covering dialogue state tracking
|
|
| 51 |
## Capabilities and Features
|
| 52 |
|
| 53 |
### π£ Conversational Agentic Abilities
|
| 54 |
-
- **Multi-turn Dialogue Mastery:** Maintains coherent conversations across multiple turns with accurate state tracking
|
| 55 |
-
- **Function Calling and API Integration:** Dynamically selects and calls APIs for task execution
|
| 56 |
-
- **ReAct-based Reasoning:** Utilizes a structured reasoning process (User-Thought-Action-Observation-Thought-Response)
|
| 57 |
- **Zero-Shot Generalization:** Excels in previously unseen function-calling tasks.
|
| 58 |
|
| 59 |
### π Benchmark Performance
|
| 60 |
-
- **MultiWOZ 2.4 (TOD):** Excels in dialogue state tracking and task completion
|
| 61 |
-
- **BFCL V3 (LA):** Demonstrates superior function-calling abilities over language agents
|
| 62 |
-
- **API-Bank (LA):** Accurately generates API calls and integrates responses into conversation flow
|
| 63 |
|
| 64 |
---
|
| 65 |
## Training Process
|
| 66 |
### π§ Fine-tuning Stages
|
| 67 |
-
1. **TOD Fine-tuning:** Optimized for dialogue state tracking (e.g., augmented SNIPS reformatted in Alpaca-style instruction tuning)
|
| 68 |
-
2. **Function Calling Fine-tuning:** Trained to select and generate well-formed API calls from LA datasets
|
| 69 |
-
3. **ReAct-based Fine-tuning:** Addresses multi-turn conversations with API integration using a structured reasoning framework
|
| 70 |
|
| 71 |
### π Training Hyperparameters
|
| 72 |
-
- **Base Model:** Llama 3.1 8B Instruct
|
| 73 |
-
- **LoRA Config:** Rank = 16, Scaling Factor = 32
|
| 74 |
-
- **Batch Size:** 8
|
| 75 |
-
- **Learning Rate:** 1e-4
|
| 76 |
-
- **Optimizer:** AdamW (betas = 0.9, 0.999, epsilon = 1e-8)
|
| 77 |
-
- **Precision:** Mixed precision (bfloat16)
|
| 78 |
-
- **Warm-up Steps:** 0.1 ratio of total steps
|
| 79 |
- **Gradient Accumulation Steps:** 1
|
| 80 |
|
| 81 |
---
|
|
@@ -116,8 +116,8 @@ oumi train -c ./oumi_train.yaml
|
|
| 116 |
```
|
| 117 |
|
| 118 |
---
|
| 119 |
-
- **Task-Specific Calibration:** While CoALM-8B generalizes well across tasks, performance can improve with domain-specific fine-tuning
|
| 120 |
-
- **Scalability to Larger Models:** Future iterations (CoALM-70B, CoALM-405B) extend capabilities to larger-scale agentic conversations
|
| 121 |
- **Open-Source Expansion:** All datasets, training scripts, and model checkpoints are publicly available to foster further research.
|
| 122 |
|
| 123 |
## Acknowledgements
|
|
@@ -142,5 +142,4 @@ If you use **CoALM-8B** in your research, please cite:
|
|
| 142 |
}
|
| 143 |
```
|
| 144 |
|
| 145 |
-
For more details, visit [Project Repository](https://github.com/oumi-ai/oumi/tree/main/configs/projects/calm) or contact **acikgoz2@illinois.edu**.
|
| 146 |
-
|
|
|
|
| 1 |
---
|
| 2 |
+
base_model:
|
| 3 |
+
- meta-llama/Llama-3.1-8B-Instruct
|
| 4 |
language:
|
| 5 |
- en
|
| 6 |
+
license: cc-by-nc-4.0
|
| 7 |
metrics:
|
| 8 |
- accuracy
|
| 9 |
+
pipeline_tag: text-generation
|
| 10 |
+
library_name: transformers
|
| 11 |
---
|
| 12 |
|
|
|
|
|
|
|
| 13 |
# CoALM-8B: Conversational Agentic Language Model
|
| 14 |
|
| 15 |
[](https://github.com/oumi-ai/oumi)
|
|
|
|
| 51 |
## Capabilities and Features
|
| 52 |
|
| 53 |
### π£ Conversational Agentic Abilities
|
| 54 |
+
- **Multi-turn Dialogue Mastery:** Maintains coherent conversations across multiple turns with accurate state tracking.\
|
| 55 |
+
- **Function Calling and API Integration:** Dynamically selects and calls APIs for task execution.\
|
| 56 |
+
- **ReAct-based Reasoning:** Utilizes a structured reasoning process (User-Thought-Action-Observation-Thought-Response).\
|
| 57 |
- **Zero-Shot Generalization:** Excels in previously unseen function-calling tasks.
|
| 58 |
|
| 59 |
### π Benchmark Performance
|
| 60 |
+
- **MultiWOZ 2.4 (TOD):** Excels in dialogue state tracking and task completion.\
|
| 61 |
+
- **BFCL V3 (LA):** Demonstrates superior function-calling abilities over language agents.\
|
| 62 |
+
- **API-Bank (LA):** Accurately generates API calls and integrates responses into conversation flow.\
|
| 63 |
|
| 64 |
---
|
| 65 |
## Training Process
|
| 66 |
### π§ Fine-tuning Stages
|
| 67 |
+
1. **TOD Fine-tuning:** Optimized for dialogue state tracking (e.g., augmented SNIPS reformatted in Alpaca-style instruction tuning).\
|
| 68 |
+
2. **Function Calling Fine-tuning:** Trained to select and generate well-formed API calls from LA datasets.\
|
| 69 |
+
3. **ReAct-based Fine-tuning:** Addresses multi-turn conversations with API integration using a structured reasoning framework.\
|
| 70 |
|
| 71 |
### π Training Hyperparameters
|
| 72 |
+
- **Base Model:** Llama 3.1 8B Instruct\
|
| 73 |
+
- **LoRA Config:** Rank = 16, Scaling Factor = 32\
|
| 74 |
+
- **Batch Size:** 8\
|
| 75 |
+
- **Learning Rate:** 1e-4\
|
| 76 |
+
- **Optimizer:** AdamW (betas = 0.9, 0.999, epsilon = 1e-8)\
|
| 77 |
+
- **Precision:** Mixed precision (bfloat16)\
|
| 78 |
+
- **Warm-up Steps:** 0.1 ratio of total steps\
|
| 79 |
- **Gradient Accumulation Steps:** 1
|
| 80 |
|
| 81 |
---
|
|
|
|
| 116 |
```
|
| 117 |
|
| 118 |
---
|
| 119 |
+
- **Task-Specific Calibration:** While CoALM-8B generalizes well across tasks, performance can improve with domain-specific fine-tuning.\
|
| 120 |
+
- **Scalability to Larger Models:** Future iterations (CoALM-70B, CoALM-405B) extend capabilities to larger-scale agentic conversations.\
|
| 121 |
- **Open-Source Expansion:** All datasets, training scripts, and model checkpoints are publicly available to foster further research.
|
| 122 |
|
| 123 |
## Acknowledgements
|
|
|
|
| 142 |
}
|
| 143 |
```
|
| 144 |
|
| 145 |
+
For more details, visit [Project Repository](https://github.com/oumi-ai/oumi/tree/main/configs/projects/calm) or contact **acikgoz2@illinois.edu**.
|
|
|