Update README.md
Browse files
README.md
CHANGED
|
@@ -17,27 +17,10 @@ language:
|
|
| 17 |
|
| 18 |
# DEMO https://youtu.be/CyHbX13AuK0
|
| 19 |
|
| 20 |
-
|
| 21 |
-
> Stop waiting on responses. Add context anytime.
|
| 22 |
-
> Non-blocking, real-time AI.
|
| 23 |
-
|
| 24 |
-
**Finetuned from model:** unsloth/Qwen3-14B-unsloth-bnb-4bit
|
| 25 |
|
| 26 |

|
| 27 |
|
| 28 |
-
## π What Makes This Model Special?
|
| 29 |
-
|
| 30 |
-
This model has been fine-tuned to handle **asynchronous tool execution** β a critical capability for building responsive, real-world AI agents. Unlike traditional function-calling models that assume tools return results immediately, this model understands and properly handles tools that take time to execute.
|
| 31 |
-
|
| 32 |
-
## π Async Tool Call Protocol
|
| 33 |
-
|
| 34 |
-
The model implements a robust async protocol:
|
| 35 |
-
|
| 36 |
-
1. **Tool Call**: The model makes a function/tool call
|
| 37 |
-
2. **ACK (Acknowledgment)**: The tool immediately returns `<tool_ack id="tN"/>` to confirm the request is received
|
| 38 |
-
3. **Processing**: The tool executes asynchronously (could be API calls, database queries, external services)
|
| 39 |
-
4. **RESPONSE**: The tool returns the actual result later
|
| 40 |
-
|
| 41 |
## π‘ Why Async Tools?
|
| 42 |
|
| 43 |
Real-world AI agents often need to:
|
|
@@ -48,7 +31,16 @@ Real-world AI agents often need to:
|
|
| 48 |
- Handle multiple tool calls in parallel
|
| 49 |
- Provide responsive user experiences without blocking
|
| 50 |
|
| 51 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 52 |
|
| 53 |
## π Example Conversation Flow
|
| 54 |
|
|
@@ -108,7 +100,7 @@ https://youtu.be/CyHbX13AuK0
|
|
| 108 |
|
| 109 |
### Gemini
|
| 110 |
|
| 111 |
-
We used Gemini Speech to Text and are currently fine tuning Gemini 2.5 Flash Lite Model.
|
| 112 |
|
| 113 |

|
| 114 |
|
|
@@ -118,7 +110,10 @@ Link to implementation in Pipecat [Pull Request](https://github.com/pipecat-ai/p
|
|
| 118 |
|
| 119 |
## 5. Tell us what you did new during the hackathon
|
| 120 |
|
| 121 |
-
At the hackathon we've
|
|
|
|
|
|
|
|
|
|
| 122 |
|
| 123 |
## π§ Training Details
|
| 124 |
|
|
@@ -128,4 +123,5 @@ This qwen3 model was trained 2x faster with [Unsloth](https://github.com/unsloth
|
|
| 128 |
|
| 129 |
#### 6. Feedback
|
| 130 |
|
| 131 |
-
At the beginning we
|
|
|
|
|
|
| 17 |
|
| 18 |
# DEMO https://youtu.be/CyHbX13AuK0
|
| 19 |
|
| 20 |
+
**Finetuned from model:** unsloth/Qwen3-14B-unsloth-bnb-4bit using [AsyncTool dataset](https://huggingface.co/datasets/qforge/AsyncTool)
|
|
|
|
|
|
|
|
|
|
|
|
|
| 21 |
|
| 22 |

|
| 23 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 24 |
## π‘ Why Async Tools?
|
| 25 |
|
| 26 |
Real-world AI agents often need to:
|
|
|
|
| 31 |
- Handle multiple tool calls in parallel
|
| 32 |
- Provide responsive user experiences without blocking
|
| 33 |
|
| 34 |
+
This model handles **asynchronous tool execution** β a critical capability for building responsive, real-world AI agents. Unlike traditional function-calling models that assume tools return results immediately, this model understands and properly handles tools that take time to execute and return results later during conversation.
|
| 35 |
+
|
| 36 |
+
## π Async Tool Call Protocol
|
| 37 |
+
|
| 38 |
+
The model implements a robust async protocol:
|
| 39 |
+
|
| 40 |
+
1. **Tool Call**: The model makes a function/tool call
|
| 41 |
+
2. **ACK (Acknowledgment)**: The tool immediately returns `<tool_ack id="tN"/>` to confirm the request is received
|
| 42 |
+
3. **Processing**: The tool executes asynchronously (could be API calls, database queries, external services)
|
| 43 |
+
4. **RESPONSE**: The tool returns the actual result later
|
| 44 |
|
| 45 |
## π Example Conversation Flow
|
| 46 |
|
|
|
|
| 100 |
|
| 101 |
### Gemini
|
| 102 |
|
| 103 |
+
We used Gemini Speech to Text and are currently fine tuning Gemini 2.5 Flash Lite Model for the same task, improved latency and accuracy.
|
| 104 |
|
| 105 |

|
| 106 |
|
|
|
|
| 110 |
|
| 111 |
## 5. Tell us what you did new during the hackathon
|
| 112 |
|
| 113 |
+
At the hackathon we've
|
| 114 |
+
- improved [AsyncTool dataset](https://huggingface.co/datasets/qforge/AsyncTool) with more variety to improve quality of responses
|
| 115 |
+
- fine tuned **unsloth/Qwen3-14B-unsloth-bnb-4bit** model using [Google Colab](https://colab.research.google.com/drive/1r6vSiTPODsN20NzdcfV58NWsu-kbcN-_) (for handling Async Tools)
|
| 116 |
+
- Prepared a draft ([Pull Request](https://github.com/pipecat-ai/pipecat/pull/2839)) to Pipecat by adding support for our new model and native behaviour
|
| 117 |
|
| 118 |
## π§ Training Details
|
| 119 |
|
|
|
|
| 123 |
|
| 124 |
#### 6. Feedback
|
| 125 |
|
| 126 |
+
At the beginning we had issues with running fine tuning on Google Vertex-ai (because of outdated documentation).
|
| 127 |
+
Loved the test coverage and dev environment of pipecat.
|