sinatras
/

Qwen2.5-1.5B-Auto-FunctionCaller

@@ -1,64 +1,75 @@
----
-license: cc-by-nc-nd-4.0
-base_model:
-- Qwen/Qwen2.5-1.5B
-language:
-- en
-- de
-tags:
-- Function_Call
-- Automotive
-- SLM
-- GGUF
----
-# Qwen2.5-1.5B-Auto-FunctionCaller
-## Model Details
-* **Model Name:** Qwen2.5-1.5B-Auto-FunctionCaller
-* **Base Model:** [Qwen/Qwen2.5-1.5B](https://huggingface.co/Qwen/Qwen2.5-1.5B)
-* **Model Type:** Language Model fine-tuned for Function Calling.
-* **Recommended Quantization:** `Qwen2.5-1.5B-Auto-FunctionCaller.Q4_K_M_I.gguf`
-    * This GGUF file using Q4\_K\_M quantization with Importance Matrix is recommended as offering the best balance between performance and computational efficiency (inference speed, memory usage) based on evaluation.
-## Intended Use
-* **Primary Use:** Function calling extraction from natural language queries within an automotive context. The model is designed to identify user intent and extract relevant parameters (arguments/slots) for triggering vehicle functions or infotainment actions.
-* **Research Context:** This model was specifically developed and fine-tuned as part of a research publication investigating the feasibility and performance of Small Language Models (SLMs) for function-calling tasks in resource-constrained automotive environments.
-* **Target Environment:** Embedded systems or edge devices within vehicles where computational resources may be limited.
-* **Out-of-Scope Uses:** General conversational AI, creative writing, tasks outside automotive function calling, safety-critical vehicle control.
-## Performance Metrics
-The following metrics were evaluated on the `Qwen2.5-1.5B-Auto-FunctionCaller.Q4_K_M_I.gguf` model:
-* **Evaluation Setup:**
-    * Total Evaluation Samples: 2074
-* **Performance:**
-    * **Exact Match Accuracy:** 0.8414
-    * **Average Component Accuracy:** 0.9352
-* **Efficiency & Confidence:**
-    * **Throughput:** 10.31 tokens/second
-    * **Latency (Per Token):** 0.097 seconds
-    * **Latency (Per Instruction):** 0.427 seconds
-    * **Average Model Confidence:** 0.9005
-    * **Calibration Error:** 0.0854
-*Note: Latency and throughput figures are hardware-dependent and should be benchmarked on the target deployment environment.*
-## Limitations
-* **Domain Specificity:** Performance is optimized for automotive function calling. Generalization to other domains or complex, non-structured conversations may be limited.
-* **Quantization Impact:** The `Q4_K_M_I` quantization significantly improves efficiency but may result in a slight reduction in accuracy compared to higher-precision versions (e.g., FP16).
-* **Complex Queries:** May struggle with highly nested, ambiguous, or unusually phrased requests not well-represented in the fine-tuning data.
-* **Safety Criticality:** This model is **not** intended or validated for safety-critical vehicle operations (e.g., braking, steering). Use should be restricted to non-critical systems like infotainment and comfort controls.
-* **Bias:** Like any model, performance and fairness depend on the underlying data. Biases present in the fine-tuning or evaluation datasets may be reflected in the model's behavior.
-## Training Data (Summary)
-The model was fine-tuned on a synthetic dataset specifically curated for automotive function calling tasks. Details will be referenced in the associated publication.
-## Citation
 TBD

+---
+license: cc-by-nc-nd-4.0
+base_model:
+- Qwen/Qwen2.5-1.5B
+language:
+- zho
+- eng
+- fra
+- spa
+- por
+- deu
+- ita
+- rus
+- jpn
+- kor
+- vie
+- tha
+- ara
+tags:
+- Function_Call
+- Automotive
+- SLM
+- GGUF
+---
+# Qwen2.5-1.5B-Auto-FunctionCaller
+## Model Details
+* **Model Name:** Qwen2.5-1.5B-Auto-FunctionCaller
+* **Base Model:** [Qwen/Qwen2.5-1.5B](https://huggingface.co/Qwen/Qwen2.5-1.5B)
+* **Model Type:** Language Model fine-tuned for Function Calling.
+* **Recommended Quantization:** `Qwen2.5-1.5B-Auto-FunctionCaller.Q4_K_M_I.gguf`
+    * This GGUF file using Q4\_K\_M quantization with Importance Matrix is recommended as offering the best balance between performance and computational efficiency (inference speed, memory usage) based on evaluation.
+## Intended Use
+* **Primary Use:** Function calling extraction from natural language queries within an automotive context. The model is designed to identify user intent and extract relevant parameters (arguments/slots) for triggering vehicle functions or infotainment actions.
+* **Research Context:** This model was specifically developed and fine-tuned as part of a research publication investigating the feasibility and performance of Small Language Models (SLMs) for function-calling tasks in resource-constrained automotive environments.
+* **Target Environment:** Embedded systems or edge devices within vehicles where computational resources may be limited.
+* **Out-of-Scope Uses:** General conversational AI, creative writing, tasks outside automotive function calling, safety-critical vehicle control.
+## Performance Metrics
+The following metrics were evaluated on the `Qwen2.5-1.5B-Auto-FunctionCaller.Q4_K_M_I.gguf` model:
+* **Evaluation Setup:**
+    * Total Evaluation Samples: 2074
+* **Performance:**
+    * **Exact Match Accuracy:** 0.8414
+    * **Average Component Accuracy:** 0.9352
+* **Efficiency & Confidence:**
+    * **Throughput:** 10.31 tokens/second
+    * **Latency (Per Token):** 0.097 seconds
+    * **Latency (Per Instruction):** 0.427 seconds
+    * **Average Model Confidence:** 0.9005
+    * **Calibration Error:** 0.0854
+*Note: Latency and throughput figures are hardware-dependent and should be benchmarked on the target deployment environment.*
+## Limitations
+* **Domain Specificity:** Performance is optimized for automotive function calling. Generalization to other domains or complex, non-structured conversations may be limited.
+* **Quantization Impact:** The `Q4_K_M_I` quantization significantly improves efficiency but may result in a slight reduction in accuracy compared to higher-precision versions (e.g., FP16).
+* **Complex Queries:** May struggle with highly nested, ambiguous, or unusually phrased requests not well-represented in the fine-tuning data.
+* **Safety Criticality:** This model is **not** intended or validated for safety-critical vehicle operations (e.g., braking, steering). Use should be restricted to non-critical systems like infotainment and comfort controls.
+* **Bias:** Like any model, performance and fairness depend on the underlying data. Biases present in the fine-tuning or evaluation datasets may be reflected in the model's behavior.
+## Training Data (Summary)
+The model was fine-tuned on a synthetic dataset specifically curated for automotive function calling tasks. Details will be referenced in the associated publication.
+## Citation
 TBD