|
|
--- |
|
|
license: apache-2.0 |
|
|
language: |
|
|
- en |
|
|
base_model: |
|
|
- Qwen/Qwen3-0.6B |
|
|
library_name: transformers |
|
|
--- |
|
|
# Vex-Amber-Mini 1.1 |
|
|
|
|
|
> ⚡ **World Record Holder: Most Parameter-Efficient Sub-1B Language Model** |
|
|
|
|
|
> Exquisitely fine-tuned from **Qwen3-0.6B** for unparalleled code generation and versatile text processing. |
|
|
|
|
|
--- |
|
|
|
|
|
[](https://huggingface.co/Arioron/Vex-Amber-Mini-1.0) |
|
|
[](https://www.apache.org/licenses/LICENSE-2.0) |
|
|
[](https://huggingface.co/Arioron/Vex-Amber-Mini-1.0#evaluation) |
|
|
[](https://huggingface.co/Arioron/Vex-Amber-Mini-1.0) |
|
|
[](https://github.com/Arioron-International/Vex-Amber-Mini-1.0) |
|
|
|
|
|
--- |
|
|
|
|
|
## Overview |
|
|
|
|
|
**Vex-Amber-Mini 1.1** is a groundbreaking small language model (SLM) that holds the **world record for the most parameter-efficient model with fewer than 1 billion parameters**. Meticulously optimized for code generation and general-purpose text tasks, it delivers exceptional performance within a compact 0.6B parameter framework. |
|
|
|
|
|
- **Base Model:** [Qwen3-0.6B](https://huggingface.co/Qwen/Qwen3-0.6B) |
|
|
- **Fine-tuning:** LoRA with optional full-weight fine-tuning for enhanced adaptability |
|
|
- **Parameter Count:** 0.6B (preserved for maximum efficiency) |
|
|
- **Frameworks:** [Hugging Face Transformers](https://huggingface.co/docs/transformers/index), [PyTorch](https://pytorch.org/) |
|
|
- **Files:** `.safetensors`, `tokenizer.json` |
|
|
|
|
|
--- |
|
|
|
|
|
## Installation |
|
|
|
|
|
To harness the power of **Vex-Amber-Mini 1.1**, install the required dependencies: |
|
|
|
|
|
```bash |
|
|
pip install transformers torch |
|
|
``` |
|
|
|
|
|
Ensure Python 3.8+ and the latest versions of the required libraries for seamless compatibility. |
|
|
|
|
|
--- |
|
|
|
|
|
## Usage Example |
|
|
|
|
|
Experience the model’s elegance with this example of generating a Python function: |
|
|
|
|
|
```python |
|
|
from transformers import AutoModelForCausalLM, AutoTokenizer |
|
|
|
|
|
# Initialize the tokenizer and model |
|
|
tokenizer = AutoTokenizer.from_pretrained("Arioron/Vex-Amber-Mini-1.0") |
|
|
model = AutoModelForCausalLM.from_pretrained("Arioron/Vex-Amber-Mini-1.0") |
|
|
|
|
|
# Craft the input prompt |
|
|
prompt = "Write a Python function to compute Fibonacci numbers:" |
|
|
inputs = tokenizer(prompt, return_tensors="pt") |
|
|
|
|
|
# Generate refined output |
|
|
outputs = model.generate(**inputs, max_length=100, temperature=0.7) |
|
|
|
|
|
# Decode and present the result |
|
|
print(tokenizer.decode(outputs[0], skip_special_tokens=True)) |
|
|
``` |
|
|
|
|
|
--- |
|
|
|
|
|
## Model Details |
|
|
|
|
|
- **Architecture:** Transformer-based, derived from [Qwen3-0.6B](https://huggingface.co/Qwen/Qwen3-0.6B) |
|
|
- **Training Data:** Fine-tuned on a curated dataset optimized for code generation and versatile text tasks |
|
|
- **Performance:** Achieves a HumanEval Pass@1 score of **20.12%**, setting a benchmark for sub-1B models and earning the title of the **most parameter-efficient sub-1B model** |
|
|
- **Use Cases:** Ideal for code generation, text completion, and lightweight NLP applications |
|
|
- **Context Length:** Supports up to 2048 tokens for efficient processing |
|
|
|
|
|
--- |
|
|
|
|
|
## Performance Metrics |
|
|
|
|
|
The following table compares the HumanEval performance of **Vex-Amber-Mini 1.1** against other code generation models. Note that scores for rival models are approximate, as indicated by "~", based on available benchmarks: |
|
|
|
|
|
| Model | Parameters | HumanEval Pass@1 | Notes | |
|
|
|------------------------|------------|------------------|----------------------------------------------------------------| |
|
|
| **Vex-Amber-Mini 1.0** | 0.6B | 20.21% | Compact model optimized for code generation. | |
|
|
| **Code Llama** | 7B | ~24% | Developed by Meta, optimized for code tasks. | |
|
|
| **StarCoder** | 7B | ~25% | Developed by Hugging Face and ServiceNow, fine-tuned for code. | |
|
|
| **CodeGen** | 6B | ~22% | Developed by Salesforce, optimized for code generation. | |
|
|
| **CodeT5** | 3B | ~20% | Developed by Google, fine-tuned for code tasks. | |
|
|
| **PolyCoder** | 12.7B | ~28% | Developed by Berkeley, optimized for code generation. | |
|
|
|
|
|
*Note: The HumanEval Pass@1 score reflects the model's ability to generate correct code solutions on the first attempt. Vex-Amber-Mini 1.0 achieves competitive performance for its size, outperforming larger models in parameter efficiency.* |
|
|
|
|
|
--- |
|
|
|
|
|
## Limitations |
|
|
|
|
|
- **Compact Design:** The 0.6B parameter count, while highly efficient, may limit performance on highly complex tasks compared to larger models. |
|
|
- **Domain-Specific Fine-tuning:** Optimal results may require additional tuning for specialized applications. |
|
|
- **Context Constraints:** Limited to 2048 tokens, which may impact performance in extended context scenarios. |
|
|
|
|
|
--- |
|
|
|
|
|
## License |
|
|
|
|
|
This project is proudly licensed under the [Apache 2.0 License](https://www.apache.org/licenses/LICENSE-2.0), ensuring open and flexible usage. |
|
|
|
|
|
--- |
|
|
|
|
|
## Acknowledgments |
|
|
|
|
|
- Built upon the robust foundation of [Qwen3-0.6B](https://huggingface.co/Qwen/Qwen3-0.6B). |
|
|
- Gratitude to the [Hugging Face](https://huggingface.co/) team for their exceptional Transformers library. |
|
|
- Heartfelt thanks to the open-source community for their invaluable contributions and feedback. |
|
|
|
|
|
--- |
|
|
|
|
|
## Contact |
|
|
|
|
|
For inquiries, collaboration, or to report issues, please visit: |
|
|
- **Hugging Face Repository**: [Arioron/Vex-Amber-Mini-1.0](https://huggingface.co/Arioron/Vex-Amber-Mini-1.0) |
|
|
- **GitHub Repository**: [Vex-Amber-Mini-1.0](https://github.com/Arioron-International/Vex-Amber-Mini-1.0) |
|
|
- **Community Discussion**: Join the conversation on [Hugging Face Discussions](https://huggingface.co/Arioron/Vex-Amber-Mini-1.0/discussions) |
|
|
|
|
|
--- |
|
|
|
|
|
## Contribute |
|
|
|
|
|
We warmly welcome contributions! Please submit pull requests or issues via the [GitHub repository](https://github.com/Arioron-International/Vex-Amber-Mini-1.0) to help refine and elevate **Vex-Amber-Mini 1.1**. |
