|
|
--- |
|
|
license: apache-2.0 |
|
|
datasets: |
|
|
- Magpie-Align/Magpie-Pro-300K-Filtered |
|
|
language: |
|
|
- de |
|
|
- en |
|
|
base_model: |
|
|
- unsloth/Llama-3.2-3B-bnb-4bit |
|
|
pipeline_tag: text-generation |
|
|
library_name: adapter-transformers |
|
|
tags: |
|
|
- efficient |
|
|
- llama |
|
|
- llama3 |
|
|
- gguf |
|
|
- ollama |
|
|
- instruction-finetuning |
|
|
--- |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
<p align="center"> |
|
|
<img src="https://cdn-uploads.huggingface.co/production/uploads/6921fa6332f7fb129563d495/jemMkfi73Fck611ID18ts.png" width="128"> |
|
|
|
|
|
</p> |
|
|
|
|
|
# Nomi 1.0-3b |
|
|
|
|
|
## Introduction |
|
|
|
|
|
Nomi-1.0 is a **refined mid-range Large Language Model** based on the **Llama-3.2-3B** architecture. It was specifically developed to outperform standard 3B models in **structured reporting**, |
|
|
**markdown formatting**, and **Python coding**, making it an ideal assistant for local deployment on consumer hardware. |
|
|
|
|
|
**It is the first Model of the Nomi-Series** |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
## 🌟 Key Features & Improvements |
|
|
|
|
|
* **Architecture:** Llama-3.2-3B (Optimized for 8GB VRAM GPUs like RTX 4060). |
|
|
* **Formatting Master:** Specifically trained to use H1, H2, tables, and bold text to make information instantly scannable. |
|
|
* **Coding Proficiency:** Fine-tuned on the Magpie-Pro dataset to write cleaner Python code with built-in error handling (`try-except`). |
|
|
* **Multilingual Support:** Excellent performance in both German and English. |
|
|
* **Efficiency:** High-speed inference (~60+ tokens/sec) with a very low memory footprint. |
|
|
|
|
|
|
|
|
|
|
|
--- |
|
|
|
|
|
## 🧠 Training Details |
|
|
|
|
|
The goal of Nomi-1.0 was to create a "bridge" model that feels as smart as a 7B model but runs with the speed of a 3B model. |
|
|
|
|
|
* **Base Model:** `unsloth/Llama-3.2-3B-Instruct-bnb-4bit` |
|
|
* **Fine-tuning:** SFT (Supervised Fine-Tuning) using the **Magpie-Pro** dataset. |
|
|
* **Training Tool:** **Unsloth** (for 4-bit optimized training). |
|
|
* **Optimization:** High LoRA Rank (r=32) was used to ensure the model captures complex structural nuances. |
|
|
|
|
|
--- |
|
|
|
|
|
## 📝 Prompt Template (ChatML/Llama-3.2) |
|
|
|
|
|
For the best results in Ollama or LM Studio, use the following template: |
|
|
|
|
|
```text |
|
|
<|start_header_id|>system<|end_header_id|> |
|
|
|
|
|
You are Nomi-1.0, a high-performance 3B model. You provide superior, structured, and deep responses. Always use Markdown for clarity.<|eot_id|> |
|
|
<|start_header_id|>user<|end_header_id|> |
|
|
|
|
|
{Your Question}<|eot_id|> |
|
|
<|start_header_id|>assistant<|end_header_id|> |
|
|
``` |
|
|
|
|
|
|
|
|
## 🛠️ Usage (Ollama) |
|
|
|
|
|
1. **Download** the `Nomi-1.0.gguf`. |
|
|
2. **Create a Modelfile** with the following content: |
|
|
|
|
|
```text |
|
|
FROM ./Nomi-1.0.gguf |
|
|
PARAMETER temperature 0.6 |
|
|
SYSTEM "You are Nomi-1.0, a high-performance 3B model. You provide superior, structured, and deep responses. Always use Markdown for clarity." |
|
|
``` |
|
|
3. **Run the following command in your terminal:** |
|
|
|
|
|
```text |
|
|
ollama create Nomi-1.0 -f Modelfile |
|
|
``` |
|
|
|
|
|
|
|
|
## ⚠️ Limitations As a 3B parameter model, Nomi-1.0 is not a replacement for GPT-4 or large 70B models when it comes to deep world knowledge or complex mathematical reasoning. It is a specialized tool for speed, local privacy, and high-quality document structure. |
|
|
|
|
|
|
|
|
<p align="center"> |
|
|
|
|
|
|
|
|
|
|
|
<img src="https://cdn-uploads.huggingface.co/production/uploads/6921fa6332f7fb129563d495/gIgSlItZstAhmua_u-JDX.png" width="300"> |
|
|
</p> |
