File size: 2,108 Bytes
8651d52 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 | # Atlas-1B: Lightweight Fine-tuned LLM for Edge and Low-Memory Devices
🚀 **Atlas-1B** is a 1.2-billion parameter model fine-tuned from **BaseLLM-1B** to deliver improved accuracy, reasoning, and efficiency on low-power inference devices (e.g., Jetson, Ryzen APU, and mobile-based LLM frameworks).
This version introduces **quantization-aware finetuning**, **dataset specialization**, and **token efficiency optimization**, making it a solid drop-in model for on-device AI use cases.
---
## 🧠 Model Overview
- **Base model:** BaseLLM-1B v1.3 (transformer-based autoregressive)
- **Architecture:** Decoder-only transformer
- **Parameters:** 1.2B
- **Precision support:** FP16 / INT8 / INT4
- **Context length:** 16K tokens
- **Tokenizer:** SentencePiece (32K vocab)
- **Frameworks supported:** PyTorch, vLLM, and sglang
This model was optimized specifically for **edge inference** and **multi-request throughput**, providing ~30% lower memory bandwidth usage at batch=4 compared to the base model.
---
## 🧩 Use Cases
- On-device chat assistants
- Smart IoT response systems
- Embedded analytics (offline summarization, intent detection, etc.)
- Lightweight reasoning for robotics
---
## 🔧 Fine-tuning Details
| Attribute | Description |
|------------|-------------|
| **Dataset** | Blend of 50M tokens curated for code, chat, and reasoning |
| **Training framework** | PyTorch + DeepSpeed ZeRO-2 |
| **Optimizer** | AdamW |
| **Learning rate** | 2e-5 (cosine decay) |
| **Batch size** | 512 tokens per GPU |
| **Epochs** | 3 |
| **Loss function** | Cross-entropy (token-level) |
| **Special techniques** | LoRA adapters (rank=8), QLoRA-aware finetuning, FlashAttention-2 integration |
---
## 🧪 Performance Benchmarks
| Metric | BaseLLM-1B | Atlas-1B |
|--------|-------------|----------|
| **MMLU (Subset)** | 30.2 | 38.7 |
| **CodeEval (Python)** | 22.4 | 29.1 |
| **Average latency (Jetson Orin, INT4)** | 213ms | 158ms |
| **Memory usage (FP16)** | 7.9GB | 5.4GB |
> Benchmarks measured with vLLM 0.4.2 and sglang backend on an RTX 3060 (12GB) and Jetson Orin AGX.
|