| # Atlas-1B: Lightweight Fine-tuned LLM for Edge and Low-Memory Devices | |
| 🚀 **Atlas-1B** is a 1.2-billion parameter model fine-tuned from **BaseLLM-1B** to deliver improved accuracy, reasoning, and efficiency on low-power inference devices (e.g., Jetson, Ryzen APU, and mobile-based LLM frameworks). | |
| This version introduces **quantization-aware finetuning**, **dataset specialization**, and **token efficiency optimization**, making it a solid drop-in model for on-device AI use cases. | |
| --- | |
| ## 🧠 Model Overview | |
| - **Base model:** BaseLLM-1B v1.3 (transformer-based autoregressive) | |
| - **Architecture:** Decoder-only transformer | |
| - **Parameters:** 1.2B | |
| - **Precision support:** FP16 / INT8 / INT4 | |
| - **Context length:** 16K tokens | |
| - **Tokenizer:** SentencePiece (32K vocab) | |
| - **Frameworks supported:** PyTorch, vLLM, and sglang | |
| This model was optimized specifically for **edge inference** and **multi-request throughput**, providing ~30% lower memory bandwidth usage at batch=4 compared to the base model. | |
| --- | |
| ## 🧩 Use Cases | |
| - On-device chat assistants | |
| - Smart IoT response systems | |
| - Embedded analytics (offline summarization, intent detection, etc.) | |
| - Lightweight reasoning for robotics | |
| --- | |
| ## 🔧 Fine-tuning Details | |
| | Attribute | Description | | |
| |------------|-------------| | |
| | **Dataset** | Blend of 50M tokens curated for code, chat, and reasoning | | |
| | **Training framework** | PyTorch + DeepSpeed ZeRO-2 | | |
| | **Optimizer** | AdamW | | |
| | **Learning rate** | 2e-5 (cosine decay) | | |
| | **Batch size** | 512 tokens per GPU | | |
| | **Epochs** | 3 | | |
| | **Loss function** | Cross-entropy (token-level) | | |
| | **Special techniques** | LoRA adapters (rank=8), QLoRA-aware finetuning, FlashAttention-2 integration | | |
| --- | |
| ## 🧪 Performance Benchmarks | |
| | Metric | BaseLLM-1B | Atlas-1B | | |
| |--------|-------------|----------| | |
| | **MMLU (Subset)** | 30.2 | 38.7 | | |
| | **CodeEval (Python)** | 22.4 | 29.1 | | |
| | **Average latency (Jetson Orin, INT4)** | 213ms | 158ms | | |
| | **Memory usage (FP16)** | 7.9GB | 5.4GB | | |
| > Benchmarks measured with vLLM 0.4.2 and sglang backend on an RTX 3060 (12GB) and Jetson Orin AGX. | |