README.md

---
license: mit
base_model: Qwen/Qwen2.5-0.5B
tags:
- text-generation
- causal-lm
- grpo
- reasoning
- reinforcement-learning
- mini-llm
- text-generation-inference
- lm-evaluation-harness
datasets:
- openai/gsm8k
- openai/openai_humaneval
- cais/mmlu
- allenai/ai2_arc
- bespokelabs/Bespoke-Stratos-17k
language:
- en
pipeline_tag: text-generation
metrics:
- accuracy
- exact_match
model-index:
- name: Atomight-V2.1-0.5B-Inference
  results:
  - task:
      type: text-generation
      name: ARC-Challenge
    dataset:
      name: AI2 ARC-Challenge
      type: allenai/ai2_arc
      config: ARC-Challenge
      split: test
    metrics:
    - name: Accuracy
      type: acc
      value: 0.30802
    - name: Accuracy (Norm)
      type: acc_norm
      value: 0.33788
  - task:
      type: text-generation
      name: GSM8K
    dataset:
      name: GSM8K
      type: openai/gsm8k
      config: main
      split: test
    metrics:
    - name: Exact Match (Strict)
      type: exact_match
      value: 0.19788
    - name: Exact Match (Flexible)
      type: exact_match
      value: 0.32449
  - task:
      type: text-generation
      name: ARC-Easy
    dataset:
      name: AI2 ARC-Easy
      type: allenai/ai2_arc
      config: ARC-Easy
      split: test
    metrics:
    - name: Accuracy
      type: acc
      value: 0.65446
    - name: Accuracy (Norm)
      type: acc_norm
      value: 0.59343
  - task:
      type: text-generation
      name: HellaSwag
    dataset:
      name: HellaSwag
      type: Rowan/hellaswag
      split: validation
    metrics:
    - name: Accuracy
      type: acc
      value: 0.40709
    - name: Accuracy (Norm)
      type: acc_norm
      value: 0.52350
---

# Atomight-V2.1-0.5B-Inference

<p align="center">
  <img src="OfficialAtomight.png" alt="Atomight Logo" width="500" style="max-width: 100%;">
</p>

**Atomight-V2.1-0.5B-Inference** is an ultra-compact, reasoning-oriented causal language model developed under the **Atomight Ecosystem**. Built on a Qwen-derived 494M parameter foundation, the model has been refined using **GRPO (Group Relative Policy Optimization)** reinforcement tuning. 

Despite its tiny physical footprint, Atomight-V2.1-0.5B targets highly efficient edge-device reasoning, structured text outputs, lightweight coding assistance, and rapid deployment workflows under severe compute constraints.

### 🚀 Key Highlights
- **Parameter Footprint:** ~494M parameters (Loads into ~1GB VRAM at FP16).
- **Training Paradigm:** GRPO reinforcement learning focusing on high-signal reasoning vectors instead of brute-force dataset scale.
- **Edge-Optimized:** Designed specifically for low-overhead mobile, local, and browser-based inference loops (Google Colab / Kaggle native workflow).

---

## 📊 Evaluation & Benchmark Results

Official evaluations were conducted using the **EleutherAI LM Evaluation Harness** at FP16 precision. 

### Core Evaluation Metrics
| Benchmark Task | Metric Typology | Atomight-V2.1-0.5B Score | Focus Domain |
| :--- | :--- | :--- | :--- |
| **ARC-Easy** | Accuracy (Normalized) | **59.34%** | Scientific Fact Retrieval |
| **HellaSwag** | Accuracy (Normalized) | **52.35%** | Commonsense Reasoning & Next-Sentence Prediction |
| **ARC-Challenge** | Accuracy (Normalized) | **33.79%** | Hard Analytical Exclusion Logic |
| **GSM8K (Flexible Extract)** | Exact Match (Regex Clean) | **32.45%** | Mathematical Thought & Resolution |
| **GSM8K (Strict)** | Exact Match (Rigid Parse) | **19.79%** | Formatted Mathematical Output |

<p align="center">
  <img src="OfficialBenchmarkAtomight2.1.png" alt="Atomight V2.1 Benchmark" width="500" style="max-width: 100%;">
</p>

### 🔍 Comparative Engineering Insights

* **Punching Above Weight Classes:** Atomight-V2.1-0.5B outpaces Meta's larger **Llama-3.2-1B-Instruct** on localized logic-retrieval metrics, clearing **59.3%** on ARC-Easy and **33.8%** on ARC-Challenge compared to Llama's *56.7%* and *31.8%* respectively.
* **The Reasoning Gap:** On mathematical reasoning (GSM8K), when evaluated with **Flexible Extraction parsing (32.45%)**, Atomight demonstrates higher raw mathematical accuracy than both Qwen2.5-0.5B-Instruct (*26.8%*) and Llama-3.2-1B-Instruct (*24.4%*). 
* **The Formatting Note:** The delta between Atomight's Strict Math score (19.8%) and Flexible Math score (32.5%) stems from the internal reasoning tokens generated during the inference step. While the mathematical conclusion is correct nearly 1/3 of the time, the model frequently bypasses rigid formatting constraints in favor of dense thinking traces.

---

## 💻 Quickstart: Inference Execution

Atomight utilizes system and sequence prompts to partition thinking spaces. For optimal reasoning convergence, use explicit `<thinking>` and `<answer>` encapsulation layers.

```python
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "NovatasticRoScript/Atomight-V2.1-0.5B-Inference"

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
    model_id, 
    torch_dtype=torch.float16, 
    device_map="auto"
)

# Structuring system guidelines for GRPO activation
messages = [
    {
        "role": "system", 
        "content": "You are a reasoning model. Think inside <thinking> and answer inside <answer>."
    },
    {
        "role": "user", 
        "content": "A farmer has 12 apples. He gives 4 to his neighbor and loses 2 on the way home. How many apples does he have left?"
    }
]

inputs = tokenizer.apply_chat_template(
    messages, 
    tokenize=True, 
    add_generation_prompt=True, 
    return_tensors="pt"
).to("cuda")

with torch.no_grad():
    outputs = model.generate(
        inputs, 
        max_new_tokens=250, 
        temperature=0.01,
        pad_token_id=tokenizer.eos_token_id
    )

print(tokenizer.decode(outputs[0], skip_special_tokens=True))