---
license: apache-2.0
language:
- en
- hi
tags:
- sarus
- viuai
- reasoning
- cot
- hindi
- english
- causal-lm
pipeline_tag: text-generation
---

<div align="center">

# 🪶 Sarus-500M

### *A Lightweight Bilingual Cognitive Reasoning Model by ViuAI*

[![License](https://img.shields.io/badge/License-Apache%202.0-blue.svg)](https://opensource.org/licenses/Apache-2.0)
[![Parameters](https://img.shields.io/badge/Parameters-500M-green.svg)](#model-overview)
[![Context](https://img.shields.io/badge/Context-2048-orange.svg)](#model-overview)
[![Reasoning](https://img.shields.io/badge/Reasoning-Cognitive%20Monologue-purple.svg)](#features)

</div>

## 📌 Model Overview

**Sarus-500M** is a 500M-parameter compact causal language model developed by **ViuAI**. The model is designed for efficient, low-latency reasoning and bilingual text generation across English and Hindi (Hinglish).

Sarus executes an internal **Cognitive Reasoning Monologue** (`[THINK] ... [/THINK]`) before producing its final answer, enabling structured step-by-step problem solving in a lightweight architecture.

---

## 🌟 Key Features

* **🧠 Cognitive Reasoning**: Structured chain-of-thought monologue enclosed in `[THINK] ... [/THINK]` tags.
* **🇮🇳 Bilingual Fluency**: Native support for English, Hindi, and romanized Hinglish.
* **⚡ High Efficiency**: Optimized for low-latency edge and server deployment using Grouped Query Attention (GQA).
* **💬 Conversational**: Multi-turn dialogue support with standard chat formatting.

---

## 📐 Architecture Summary

| Feature | Value |
| :--- | :--- |
| **Parameters** | ~500M |
| **Hidden Size** | 1280 |
| **Layers** | 24 |
| **Attention Heads** | 20 (Query) / 4 (KV) |
| **Max Context** | 2048 |
| **Vocabulary Size** | 64,003 |

---

## 🚀 Quickstart

### Chat Template Format

```
<|user|>
{user_message}<|endofturn|>
<|assistant|>
[THINK]
{internal_reasoning}
[/THINK]
{final_answer}<|endofturn|>
```

### Python Inference

```python
from transformers import AutoTokenizer, AutoModelForCausalLM

# Load Tokenizer & Model
model_id = "ViuAI/ViuAI-500M"
tokenizer = AutoTokenizer.from_pretrained(model_id)
# model = AutoModelForCausalLM.from_pretrained(model_id, torch_dtype="auto", device_map="auto")
#
# NOTE: This repo trains a custom architecture (see `code/model.py`, `code/config.py`).
# Canonical vocab size is 64,003 with special tokens:
#   <|user|> = 64000, <|assistant|> = 64001, <|endofturn|> = 64002
# Set HF_TOKEN env var for any Hugging Face downloads. Never hardcode tokens.
```

---

## 📄 License

Sarus-500M is licensed under the **Apache 2.0 License**.

<div align="center">

*Developed by the ViuAI Team.*

</div>