README.md

---
license: mit
language:
- en
pipeline_tag: text-generation
library_name: transformers
tags:
- Rapnss
- RA1
- code
- pytorch
- India
- alpaca
---

# Rapnss DevOps-Ultra-125M 🚀

**DevOps-Ultra-125M** is a highly optimized, lightweight **125 Million parameter** language model designed specifically for Python code generation and code editing. It was built from scratch and trained entirely by **Rapnss**. 

This model features a completely custom Transformer architecture (**RapnssCodeTransformer v2**) that incorporates state-of-the-art AI technologies like **RoPE**, **SwiGLU**, and **RMSNorm**, making it highly efficient for its size.

---

## 📌 Model Details

- **Model Name:** DevOps-Ultra-125M
- **Parent Company:** Rapnss
- **Model Type:** Causal Language Model (Auto-regressive Next-Token Prediction)
- **Parameter Count:** ~125 Million
- **Language:** Python
- **Architecture:** Custom `RapnssCodeTransformer` (Built in PyTorch)

### Architecture Enhancements (v2)
Unlike a standard GPT-2 model, DevOps-Ultra-125M uses modern LLM techniques inspired by LLaMA and Mistral:
1. **Rotary Position Embeddings (RoPE):** Replaces standard absolute position embeddings for a much better understanding of sequence context and length.
2. **SwiGLU Activations:** Replaces standard GELU/ReLU in the Feed-Forward networks for improved reasoning capabilities.
3. **RMSNorm (Root Mean Square Normalization):** Replaces standard Layer Normalization for faster and more stable training.

---

## 💻 Hardware & Training Details

DevOps-Ultra-125M was purposefully designed to be trained and run on edge devices and consumer hardware.
- **Hardware Used:** NVIDIA RTX 3050 Laptop GPU (4GB VRAM)
- **Training Time:** ~1.5 Hours
- **Epochs:** 2
- **Dataset:** Fine-tuned from scratch on `iamtarun/python_code_instructions_18k_alpaca` (18,000+ instruction-following Python code snippets).
- **VRAM Optimizations:** Trained using **Automatic Mixed Precision (AMP)** and **Gradient Accumulation** (Effective Batch Size of 8) to fit within strict 4GB VRAM limitations without crashing.

---

## 🚀 How to Use the Model

Because DevOps-Ultra-125M uses a completely custom architecture, you cannot load it using the standard Hugging Face `AutoModelForCausalLM`. You must download the `rapnss_model.py` architecture file alongside the `.pt` weights.

### 1. Requirements
Ensure you have PyTorch and Transformers installed:
```bash
pip install torch transformers
```

### 2. Loading the Model
Place the `rapnss_model.py` file in your directory and run the following code:

```python
import torch
from transformers import AutoTokenizer
from rapnss_model import RapnssCodeTransformer, RapnssConfig

# 1. Load the GPT-2 Tokenizer
tokenizer = AutoTokenizer.from_pretrained("gpt2")
tokenizer.pad_token = tokenizer.eos_token

# 2. Initialize the Rapnss Architecture
config = RapnssConfig(
    vocab_size=tokenizer.vocab_size,
    max_seq_len=512
)
model = RapnssCodeTransformer(config)

# 3. Load the Weights
# Make sure you have downloaded 'Rapnss-Coder-125M.pt' from this repository
model.load_state_dict(torch.load("Rapnss-Coder-125M.pt", map_location='cpu'))
model.eval()

# 4. Generate Code
instruction = "Write a Python program to print Hello World!"
prompt = f"Instruction: {instruction}\nOutput:\n"
input_ids = tokenizer.encode(prompt, return_tensors="pt")

output_ids = model.generate(input_ids, max_new_tokens=50, temperature=0.7)
response = tokenizer.decode(output_ids[0], skip_special_tokens=True)

print(response)
```

---

## ⚠️ Limitations & Bias

DevOps-Ultra-125M is a **Proof-of-Concept** model.
- **Semantic Logic:** With only 125 million parameters, the model is incredibly small. While it has successfully learned the syntax, grammar, and structural formatting of Python code, it lacks the parameter capacity for advanced reasoning or solving complex coding algorithms.
- **Scope:** It only knows Python. It cannot generate HTML, CSS, JavaScript, or general conversational text. 
- **Use Case:** It is best used for educational purposes, architecture testing, or running extremely fast on CPU-only free-tier servers. 

---
*Developed with ❤️ by Rapnss*