---
license: apache-2.0
language:
- en
library_name: transformers
tags:
- LLM
- CasualLLM
- StoryLLM
- SLM
- Nova-Verse
base_model:
- harshit36/Nova-Casual-LLM
datasets:
- harshit36/Decision-driven-Stories
---

<img src="NOVA-poster.png" alt="Logo" style="border-radius: 30px;" width="100%"/>

# Nova-Verse

A NOVA Finetuned model which is specifically trained for decision-driven Story generator.

## Model Summary

- **Model Name**: NovaForCausalLM
- **Architecture**: Custom decoder-only transformer (`NOVA`)
- **Model Type**: `nova` (Fine-tuned version)
- **Use Case**: Causal Language Modeling (text generation, auto-completion)
- **Parameters**: 14,412,400 trainable parameters
- **Pretrained Tokenizer**: `PreTrainedTokenizerFast`
- **Framework**: PyTorch
- **Hugging Face Integration**: Compatible with `transformers` via custom `AutoModel` and `AutoConfig` registration.

---

## Files Included

| File                       | Description                                       |
|---------------------------|----------------------------------------------------|
| `config.json`             | Configuration of model hyperparameters             |
| `model.safetensors`       | Serialized model weights (efficient format)        |
| `nova_modelling.py`       | Custom model and config class definitions          |
| `tokenizer.json`          | Serialized tokenizer                               |
| `tokenizer_config.json`   | Tokenizer configuration metadata                   |
| `special_tokens_map.json` | Mapping for special tokens (e.g., BOS, EOS)        |
| `README.md`               | Model card (you’re reading it!)                    |

---

## Model Architecture

### `NovaForCausalLM`

The model consists of:
- Embedding layers: token + positional
- Stack of transformer decoder blocks
- Multi-head attention with 640 individual heads
- Layer normalization
- Final linear head for vocabulary logits

### Configuration (`NovaConfig`)

```json
{
  "model_type": "nova",
  "vocab_size": 6000,
  "block_size": 256,
  "n_embd": 640,
  "n_layer": 4,
  "n_head": 8
}
```

## 🚀 Usage

### Step 1: Clone the repo (to get the `nova_modelling.py`)

```bash
git clone https://huggingface.co/harshit36/Nova-Verse
cd Nova-Verse
```

```python
import sys
sys.path.append("./Nova-Verse/")  # add current dir to path

from transformers import PreTrainedTokenizerFast
from nova_modelling import NovaConfig, NovaForCausalLM

# Load tokenizer
tokenizer = PreTrainedTokenizerFast.from_pretrained("harshit36/Nova-Verse")

# Load config
config = NovaConfig.from_pretrained("harshit36/Nova-Verse")

# Instantiate model using your custom class
model = NovaForCausalLM(config)
model = model.from_pretrained("harshit36/Nova-Verse")

# Use the model
input_ids = tokenizer("Hello world", return_tensors="pt").input_ids
output = model.generate(input_ids)
print(tokenizer.decode(output[0], skip_special_tokens=True).replace(" ","").replace("Ġ"," ").replace("Ċ","\n"))

```

## Intended Use

Story text generation

Hybrid Positional Encoding Research model (Combination of Sinusoidal and learnable encodings)

Educational demonstrations of custom HF model integration

Rapid prototyping of transformer models