README.md

---
base_model: unsloth/gpt-oss-20b-unsloth-bnb-4bit
tags:
- text-generation-inference
- transformers
- unsloth
- gpt_oss
license: apache-2.0
language:
- en
---
This is a fine-tuned causal language model specialized for **Nigerian Pidgin English** (Naijá), trained on the `openai/gpt-oss-20b` base model using efficient fine-tuning techniques.

## 📋 Model Details

- **Model Type**: Causal Language Model (CLM)
- **Base Model**: [openai/gpt-oss-20b](https://huggingface.co/openai/gpt-oss-20b)
- **Language**: Nigerian Pidgin English (ISO 639-3: `pcm`)
- **Training Framework**: [Unsloth](https://github.com/unslothai/unsloth) (optimized fine-tuning)
- **Model Size**: ~20B parameters
- **Status**: ✅ Fully merged (standalone model, no adapter required)

## 🎯 Intended Use

This model is designed for:

- **Text Generation**: Generate fluent Nigerian Pidgin text
- **Conversational AI**: Build chatbots that speak Pidgin
- **Language Translation**: Translate to/from Pidgin English
- **Content Creation**: Write stories, dialogues, or scripts in Pidgin
- **Educational Tools**: Help people learn Nigerian Pidgin
- **Cultural Preservation**: Document and preserve Pidgin language

## 🚀 Quick Start

### Installation
```bash
pip install transformers torch accelerate
```

### Basic Usage
```python
from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

# Load model and tokenizer
model = AutoModelForCausalLM.from_pretrained(
    "Ephraimmm/pidgin_finetuned_model",
    torch_dtype=torch.float16,
    device_map="auto"
)

tokenizer = AutoTokenizer.from_pretrained("Ephraimmm/pidgin_finetuned_model")

# Generate text
prompt = "Wetin you dey do today?"
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)

outputs = model.generate(
    **inputs,
    max_new_tokens=100,
    temperature=0.7,
    top_p=0.9,
    do_sample=True,
    repetition_penalty=1.1
)

response = tokenizer.decode(outputs[0], skip_special_tokens=True)
print(response)
```

### Advanced Usage with Generation Config
```python
from transformers import GenerationConfig

# Create generation config for better control
generation_config = GenerationConfig(
    max_new_tokens=150,
    temperature=0.8,
    top_p=0.95,
    top_k=50,
    repetition_penalty=1.2,
    do_sample=True,
    pad_token_id=tokenizer.pad_token_id,
    eos_token_id=tokenizer.eos_token_id,
)

# Generate with config
outputs = model.generate(
    **inputs,
    generation_config=generation_config
)

text = tokenizer.decode(outputs[0], skip_special_tokens=True)
print(text)
```

### Chat/Conversation Format
```python
def generate_pidgin_response(prompt, model, tokenizer, max_length=100):
    """Generate a Pidgin response to a prompt"""
    inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
    
    with torch.no_grad():
        outputs = model.generate(
            **inputs,
            max_new_tokens=max_length,
            temperature=0.7,
            top_p=0.9,
            do_sample=True,
            repetition_penalty=1.1,
            pad_token_id=tokenizer.pad_token_id,
        )
    
    return tokenizer.decode(outputs[0], skip_special_tokens=True)

# Example conversation
prompts = [
    "How you dey?",
    "Wetin you wan chop?",
    "Make we go market?",
    "I tire o!"
]

for prompt in prompts:
    response = generate_pidgin_response(prompt, model, tokenizer)
    print(f"Input: {prompt}")
    print(f"Output: {response}\\n")
```

## 💡 Example Outputs

### Example 1: Greeting
**Input:** `"Wetin you dey do?"`  
**Output:** `"I dey here dey work small. You sef, how you dey?"`

### Example 2: Question
**Input:** `"How person go reach there?"`  
**Output:** `"You fit take bus from here, den you go come down for junction"`

### Example 3: Storytelling
**Input:** `"One day, one man"`  
**Output:** `"One day, one man waka go market to buy something for him family. As e reach there, e see say..."`

## ⚙️ Training Details

### Training Procedure

- **Fine-tuning Method**: LoRA (Low-Rank Adaptation) merged into base model
- **Training Framework**: Unsloth (memory-efficient training)
- **Hardware**: GPU-accelerated training
- **Precision**: Mixed precision (FP16/BF16)
- **Optimization**: Efficient fine-tuning with quantization

### Hyperparameters
```yaml
base_model: openai/gpt-oss-20b
training_framework: unsloth
fine_tuning_method: lora
merged: true
```

## 📊 Performance Characteristics

- **Fluency**: Generates natural-sounding Nigerian Pidgin
- **Vocabulary**: Covers common Pidgin expressions and phrases
- **Context Understanding**: Maintains context in conversations
- **Cultural Relevance**: Understands Nigerian cultural context

## ⚠️ Limitations

- **Training Data**: Limited to the scope of training data provided
- **Formal vs Informal**: May not distinguish between different formality levels
- **Regional Variations**: Nigerian Pidgin has regional variations; model may favor certain dialects
- **Code-Switching**: May occasionally mix English and Pidgin
- **Offensive Content**: May generate inappropriate content; use with moderation
- **Factual Accuracy**: As a language model, it may generate plausible-sounding but incorrect information

## 🔧 Technical Specifications

### Model Architecture
- Architecture: Causal Language Model
- Parameters: ~20 billion
- Precision: FP16/FP32
- Context Length: 2048 tokens (varies by base model)

### System Requirements

**Minimum (Inference):**
- GPU: 16GB VRAM (e.g., RTX 4090, V100)
- RAM: 32GB
- Storage: 50GB

**Recommended:**
- GPU: 24GB+ VRAM (e.g., RTX 6000, A5000, A100)
- RAM: 64GB+
- Storage: 100GB

**For CPU-only inference:**
- RAM: 64GB+
- Warning: Very slow generation

### Memory Optimization
```python
# For limited VRAM, use 8-bit quantization
from transformers import BitsAndBytesConfig

quantization_config = BitsAndBytesConfig(
    load_in_8bit=True,
    llm_int8_threshold=6.0
)

model = AutoModelForCausalLM.from_pretrained(
    "Ephraimmm/pidgin_finetuned_model",
    quantization_config=quantization_config,
    device_map="auto"
)
```

## 🌍 Language Information

**Nigerian Pidgin (Naijá)** is an English-based creole language spoken as a lingua franca across Nigeria. It's estimated that between 75-100 million people speak Nigerian Pidgin, making it one of the most widely spoken languages in West Africa.

### Key Features:
- Simplified grammar compared to English
- Vocabulary borrowed from English, local Nigerian languages, and Portuguese
- Used in informal communication, music, comedy, and increasingly in media
- No standardized written form, but phonetic spelling is common

## 📝 Citation

If you use this model in your research or application, please cite:
```bibtex
@misc{pidgin_finetuned_model_2026,
  author = {Ephraimmm},
  title = {Pidgin English Fine-tuned Language Model},
  year = {2026},
  publisher = {HuggingFace},
  journal = {HuggingFace Model Hub},
  howpublished = {\\url{https://huggingface.co/Ephraimmm/pidgin_finetuned_model}}
}
```


# Uploaded finetuned  model

- **Developed by:** Ephraimmm
- **License:** apache-2.0
- **Finetuned from model :** unsloth/gpt-oss-20b-unsloth-bnb-4bit

This gpt_oss model was trained 2x faster with [Unsloth](https://github.com/unslothai/unsloth) and Huggingface's TRL library.

[<img src="https://raw.githubusercontent.com/unslothai/unsloth/main/images/unsloth%20made%20with%20love.png" width="200"/>](https://github.com/unslothai/unsloth)