Ephraimmm/pidgin_finetuned_model

This is a fine-tuned causal language model specialized for Nigerian Pidgin English (Naijá), trained on the openai/gpt-oss-20b base model using efficient fine-tuning techniques.

📋 Model Details

• Model Type: Causal Language Model (CLM)
• Base Model: openai/gpt-oss-20b
• Language: Nigerian Pidgin English (ISO 639-3: pcm)
• Training Framework: Unsloth (optimized fine-tuning)
• Model Size: ~20B parameters
• Status: ✅ Fully merged (standalone model, no adapter required)

🎯 Intended Use

This model is designed for:

• Text Generation: Generate fluent Nigerian Pidgin text
• Conversational AI: Build chatbots that speak Pidgin
• Language Translation: Translate to/from Pidgin English
• Content Creation: Write stories, dialogues, or scripts in Pidgin
• Educational Tools: Help people learn Nigerian Pidgin
• Cultural Preservation: Document and preserve Pidgin language

🚀 Quick Start

Installation

pip install transformers torch accelerate

Basic Usage

from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

# Load model and tokenizer
model = AutoModelForCausalLM.from_pretrained(
    "Ephraimmm/pidgin_finetuned_model",
    torch_dtype=torch.float16,
    device_map="auto"
)

tokenizer = AutoTokenizer.from_pretrained("Ephraimmm/pidgin_finetuned_model")

# Generate text
prompt = "Wetin you dey do today?"
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)

outputs = model.generate(
    **inputs,
    max_new_tokens=100,
    temperature=0.7,
    top_p=0.9,
    do_sample=True,
    repetition_penalty=1.1
)

response = tokenizer.decode(outputs[0], skip_special_tokens=True)
print(response)

Advanced Usage with Generation Config

from transformers import GenerationConfig

# Create generation config for better control
generation_config = GenerationConfig(
    max_new_tokens=150,
    temperature=0.8,
    top_p=0.95,
    top_k=50,
    repetition_penalty=1.2,
    do_sample=True,
    pad_token_id=tokenizer.pad_token_id,
    eos_token_id=tokenizer.eos_token_id,
)

# Generate with config
outputs = model.generate(
    **inputs,
    generation_config=generation_config
)

text = tokenizer.decode(outputs[0], skip_special_tokens=True)
print(text)

Chat/Conversation Format

def generate_pidgin_response(prompt, model, tokenizer, max_length=100):
    """Generate a Pidgin response to a prompt"""
    inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
    
    with torch.no_grad():
        outputs = model.generate(
            **inputs,
            max_new_tokens=max_length,
            temperature=0.7,
            top_p=0.9,
            do_sample=True,
            repetition_penalty=1.1,
            pad_token_id=tokenizer.pad_token_id,
        )
    
    return tokenizer.decode(outputs[0], skip_special_tokens=True)

# Example conversation
prompts = [
    "How you dey?",
    "Wetin you wan chop?",
    "Make we go market?",
    "I tire o!"
]

for prompt in prompts:
    response = generate_pidgin_response(prompt, model, tokenizer)
    print(f"Input: {prompt}")
    print(f"Output: {response}\\n")

💡 Example Outputs

Example 1: Greeting

Input: "Wetin you dey do?"
Output: "I dey here dey work small. You sef, how you dey?"

Example 2: Question

Input: "How person go reach there?"
Output: "You fit take bus from here, den you go come down for junction"

Example 3: Storytelling

Input: "One day, one man"
Output: "One day, one man waka go market to buy something for him family. As e reach there, e see say..."

⚙️ Training Details

Training Procedure

• Fine-tuning Method: LoRA (Low-Rank Adaptation) merged into base model
• Training Framework: Unsloth (memory-efficient training)
• Hardware: GPU-accelerated training
• Precision: Mixed precision (FP16/BF16)
• Optimization: Efficient fine-tuning with quantization

Hyperparameters

base_model: openai/gpt-oss-20b
training_framework: unsloth
fine_tuning_method: lora
merged: true

📊 Performance Characteristics

• Fluency: Generates natural-sounding Nigerian Pidgin
• Vocabulary: Covers common Pidgin expressions and phrases
• Context Understanding: Maintains context in conversations
• Cultural Relevance: Understands Nigerian cultural context

⚠️ Limitations

• Training Data: Limited to the scope of training data provided
• Formal vs Informal: May not distinguish between different formality levels
• Regional Variations: Nigerian Pidgin has regional variations; model may favor certain dialects
• Code-Switching: May occasionally mix English and Pidgin
• Offensive Content: May generate inappropriate content; use with moderation
• Factual Accuracy: As a language model, it may generate plausible-sounding but incorrect information

🔧 Technical Specifications

Model Architecture

• Architecture: Causal Language Model
• Parameters: ~20 billion
• Precision: FP16/FP32
• Context Length: 2048 tokens (varies by base model)

System Requirements

Minimum (Inference):

• GPU: 16GB VRAM (e.g., RTX 4090, V100)
• RAM: 32GB
• Storage: 50GB

Recommended:

• GPU: 24GB+ VRAM (e.g., RTX 6000, A5000, A100)
• RAM: 64GB+
• Storage: 100GB

For CPU-only inference:

• RAM: 64GB+
• Warning: Very slow generation

Memory Optimization

# For limited VRAM, use 8-bit quantization
from transformers import BitsAndBytesConfig

quantization_config = BitsAndBytesConfig(
    load_in_8bit=True,
    llm_int8_threshold=6.0
)

model = AutoModelForCausalLM.from_pretrained(
    "Ephraimmm/pidgin_finetuned_model",
    quantization_config=quantization_config,
    device_map="auto"
)

🌍 Language Information

Nigerian Pidgin (Naijá) is an English-based creole language spoken as a lingua franca across Nigeria. It's estimated that between 75-100 million people speak Nigerian Pidgin, making it one of the most widely spoken languages in West Africa.

Key Features:

• Simplified grammar compared to English
• Vocabulary borrowed from English, local Nigerian languages, and Portuguese
• Used in informal communication, music, comedy, and increasingly in media
• No standardized written form, but phonetic spelling is common

📝 Citation

If you use this model in your research or application, please cite:

@misc{pidgin_finetuned_model_2026,
  author = {Ephraimmm},
  title = {Pidgin English Fine-tuned Language Model},
  year = {2026},
  publisher = {HuggingFace},
  journal = {HuggingFace Model Hub},
  howpublished = {\\url{https://huggingface.co/Ephraimmm/pidgin_finetuned_model}}
}

Uploaded finetuned model

• Developed by: Ephraimmm
• License: apache-2.0
• Finetuned from model : unsloth/gpt-oss-20b-unsloth-bnb-4bit

This gpt_oss model was trained 2x faster with Unsloth and Huggingface's TRL library.