CoRover/BharatGPT-mini

Model Description

BharatGPT mini is a Transformer-based language model pretrained on a large corpus of publicly available text data using a self-supervised learning approach. This means the model was trained without any human-labeled annotations—learning directly from raw text using an automatic mechanism to generate training signals.

During pretraining, BharatGPT mini was optimized for the causal language modeling task: given a sequence of tokens, the model learns to predict the next token in the sequence. More specifically, it takes a sequence of continuous text as input and is trained to predict the next word or subword by shifting the target sequence one position to the right. A masking mechanism ensures that predictions for token i are based only on tokens from positions 1 to i, without peeking at future tokens. This preserves the autoregressive nature of language modeling.

Through this training process, BharatGPT mini develops a deep internal understanding of language patterns, grammar, and semantics. While it can be fine-tuned for various downstream tasks such as classification, summarization, or question answering, it performs best in text generation tasks, which align with its original training objective.

import torch
from transformers import GPT2Tokenizer, GPT2LMHeadModel

# Load tokenizer and model
tokenizer = GPT2Tokenizer.from_pretrained("CoRover/BharatGPT-mini")
model = GPT2LMHeadModel.from_pretrained("CoRover/BharatGPT-mini")

model.eval()

# Input text
text = "Future of AI"

# Tokenize
inputs = tokenizer(
    text,
    return_tensors="pt"
)

# Generate text
with torch.no_grad():
    output_ids = model.generate(
        **inputs,
        max_length=100,
        do_sample=True,
        top_p=0.95,
        top_k=50,
        temperature=0.8,
        repetition_penalty=1.1,
        eos_token_id=tokenizer.eos_token_id
    )

# Decode output
generated_text = tokenizer.decode(output_ids[0], skip_special_tokens=True)
print(generated_text)

It is best suited for S-RAG (Secure Retrieval-Augmented Generation) or fine-tuning with your own data. For enhanced performance, integration with Conversational Agentic AI platform is recommended (though not mandatory). This platform enables the creation of multi-modal and multi-lingual AI Agents, Co-Pilots, and Virtual Assistants (such as ChatBots, VoiceBots, and VideoBots) using a sovereign AI and composite AI approach. It leverages classic NLP, grounded generative AI with BharatGPT, and Generally Available LLMs to deliver powerful, versatile AI solutions.

Usage and Limitations

• License: Non-Commercial. For academic and research purposes only. For commercial use, please visit Conversational Gen AI platform or Contact Us.

• Terms of Use: Terms and Conditions

• Responsible AI Framework: CoRover's Responsible AI Framework

• Developed by: CoRover.ai