Instructions to use divyanshukunwar/SASTRI_1_9B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use divyanshukunwar/SASTRI_1_9B with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="divyanshukunwar/SASTRI_1_9B")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("divyanshukunwar/SASTRI_1_9B")
model = AutoModelForCausalLM.from_pretrained("divyanshukunwar/SASTRI_1_9B")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps

vLLM

How to use divyanshukunwar/SASTRI_1_9B with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "divyanshukunwar/SASTRI_1_9B"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "divyanshukunwar/SASTRI_1_9B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/divyanshukunwar/SASTRI_1_9B

SGLang

How to use divyanshukunwar/SASTRI_1_9B with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "divyanshukunwar/SASTRI_1_9B" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "divyanshukunwar/SASTRI_1_9B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "divyanshukunwar/SASTRI_1_9B" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "divyanshukunwar/SASTRI_1_9B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use divyanshukunwar/SASTRI_1_9B with Docker Model Runner:
```
docker model run hf.co/divyanshukunwar/SASTRI_1_9B
```

SASTRI_1_9B is a new model fine tuned for bhojpuri & hindi languages. More detail about the dataset used and evaluation will be shared soon. Model trained on similar languages can be adapted for a very low resource language.

Let's see what model has to say about itself.

You can also download the GGUF version from : https://huggingface.co/divyanshukunwar/SASTRI_1_9B_GGUF

User : Tell me about yourself
Assistant : Hi there! I am SASTRI, an AI assistant designed to help you with various tasks and answer questions as best as my programming allows.
As a large language model, I don't have personal feelings or emotions like humans do but rest assured that I am here to assist you with anything within my capabilities!

User : aap or kon kon si bhaasa me likh skte hai ?
Assistant : मैं कई भाषाओं को समझता हूँ और उनमें बातचीत कर सकता हूँ, हिंदी में लिखने के अलावा मैं अंग्रेजी और भोजपुरी भी जानता हूं। 🇮🇳

Evaluation Result for English Language

Dataset	Metric	Score
Arc-Easy	Accuracy	94.2
Arc-Challenge	Accuracy	86.1
MMLU	Accuracy	61.0
BoolQ	Accuracy	79.04

Open LLM Leaderboard Result

Evaluation Result for Hindi Language

Dataset	Metric	Score
Arc-Easy	Accuracy	80.2
Arc-Challenge	Accuracy	68.35
MMLU	Accuracy	43.11
BoolQ	Accuracy	71.61

Evaluation Result for Bhojpuri Language coming soon ..

Model may show undersired behavior such as hallucinations and unintended biases use it at your own risk. This model is not optimized for user preference. This model is released as a part of study for Large Language model for very low resource languages.

Comparison with other Indic models on English Evaluation Benchmark	Comparison with other models on Hindi Evaluation Benchmark

Downloads last month: 4

Safetensors

Model size

10B params

Tensor type

F32

F16

Model tree for divyanshukunwar/SASTRI_1_9B

Base model

google/gemma-2-9b

Quantized

(52)

this model