Instructions to use KissanAI/Dhenu2-In-Llama3.2-3B-Instruct with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use KissanAI/Dhenu2-In-Llama3.2-3B-Instruct with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="KissanAI/Dhenu2-In-Llama3.2-3B-Instruct")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("KissanAI/Dhenu2-In-Llama3.2-3B-Instruct")
model = AutoModelForCausalLM.from_pretrained("KissanAI/Dhenu2-In-Llama3.2-3B-Instruct")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Inference
Notebooks
Google Colab
Kaggle
Local Apps

vLLM

How to use KissanAI/Dhenu2-In-Llama3.2-3B-Instruct with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "KissanAI/Dhenu2-In-Llama3.2-3B-Instruct"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "KissanAI/Dhenu2-In-Llama3.2-3B-Instruct",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/KissanAI/Dhenu2-In-Llama3.2-3B-Instruct

SGLang

How to use KissanAI/Dhenu2-In-Llama3.2-3B-Instruct with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "KissanAI/Dhenu2-In-Llama3.2-3B-Instruct" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "KissanAI/Dhenu2-In-Llama3.2-3B-Instruct",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "KissanAI/Dhenu2-In-Llama3.2-3B-Instruct" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "KissanAI/Dhenu2-In-Llama3.2-3B-Instruct",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use KissanAI/Dhenu2-In-Llama3.2-3B-Instruct with Docker Model Runner:
```
docker model run hf.co/KissanAI/Dhenu2-In-Llama3.2-3B-Instruct
```
Browse Quantizations to use this model in llama.cpp, Ollama, LM Studio, or any compatible app.

Dhenu2 India 3B

Model Overview

Model Name: Llama3.2-Dhenu2-In-3B-Instruct

Architecture: Llama3.2

Parameters: 3 Billion

Release Date: 24th October, 2024

License: Llama 3.2 Community License

Description

Dhenu2 India 3B offers a balanced performance, making it versatile for a wide range of agricultural applications. Built on the Llama3.2 architecture, this model is optimized to provide reliable conversational capabilities, ensuring a harmonious blend of knowledge depth and responsiveness. It is ideal for conversational applications that require accurate information and interactive engagement in the agricultural domain.

Intended Use

Chatbots and Virtual Assistants: Develop interactive tools that assist farmers with daily agricultural queries and tasks.

Training Data

Dhenu2 India 3B was trained on a curated dataset that includes:

Instruction Set: Over 1.5 million instructions from real and synthetic conversations.
Synthetic Instructions: Generated to encompass more than 4,000 diverse agricultural topics.
Data Sources: Mobile extension service logs, farmer feedback, agricultural package of practices, and localized studies.

Training Procedure

Techniques: Utilized a combination of full fine-tuning and Low-Rank Adaptation (LoRA) to enhance model performance while conserving computational resources.
Hardware: Trained on multi-GPU configurations with NVIDIA A100 GPUs, employing DeepSpeed for efficient distributed training.
Optimization: Applied flash attention mechanisms to improve computational efficiency and reduce memory usage.

Evaluation

Human Evaluation: Reviewed by agricultural experts for the accuracy and relevance of responses.
Synthetic Evaluation: Conducted peer assessments using other LLMs to verify consistency and correctness.
Performance Metrics: Measured based on domain-specific accuracy, response relevancy, and conversational fluency.

Limitations

Dhenu2 India 3B is tailored for agricultural applications and may not perform optimally outside this domain. It is essential to deploy this model within relevant agricultural contexts to maintain the accuracy and reliability of its responses.

API

Use our platform Dhenu with a generous free quota to start building your agriculture applications.

A note of gratitude

We want to thank our partners Microsoft and Microsoft for Startups for landing us compute. We would also like to thank our partner, Meta, for the open-source Llama models.

Contact

For more information, support, or collaboration inquiries, please contact us at [support@kissan.ai].

Downloads last month: 4

Safetensors

Model size

3B params

Tensor type

BF16

Model tree for KissanAI/Dhenu2-In-Llama3.2-3B-Instruct

Quantizations

2 models