Instructions to use 14maddy/AgrithmAi_bot with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use 14maddy/AgrithmAi_bot with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="14maddy/AgrithmAi_bot")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("14maddy/AgrithmAi_bot")
model = AutoModelForCausalLM.from_pretrained("14maddy/AgrithmAi_bot")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps

vLLM

How to use 14maddy/AgrithmAi_bot with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "14maddy/AgrithmAi_bot"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "14maddy/AgrithmAi_bot",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/14maddy/AgrithmAi_bot

SGLang

How to use 14maddy/AgrithmAi_bot with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "14maddy/AgrithmAi_bot" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "14maddy/AgrithmAi_bot",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "14maddy/AgrithmAi_bot" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "14maddy/AgrithmAi_bot",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use 14maddy/AgrithmAi_bot with Docker Model Runner:
```
docker model run hf.co/14maddy/AgrithmAi_bot
```

Dhenu2 India 1B

Model Overview

Model Name: Llama3.2-Dhenu2-In-1B-Instruct

Architecture: Llama3.2

Parameters: 1 Billion

Release Date: 24th October, 2024

License: Llama 3.2 Community License

Description

Designed for efficiency, Dhenu2 India 1B provides swift agricultural insights and is optimized for deployment on resource-constrained devices. Built on the Llama3.2 architecture with 1 billion parameters, this lightweight model ensures rapid responses, making it perfect for on-device applications and mobile advisory tools used by farmers and agricultural workers.

Intended Use

Mobile Applications: Embed Dhenu2 India 1B in mobile apps to provide farmers with real-time assistance and insights directly on their smartphones.
On-Device Advisory Tools: Develop lightweight advisory systems that operate efficiently on limited hardware resources.
Field Operations: Utilize in-field devices for immediate agricultural support without the need for constant internet connectivity.

Training Data

Dhenu2 India 1B was trained on a specialized dataset that reflects the varied landscape of Indian agriculture, including:

Instruction Set: Over 1.5 million instructions from real and synthetic conversations.
Synthetic Instructions: Generated through sophisticated pipelines to ensure comprehensive coverage of more than 4,000 agricultural topics.
Data Sources: Mobile extension service logs, farmer feedback, agricultural package of practices, and localized studies.

Training Procedure

Techniques: Employed full fine-tuning to optimize the model’s performance while ensuring resource efficiency.
Hardware: Trained using multi-GPU setups with NVIDIA A100 GPUs, utilizing DeepSpeed for distributed training and memory management.
Optimization: Implemented flash attention mechanisms to enhance computational efficiency and reduce memory consumption, enabling seamless deployment on mobile devices.

Evaluation

Dhenu2 India 1B was evaluated to ensure its effectiveness and efficiency for on-device applications:

Human Evaluation: Tested by agricultural professionals for relevance, speed, and accuracy of responses.
Synthetic Evaluation: Conducted peer assessments using other LLMs to validate consistency and correctness.
Performance Metrics: Assessed based on response time, accuracy in delivering insights, and efficiency in resource usage.

Limitations

While Dhenu2 India 1B excels in efficiency and speed, it may have limited depth compared to larger models. It is optimized for quick insights and may not handle highly complex or detailed agricultural queries as effectively as its larger counterparts.

API

Use our platform Dhenu with a generous free quota to start building your agriculture applications.

A note of gratitude

We want to thank our partners Microsoft and Microsoft for Startups for landing us compute. We would also like to thank our partner, Meta, for the open-source Llama models.

Contact

For more information, support, or collaboration inquiries, please contact us at [support@kissan.ai].

Downloads last month: 2

Safetensors

Model size

1B params

Tensor type

BF16