Instructions to use nickmalhotra/ProjectIndus with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use nickmalhotra/ProjectIndus with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="nickmalhotra/ProjectIndus")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("nickmalhotra/ProjectIndus")
model = AutoModelForCausalLM.from_pretrained("nickmalhotra/ProjectIndus")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps

vLLM

How to use nickmalhotra/ProjectIndus with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "nickmalhotra/ProjectIndus"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "nickmalhotra/ProjectIndus",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/nickmalhotra/ProjectIndus

SGLang

How to use nickmalhotra/ProjectIndus with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "nickmalhotra/ProjectIndus" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "nickmalhotra/ProjectIndus",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "nickmalhotra/ProjectIndus" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "nickmalhotra/ProjectIndus",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use nickmalhotra/ProjectIndus with Docker Model Runner:
```
docker model run hf.co/nickmalhotra/ProjectIndus
```

zicsx commited on May 1, 2024

Commit

0ebbf98

verified ·

1 Parent(s): b2e94cf

Update README.md

Browse files

Files changed (1) hide show

README.md +20 -2

README.md CHANGED Viewed

@@ -421,9 +421,9 @@ To begin using Project Indus LLM for your projects, follow these steps to set up
 # Load model directly
 ```python
-from transformers import AutoModel, AutoTokenizer
-model = AutoModel.from_pretrained("nickmalhotra/ProjectIndus")
 tokenizer = AutoTokenizer.from_pretrained("nickmalhotra/ProjectIndus")
 # Example inference
@@ -454,3 +454,21 @@ output = model.generate(input_ids,
                         num_return_sequences=1,
                         )
 print(tokenizer.decode(output[0], skip_special_tokens=False))

 # Load model directly
 ```python
+from transformers import AutoModelForCausalLM, AutoTokenizer
+model = AutoModelForCausalLM.from_pretrained("nickmalhotra/ProjectIndus")
 tokenizer = AutoTokenizer.from_pretrained("nickmalhotra/ProjectIndus")
 # Example inference
                         num_return_sequences=1,
                         )
 print(tokenizer.decode(output[0], skip_special_tokens=False))
+## Disclaimer
+#### Model Limitations
+Project Indus LLM is trained with single instruction tuning, which may result in hallucinations—instances where the model generates plausible but inaccurate information. Users should exercise caution, especially in scenarios requiring high factual accuracy.
+#### Adaptation for Specific Use Cases
+Project Indus LLM is designed as a foundational model suitable for further development and fine-tuning. Users are encouraged to adapt and refine the model to meet specific requirements of their applications.
+#### Recommendations for Fine-Tuning
+- **Identify Specific Needs**: Clearly define the requirements of your use case to guide the fine-tuning process.
+- **Curate Targeted Data**: Ensure the training data is relevant and of high quality to improve model performance.
+- **Continuous Evaluation**: Regularly assess the model's performance during and after fine-tuning to maintain accuracy and reduce biases.
+This disclaimer aims to provide users with a clear understanding of the model's capabilities and limitations, facilitating its effective application and development.