Instructions to use IndexTeam/Index-1.9B-Character with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use IndexTeam/Index-1.9B-Character with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="IndexTeam/Index-1.9B-Character", trust_remote_code=True)
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoModelForCausalLM
model = AutoModelForCausalLM.from_pretrained("IndexTeam/Index-1.9B-Character", trust_remote_code=True, dtype="auto")

Notebooks
Google Colab
Kaggle
Local Apps

vLLM

How to use IndexTeam/Index-1.9B-Character with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "IndexTeam/Index-1.9B-Character"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "IndexTeam/Index-1.9B-Character",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/IndexTeam/Index-1.9B-Character

SGLang

How to use IndexTeam/Index-1.9B-Character with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "IndexTeam/Index-1.9B-Character" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "IndexTeam/Index-1.9B-Character",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "IndexTeam/Index-1.9B-Character" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "IndexTeam/Index-1.9B-Character",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use IndexTeam/Index-1.9B-Character with Docker Model Runner:
```
docker model run hf.co/IndexTeam/Index-1.9B-Character
```

Index-1.9B-Character

Commit History

Update config.json

5d9e9b6
verified

AsirAsir commited on Jun 27, 2024

Upload 2 files

ad32cb1
verified

AsirAsir commited on Jun 26, 2024

Update README.md

acd3f70
verified

zhouying commited on Jun 14, 2024

Update README.md

4607f52
verified

zhouying commited on Jun 14, 2024

Update README.md

3607624
verified

koushuicao commited on Jun 14, 2024

Update config.json

bcd8846
verified

AsirAsir commited on Jun 13, 2024

Update README.md

07aa72d
verified

BitVoyage commited on Jun 13, 2024

init

be43718

root commited on Jun 13, 2024

initial commit

b0feb0e
verified

BitVoyage commited on Jun 12, 2024

Commit History

Update config.json 5d9e9b6 verified

Upload 2 files ad32cb1 verified

Update README.md acd3f70 verified

Update README.md 4607f52 verified

Update README.md 3607624 verified

Update config.json bcd8846 verified

Update README.md 07aa72d verified

init be43718

initial commit b0feb0e verified