Instructions to use IvmeLabs/Ivme-Conversate-v2-Base with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use IvmeLabs/Ivme-Conversate-v2-Base with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="IvmeLabs/Ivme-Conversate-v2-Base", trust_remote_code=True)

# Load model directly
from transformers import AutoModelForCausalLM
model = AutoModelForCausalLM.from_pretrained("IvmeLabs/Ivme-Conversate-v2-Base", trust_remote_code=True, device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use IvmeLabs/Ivme-Conversate-v2-Base with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "IvmeLabs/Ivme-Conversate-v2-Base"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "IvmeLabs/Ivme-Conversate-v2-Base",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/IvmeLabs/Ivme-Conversate-v2-Base

SGLang

How to use IvmeLabs/Ivme-Conversate-v2-Base with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "IvmeLabs/Ivme-Conversate-v2-Base" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "IvmeLabs/Ivme-Conversate-v2-Base",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "IvmeLabs/Ivme-Conversate-v2-Base" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "IvmeLabs/Ivme-Conversate-v2-Base",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use IvmeLabs/Ivme-Conversate-v2-Base with Docker Model Runner:
```
docker model run hf.co/IvmeLabs/Ivme-Conversate-v2-Base
```

ereniko commited on Jul 7

Commit

e7b0ce9

verified ·

1 Parent(s): 4dd335d

Update README.md

Browse files

Files changed (1) hide show

README.md +1 -11

README.md CHANGED Viewed

@@ -97,17 +97,7 @@ None of the benchmarks above directly test whether the model's writing holds tog
 **Prompt:** "Once upon a time, there was a"
-> Once upon a time, there was a wise old turtle named Timmy who lived on the coast of South America. In this magical place, no matter how big or large, people could look up at the sea and talk to each other.
-Timmy asked, "What do you mean, Timmy?"
-The turtle replied, "Well, I think you might see people talking about ocean creatures. They are like little waves that carry their voices. Sometimes they say they're too big or small to hear."
-After a few moments, Timmy had an idea. "Can we go on a boat-boat tour? I can't believe all the kids in the village are doing that!"
-As they sailed further, they saw many beautiful islands and vibrant colors. Each island had its unique culture and traditions. When they reached the top, they saw a group of kids playing and splashing around.
-"Wow, lookgies!" said Timmy. "They live in a big ocean full of colorful fish and
 ---

 **Prompt:** "Once upon a time, there was a"
+> Once upon a time, there was a wise old turtle named Timmy who lived on the coast of South America. In this magical place, no matter how big or large, people could look up at the sea and talk to each other. Timmy asked, "What do you mean, Timmy?" The turtle replied, "Well, I think you might see people talking about ocean creatures. They are like little waves that carry their voices. Sometimes they say they're too big or small to hear." After a few moments, Timmy had an idea. "Can we go on a boat-boat tour? I can't believe all the kids in the village are doing that!" As they sailed further, they saw many beautiful islands and vibrant colors. Each island had its unique culture and traditions. When they reached the top, they saw a group of kids playing and splashing around. "Wow, lookgies!" said Timmy. "They live in a big ocean full of colorful fish and
 ---