Instructions to use OpenModel-Labs/Comet-1-Preview with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use OpenModel-Labs/Comet-1-Preview with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="OpenModel-Labs/Comet-1-Preview", trust_remote_code=True)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoModelForCausalLM
model = AutoModelForCausalLM.from_pretrained("OpenModel-Labs/Comet-1-Preview", trust_remote_code=True, device_map="auto")

COMET

How to use OpenModel-Labs/Comet-1-Preview with COMET:

# No code snippets available yet for this library.

# To use this model, check the repository files and the library's documentation.

# Want to help? PRs adding snippets are welcome at:
# https://github.com/huggingface/huggingface.js

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use OpenModel-Labs/Comet-1-Preview with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "OpenModel-Labs/Comet-1-Preview"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "OpenModel-Labs/Comet-1-Preview",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/OpenModel-Labs/Comet-1-Preview

SGLang

How to use OpenModel-Labs/Comet-1-Preview with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "OpenModel-Labs/Comet-1-Preview" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "OpenModel-Labs/Comet-1-Preview",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "OpenModel-Labs/Comet-1-Preview" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "OpenModel-Labs/Comet-1-Preview",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use OpenModel-Labs/Comet-1-Preview with Docker Model Runner:
```
docker model run hf.co/OpenModel-Labs/Comet-1-Preview
```

Here

by Bc-AI - opened 3 days ago

Discussion

Bc-AI

OpenModel Labs org 3 days ago

@Hoglet-33 here its here

Bc-AI

OpenModel Labs org 3 days ago

Tell it to try improve code and math and just overall😭

Hoglet-33

OpenModel Labs org 3 days ago

@Bc-AI I need the tokenizer btw

Hoglet-33

OpenModel Labs org 3 days ago

and config.json

Hoglet-33

OpenModel Labs org 3 days ago

but i cant do anything without the tokenizer

Bc-AI

OpenModel Labs org 3 days ago

oh ok

Bc-AI

OpenModel Labs org 3 days ago

uh @Hoglet-33 what do you need in config.json?

Bc-AI

OpenModel Labs org 3 days ago

@Hoglet-33 it uses the EleutherAI/gpt-neox-20b, but i put a copy of the tokenizer.json in too

Bc-AI

OpenModel Labs org 3 days ago

training code is also in the repo now

Bc-AI

OpenModel Labs org 3 days ago

so is config @Hoglet-33

Bc-AI

OpenModel Labs org 1 day ago

@Hoglet-33 ready pls wegith edit now lol

Hoglet-33

OpenModel Labs org 1 day ago

yessir

Hoglet-33

OpenModel Labs org 1 day ago

soo it turns out the model is soo big it cant actually download the repo and edit the weights just veiw them so it made some stuff that you can to maybe make it better??