Instructions to use leeroo/LeerooDedicated-Math-7b with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use leeroo/LeerooDedicated-Math-7b with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="leeroo/LeerooDedicated-Math-7b", trust_remote_code=True)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("leeroo/LeerooDedicated-Math-7b", trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained("leeroo/LeerooDedicated-Math-7b", trust_remote_code=True)

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use leeroo/LeerooDedicated-Math-7b with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "leeroo/LeerooDedicated-Math-7b"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "leeroo/LeerooDedicated-Math-7b",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/leeroo/LeerooDedicated-Math-7b

SGLang

How to use leeroo/LeerooDedicated-Math-7b with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "leeroo/LeerooDedicated-Math-7b" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "leeroo/LeerooDedicated-Math-7b",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "leeroo/LeerooDedicated-Math-7b" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "leeroo/LeerooDedicated-Math-7b",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use leeroo/LeerooDedicated-Math-7b with Docker Model Runner:
```
docker model run hf.co/leeroo/LeerooDedicated-Math-7b
```

AAllii commited on Apr 2, 2024

Commit

4bab141

verified ·

1 Parent(s): cb2ab29

Update README.md

Browse files

Files changed (1) hide show

README.md +1 -6

README.md CHANGED Viewed

@@ -29,8 +29,6 @@ model_inputs = encodeds['input_ids'].to(device)
 generated_ids = model.generate(model_inputs, max_new_tokens=100, do_sample=False)
 decoded = tokenizer.batch_decode(generated_ids)
 print(decoded[0])
-# '<s> Natalia sold clips to 48 of her friends in April,and then she sold half as
-# many clips in May.How many clips did Natalia sell altogether in April and May?\n\n
 # Natalia sold 48 clips in April.\nIn May, she sold half as many clips as in April,
 # so she sold 48/2 = 24 clips.\nAltogether, Natalia sold 48 + 24 = 72 clips in April and May.\n#### 72\nThe answer is: 72</s>'
@@ -41,10 +39,7 @@ model_inputs = encodeds['input_ids'].to(device)
 generated_ids = model.generate(model_inputs, max_new_tokens=100, do_sample=False)
 decoded = tokenizer.batch_decode(generated_ids)
 print(decoded[0])
-# '<s> James loves to go swimming and has to swim across a 20-mile lake.  He can swim
-# at a pace of 2 miles per hour.  He swims 60% of the distance.  After that, he stops
-# on an island and rests for half as long as the swimming time.  He then finishes the
-# remaining distance while going half the speed.  How long did it take him to get across the lake?<GPT4></s>'
 ```
 ## Learn More

 generated_ids = model.generate(model_inputs, max_new_tokens=100, do_sample=False)
 decoded = tokenizer.batch_decode(generated_ids)
 print(decoded[0])
 # Natalia sold 48 clips in April.\nIn May, she sold half as many clips as in April,
 # so she sold 48/2 = 24 clips.\nAltogether, Natalia sold 48 + 24 = 72 clips in April and May.\n#### 72\nThe answer is: 72</s>'
 generated_ids = model.generate(model_inputs, max_new_tokens=100, do_sample=False)
 decoded = tokenizer.batch_decode(generated_ids)
 print(decoded[0])
+# <GPT4></s>'
 ```
 ## Learn More