Instructions to use cookinai/LlamaReflect-8B-CoT with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use cookinai/LlamaReflect-8B-CoT with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="cookinai/LlamaReflect-8B-CoT")

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("cookinai/LlamaReflect-8B-CoT")
model = AutoModelForCausalLM.from_pretrained("cookinai/LlamaReflect-8B-CoT", device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use cookinai/LlamaReflect-8B-CoT with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "cookinai/LlamaReflect-8B-CoT"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "cookinai/LlamaReflect-8B-CoT",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/cookinai/LlamaReflect-8B-CoT

SGLang

How to use cookinai/LlamaReflect-8B-CoT with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "cookinai/LlamaReflect-8B-CoT" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "cookinai/LlamaReflect-8B-CoT",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "cookinai/LlamaReflect-8B-CoT" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "cookinai/LlamaReflect-8B-CoT",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Unsloth Desktop
Docker Model Runner
How to use cookinai/LlamaReflect-8B-CoT with Docker Model Runner:
```
docker model run hf.co/cookinai/LlamaReflect-8B-CoT
```
Browse Quantizations to use this model in llama.cpp, Ollama, LM Studio, or any compatible app.

Llama 3 finetuned on my TRRR-CoT Dataset

cookinai/TRRR-CoT

This was an attempt at synthetically generating a CoT dataset and then finetuning it on a model to see its reuslts.
From what I notice, when using the correct prompt template the model almost always ues the TRRR format, but I am still awaiting benchmark tests to see if this can improve anything
TRR stand for:

Think, about your response
Respond, how you normally would
Reflect, on your response
Respond, again but this time use all the information you have now

The mode usually tries to follow this format, it may mix it up a little but usually it almost always reflects in someway. Especially if you tell it to think step by step
Intrestingly enough, when finetuned on mistral 7b, I could not get the model CoT at all, with only one epoch llama 3 got it instantly
Developed by: cookinai
License: apache-2.0
Finetuned from model : unsloth/llama-3-8b-bnb-4bit

This llama model was trained 2x faster with Unsloth and Huggingface's TRL library.

Downloads last month: 9

Model tree for cookinai/LlamaReflect-8B-CoT

Base model

meta-llama/Meta-Llama-3-8B

Quantized

unsloth/llama-3-8b-bnb-4bit

Finetuned

(3114)

this model

Quantizations

1 model