Instructions to use gr0010/ArtificialThinker-Phi2 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use gr0010/ArtificialThinker-Phi2 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="gr0010/ArtificialThinker-Phi2", trust_remote_code=True)

# Load model directly
from transformers import AutoModelForCausalLM
model = AutoModelForCausalLM.from_pretrained("gr0010/ArtificialThinker-Phi2", trust_remote_code=True, dtype="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use gr0010/ArtificialThinker-Phi2 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "gr0010/ArtificialThinker-Phi2"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "gr0010/ArtificialThinker-Phi2",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/gr0010/ArtificialThinker-Phi2

SGLang

How to use gr0010/ArtificialThinker-Phi2 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "gr0010/ArtificialThinker-Phi2" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "gr0010/ArtificialThinker-Phi2",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "gr0010/ArtificialThinker-Phi2" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "gr0010/ArtificialThinker-Phi2",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use gr0010/ArtificialThinker-Phi2 with Docker Model Runner:
```
docker model run hf.co/gr0010/ArtificialThinker-Phi2
```

ArtificialThinker-Phi2 / README.md

gr0010

Update README.md

632d749 verified 23 days ago

preview code

Raw

History Blame Contribute Delete

1.25 kB

	---
	license: mit
	inference: false
	datasets:
	- freecs/ArtificialThinkerSet
	base_model: microsoft/phi-2
	---

	# The First Open-Source Reasoning LLM

	December 28, 2023 - This model was created 11 months before OpenAI's o1 release.

	## Historical Context

	In late 2023, I was experimenting with fine-tuning open-source models. Working with limited computational resources (primarily free Colab notebooks with T4 GPUs), I focused on developing novel approaches and new paradigms to significantly enhance LLM capabilities without simply scaling the number of parameters, since that would have required substantial computational resources.

	Proof of timeline: Check the [initial commit](https://huggingface.co/freecs/ArtificialThinker-Phi2/commit/8ce7acd72fb187cd3c3e76a8c0c58b8246e85d23) - December 28, 2023.

	## Technical Approach

	The model uses a custom chat template that includes a "reasoning" step before providing the output to the user:

	```
	<\|system\|>sys_message
	<\|prompt\|>prompt
	<\|reasoning\|>reasoning
	<\|response\|>response<\|endoftext\|>
	```

	To test this approach, I created the [ArtificialThinkerSet](https://huggingface.co/datasets/freecs/ArtificialThinkerSet) dataset to fine-tune Phi-2.

	You can find me at [gr.bio](https://gr.bio/).