---
license: mit
inference: false
datasets:
  - freecs/ArtificialThinkerSet
base_model: microsoft/phi-2
---

# The First Open-Source Reasoning LLM

**December 28, 2023** - This model was created 11 months before OpenAI's o1 release.

## Historical Context

In late 2023, I was experimenting with fine-tuning open-source models. Working with limited computational resources (primarily free Colab notebooks with T4 GPUs), I focused on developing novel approaches and new paradigms to significantly enhance LLM capabilities without simply scaling the number of parameters, since that would have required substantial computational resources.

**Proof of timeline:** Check the [initial commit](https://huggingface.co/freecs/ArtificialThinker-Phi2/commit/8ce7acd72fb187cd3c3e76a8c0c58b8246e85d23) - December 28, 2023.

## Technical Approach

The model uses a custom chat template that includes a "reasoning" step before providing the output to the user:

```
<|system|>sys_message
<|prompt|>prompt
<|reasoning|>reasoning
<|response|>response<|endoftext|>
```

To test this approach, I created the [ArtificialThinkerSet](https://huggingface.co/datasets/freecs/ArtificialThinkerSet) dataset to fine-tune Phi-2.

I also wrote ["Reasoning Is All You Need"](https://freecs.org/paper.html) - a blog post explaining this approach.

You can find me at [gr.bio](https://gr.bio/).