harshwardhanjadhav/llama3.2-1b-CoT

Llama-3.2-1B-CoT (OpenThoughts-10k)

Llama-3.2-1B-CoT is a fine-tuned version of unsloth/Llama-3.2-1B-Instruct trained on a curated 10,000-sample subset of the OpenThoughts-114k dataset. It has been trained to generate structured, step-by-step internal reasoning traces before presenting final answers to complex mathematical and analytical problems.

• Developed by: harshwardhanjadhav
• Model Type: Causal Language Model (Fine-tuned for Chain-of-Thought / Systemic Reasoning)
• Base Model: unsloth/Llama-3.2-1B-Instruct
• License: Apache 2.0
• Fine-Tuning Framework: Unsloth & Hugging Face TRL

---

💡 Model Capabilities & Prompt Structure

This model is fine-tuned to parse multi-step problems by breaking down its internal analysis into distinct <|begin_of_thought|> and <|begin_of_solution|> blocks.

Expected System Prompt

To trigger the Chain-of-Thought pathway correctly, provide the following system prompt in your chat template:

Your role as an assistant involves thoroughly exploring questions through a systematic long thinking process before providing the final precise and accurate solutions.
This requires engaging in a comprehensive cycle of analysis, summarizing, exploration, reassessment, reflection, backtracing, and iteration to develop well-considered thinking process.
Please structure your response into two main sections: Thought and Solution.
In the Thought section, detail your reasoning process using the specified format: <|begin_of_thought|> {thought with steps separated with '\n\n'} <|end_of_thought|> Each step should include detailed considerations such as analisying questions, summarizing relevant findings, brainstorming new ideas, verifying the accuracy of the current steps, refining any errors, and revisiting previous steps.
In the Solution section, based on various attempts, explorations, and reflections from the Thought section, systematically present the final solution that you deem correct. The solution should remain a logical, accurate, concise expression style and detail necessary step needed to reach the conclusion, formatted as follows: <|begin_of_solution|> {final formatted, precise, and clear solution} <|end_of_solution|> Now, try to solve the following question through the above guidelines:

⚡ Quickstart & Usage⚙️ Recommended Inference Hyperparameters[!IMPORTANT]Because 1B parameter reasoning models are prone to infinite repetition during extended thinking phases, repetition_penalty=1.15 and temperature=0.1 are strictly required for output stability. ParameterRecommended ValueReasontemperature0.1Ensures deterministic, logically consistent CoT tracesrepetition_penalty1.15Prevents infinite loops inside long reasoning blockstop_p0.9Filters out low-probability token tailsmax_new_tokens4096Accommodates lengthy multi-step reasoningPython Inference Code (Unsloth) Pythonimport torch from unsloth import FastLanguageModel from transformers import TextStreamer

1. Load Model & Tokenizer

    model_name="harshwardhanjadhav/llama3.2-1b-CoT",
    max_seq_length=8192,
    dtype=torch.float16,
    load_in_4bit=False,
)
FastLanguageModel.for_inference(model)

2. Define Messages

    {
        "role": "system",
        "content": "Your role as an assistant involves thoroughly exploring questions through a systematic long thinking process before providing the final precise and accurate solutions. Enclose your internal reasoning within <|begin_of_thought|> and <|end_of_thought|> tags."
    },
    {
        "role": "user",
        "content": "A store offers a 20% discount on a $150 coat, then takes off an additional 10% from the sale price. What is the final price? Think step by step."
    }
]

inputs = tokenizer.apply_chat_template(
    messages,
    tokenize=True,
    add_generation_prompt=True,
    return_tensors="pt"
).to("cuda")

3. Stream Output

_ = model.generate(
    input_ids=inputs,
    streamer=streamer,
    max_new_tokens=4096,
    temperature=0.1,
    repetition_penalty=1.15,
    top_p=0.9,
    use_cache=True,
)

This llama model was trained 2x faster with Unsloth and Huggingface's TRL library.

📊 Training DetailsDataset: 10,000 samples from open-thoughts/OpenThoughts-114kSequence Length: 8,192 tokensPrecision: Mixed Precision (fp16 / bf16)Optimization: Fine-tuned 2x faster using Unsloth GPU kernels⚠️ Limitations & Usage AdviceParameter Scale: Being a 1B parameter model, it may struggle with highly abstract mathematical theorems or multi-nested algorithmic code generation compared to 8B+ or 70B models.Context Window: Keep total context (Prompt + Thinking Trace + Solution) under 8,192 tokens.