Update README.md

README.md

@@ -1,10 +1,98 @@
 ---
 license: mit
 datasets:
-- clement-cvll/QWQ-LongCOT-AIMO
+- Floppanacci/QWQ-LongCOT-AIMO
 base_model:
-- clement-cvll/DeepSeek-R1-Distill-Qwen-7B-Floppanacci
+- Floppanacci/DeepSeek-R1-Distill-Qwen-7B-Floppanacci
 pipeline_tag: text-generation
 tags:
 - math
----
+- qwen2.5
+- aimo
+language:
+- en
+---
+
+# DeepSeek-R1-Distill-Qwen-7B-Floppanacci (4-bit AWQ Quantized)
+
+This repository contains the 4-bit AWQ (Activation-aware Weight Quantization) version of the [`Floppanacci/DeepSeek-R1-Distill-Qwen-7B-Floppanacci`](https://huggingface.co/Floppanacci/DeepSeek-R1-Distill-Qwen-7B-Floppanacci) model.
+
+## Model Description
+
+This model is optimized for faster inference and lower memory footprint compared to the original bf16/fp16 fine-tuned model. It's designed for mathematical reasoning tasks, especially Chain-of-Thought style problem-solving relevant to the [AIMO competition](https://www.kaggle.com/competitions/ai-mathematical-olympiad-progress-prize-2).
+
+The original model was fine-tuned on the [`Floppanacci/QWQ-LongCOT-AIMO`](https://huggingface.co/datasets/Floppanacci/QWQ-LongCOT-AIMO) dataset.
+
+## How to Use
+
+### With `transformers` (and `autoawq`)
+
+You need to install the `autoawq` library:
+```bash
+pip install autoawq transformers torch
+```
+
+Then use the model with `transformers`:
+```python
+from transformers import AutoModelForCausalLM, AutoTokenizer
+import torch
+
+model_id = "Floppanacci/DeepSeek-R1-Distill-Qwen-7B-Floppanacci-AWQ"
+tokenizer = AutoTokenizer.from_pretrained(model_id)
+# Load the AWQ quantized model
+model = AutoModelForCausalLM.from_pretrained(
+    model_id,
+    device_map="auto" # Automatically uses available GPU(s)
+)
+
+# Example Prompt (adjust based on how the model expects input)
+prompt = "Question: Let $ABCD$ be a unit square. Let $P$ be a point inside the square such that $PA = \sqrt{5}/3$, $PB = \sqrt{2}/3$, and $PC = \sqrt{5}/3$. Find the distance $PD$. Answer:"
+inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
+
+# Generate
+outputs = model.generate(**inputs, max_new_tokens=300, temperature=0.1, do_sample=False) # Example settings
+response = tokenizer.decode(outputs[0], skip_special_tokens=True)
+
+print(response)
+
+```
+
+### With `vLLM` (Optimized Inference)
+
+For higher throughput and optimized inference, you can use vLLM.
+
+First, install vLLM:
+```bash
+pip install vllm
+```
+
+Then run the following Python code:
+```python
+from vllm import LLM, SamplingParams
+
+# Define prompts
+prompts = [
+    "Question: Let $ABCD$ be a unit square. Let $P$ be a point inside the square such that $PA = \sqrt{5}/3$, $PB = \sqrt{2}/3$, and $PC = \sqrt{5}/3$. Find the distance $PD$. Answer:",
+    "Question: What is the sum of the first 100 positive integers? Answer:",
+]
+
+# Define sampling parameters
+sampling_params = SamplingParams(temperature=0.1, top_p=0.95, max_tokens=300)
+
+# Initialize the LLM engine with the AWQ model
+llm = LLM(model="Floppanacci/DeepSeek-R1-Distill-Qwen-7B-Floppanacci-AWQ",
+          quantization="awq",
+          dtype="auto", # vLLM will typically use half-precision for activations (use bfloat16 on compatible hardware e.g. L4, A100, H100, etc.)
+          trust_remote_code=True
+         )
+
+# Generate responses
+outputs = llm.generate(prompts, sampling_params)
+
+# Print the outputs
+for output in outputs:
+    prompt = output.prompt
+    generated_text = output.outputs[0].text
+    print(f"Prompt: {prompt!r}, Generated text: {generated_text!r}")
+
+```