Upload README SIMPLE METRICS.md

README SIMPLE METRICS.md

@@ -0,0 +1,28 @@
+# Project: PrettyBird BCE Basic Coder 8B
+
+## Project Overview
+This project involves fine-tuning the `Qwen/Qwen2.5-Coder-7B-Instruct` model using the Unsloth library. The goal was to produce a high-performance coding assistant. Following the fine-tuning process, the model was converted into GGUF format with multiple quantization levels to optimize for various deployment scenarios.
+
+## Methodology
+- **Fine-Tuning Framework**: Unsloth (LoRA adapters)
+- **Base Model**: Qwen/Qwen2.5-Coder-7B-Instruct
+- **Inference Engine**: Ollama
+- **Hardware**: NVIDIA A100 GPU
+
+## Quantitative Results
+The models were benchmarked for inference speed (Tokens Per Second) using the `api/generate` raw mode to ensure consistent evaluation.
+
+| Model Tag | Quantization | Mean TPS | Speedup vs Baseline |
+| :--- | :--- | :--- | :--- |
+| f16 | Full Precision | ~97.0 | 1.0x |
+| q8_0 | 8-bit | ~141.0 | 1.45x |
+| q5_k_m | 5-bit | ~151.0 | 1.56x |
+| q4_k_m | 4-bit | ~161.0 | 1.66x |
+| q2_k | 2-bit | ~140.0 | 1.44x |
+
+## Performance Analysis
+- **Sweet Spot**: The `q4_k_m` model demonstrated the highest throughput, achieving approximately 161 TPS. It represents the optimal balance between speed and precision, offering a ~1.7x speedup over the full-precision baseline.
+- **The 2-bit Anomaly**: The `q2_k` model, despite being the most compressed, performed slower than the 4-bit and 5-bit variants (~140 TPS). This counter-intuitive result is attributed to the computational overhead required to dequantize highly compressed weights on-the-fly, which creates a bottleneck on high-performance hardware like the NVIDIA A100.
+
+## Recommendations
+For production deployment, the **`q4_k_m`** model is recommended as the primary candidate due to its superior throughput and efficient memory usage. The **`q5_k_m`** model serves as a high-fidelity alternative if slightly higher reasoning precision is required.