Upload 9 files

Browse files

Files changed (10) hide show

.gitattributes +8 -0
README.md +95 -177
gguf/acnoryx-0.8b-iq1_m.gguf +3 -0
gguf/acnoryx-0.8b-iq1_s.gguf +3 -0
gguf/acnoryx-0.8b-iq2_m.gguf +3 -0
gguf/acnoryx-0.8b-iq2_xs.gguf +3 -0
gguf/acnoryx-0.8b-iq2_xxs.gguf +3 -0
gguf/acnoryx-0.8b-iq3_m.gguf +3 -0
gguf/acnoryx-0.8b-q2_k.gguf +3 -0
gguf/acnoryx-0.8b-q3_k_m.gguf +3 -0

.gitattributes CHANGED Viewed

@@ -46,3 +46,11 @@ gguf/airy-q2_k_s.gguf filter=lfs diff=lfs merge=lfs -text
 gguf/airy-q2_k.gguf filter=lfs diff=lfs merge=lfs -text
 gguf/airy-tq1_0.gguf filter=lfs diff=lfs merge=lfs -text
 gguf/airy-tq2_0.gguf filter=lfs diff=lfs merge=lfs -text

 gguf/airy-q2_k.gguf filter=lfs diff=lfs merge=lfs -text
 gguf/airy-tq1_0.gguf filter=lfs diff=lfs merge=lfs -text
 gguf/airy-tq2_0.gguf filter=lfs diff=lfs merge=lfs -text
+gguf/acnoryx-0.8b-iq1_m.gguf filter=lfs diff=lfs merge=lfs -text
+gguf/acnoryx-0.8b-iq1_s.gguf filter=lfs diff=lfs merge=lfs -text
+gguf/acnoryx-0.8b-iq2_m.gguf filter=lfs diff=lfs merge=lfs -text
+gguf/acnoryx-0.8b-iq2_xs.gguf filter=lfs diff=lfs merge=lfs -text
+gguf/acnoryx-0.8b-iq2_xxs.gguf filter=lfs diff=lfs merge=lfs -text
+gguf/acnoryx-0.8b-iq3_m.gguf filter=lfs diff=lfs merge=lfs -text
+gguf/acnoryx-0.8b-q2_k.gguf filter=lfs diff=lfs merge=lfs -text
+gguf/acnoryx-0.8b-q3_k_m.gguf filter=lfs diff=lfs merge=lfs -text

README.md CHANGED Viewed

@@ -1,177 +1,95 @@
----
-language:
-  - vi
-  - en
-license: apache-2.0
-tags:
-  - gguf
-  - qwen3
-  - quantization
-  - llama-cpp
-  - acne
-  - skincare
-base_model: Qwen/Qwen3.5-0.8B
-model_type: qwen3
-pipeline_tag: text-generation
-library_name: gguf
----
-# Acnoryx/Airy GGUFs
-GGUF pack for the fine-tuned Acnoryx acne-care assistant model, prepared for direct publishing under `Acnoryx/Airy`.
-- Hugging Face repo: `Acnoryx/Airy`
-- Product app: [Acnoryx - AI for Acne Care](https://play.google.com/store/apps/details?id=com.fivecanh.acnoryx)
-## Important scope
-This repo is a **research snapshot**: it explores how far the model can be compressed while still keeping acceptable acne-domain quality.
-It is **not** the production model used inside the Acnoryx app; the app may use different quantization, prompt format, runtime settings, and evaluation criteria.
-## What this repo contains
-This repo focuses on a small set of high-value research GGUFs for Acnoryx/Airy. The most important published candidates are listed in the evaluation table below.
-## Model Quality Summary
-The tables below group the models by practical quality. Models with <50% thinking accuracy are separated into a “Low-quality / not recommended” section so the “Core + Airy” table stays focused on the usable candidates.
-### Core (release floor)
-| Model | Size | Thinking Pass | Thinking % | Non-Think Pass | Non-Think % | Think t/s | Non-Think t/s |
-|---|---:|---:|---:|---:|---:|---:|---:|
-| `acnoryx-0.8b-f16` | 1446 MB | 99/100 | 99.0% | 100/100 | 100.0% | 44 | 44 |
-| `acnoryx-0.8b-q8_0` | 774 MB | 98/100 | 98.0% | 98/100 | 98.0% | 69 | 69 |
-| `acnoryx-0.8b-q4_k_m` | 505 MB | 98/100 | 98.0% | 100/100 | 100.0% | 75 | 80 |
-| `acnoryx-0.8b-iq4_nl` | 493 MB | 99/100 | 99.0% | 100/100 | 100.0% | 93 | 93 |
-| `acnoryx-0.8b-iq4_xs` | 482 MB | 99/100 | 99.0% | 98/100 | 98.0% | 92 | 92 |
-| `acnoryx-0.8b-q3_k_m` | 445 MB | 99/100 | 99.0% | 97/100 | 97.0% | 96 | 95 |
-| `acnoryx-0.8b-iq3_m` | 433 MB | 91/100 | 91.0% | 96/100 | 96.0% | 88 | 88 |
-### Airy candidates (acceptable quality)
-| Model | Size | Thinking Pass | Thinking % | Non-Think Pass | Non-Think % | Think t/s | Non-Think t/s |
-|---|---:|---:|---:|---:|---:|---:|---:|
-| `airy-iq3_s.gguf` | 415.6 MB | 100/100 | 100% | 100/100 | 100% | ~20 | ~20 |
-| `airy-iq3_xs.gguf` | 408.4 MB | 100/100 | 100% | 98/100 | 98% | ~19 | ~19 |
-| `airy-q2_k.gguf` | 377.4 MB | 95/100 | 95% | 97/100 | 97% | ~23 | ~23 |
-| `airy-q2_k_s.gguf` | 370.2 MB | 98/100 | 98% | 93/100 | 93% | ~20 | ~20 |
-| `airy-iq2_s.gguf` | 320.2 MB | 78/100 | 78% | 81/100 | 81% | ~21 | ~21 |
-| `airy-iq2_xs.gguf` | 317.5 MB | 82/100 | 82% | 74/100 | 74% | ~23 | ~23 |
-### Low-quality / not recommended (thinking < 50%)
-These models are included for transparency but are not recommended for deployment due to very low thinking accuracy.
-| Model | Size | Thinking Pass | Thinking % | Non-Think Pass | Non-Think % | Think t/s | Non-Think t/s |
-|---|---:|---:|---:|---:|---:|---:|---:|
-| `airy-iq2_xxs.gguf` | 303.1 MB | 49/100 | 49% | 50/100 | 50% | ~21 | ~21 |
-| `airy-iq2_m.gguf` | 334.3 MB | 44/100 | 44% | 48/100 | 48% | ~22 | ~22 |
-| `airy-tq1_0.gguf` | 311.4 MB | 11/100 | 11% | 10/100 | 10% | (high) | (high) |
-| `airy-iq1_m.gguf` | 285.5 MB | 14/100 | 14% | 14/100 | 14% | ~30 | ~30 |
-| `airy-iq1_s.gguf` | 275.0 MB | 7/100 | 7% | 6/100 | 6% | ~30 | ~30 |
-## Key Takeaways
-| Finding | Notes |
-|---|---|
-| `airy-iq3_s.gguf` is the best overall file | Highest quality, matches or exceeds reference floor while being smaller.
-| `airy-q2_k.gguf` is the best value | Largest size reduction while keeping high accuracy.
-| `airy-q2_k_s.gguf` is the best aggressive option | Very high thinking accuracy with a modest non-thinking drop.
-| `airy-iq2_s.gguf` and `airy-iq2_xs.gguf` are the lower usable edge | These are the smallest usable models before quality drops sharply.
-| `airy-iq2_xxs.gguf` and below are not reliable | Performance and accuracy degrade too far for typical deployment.
-| Some files lack final benchmark records | `airy-iq3_xxs.gguf` and `airy-tq2_0.gguf` were generated but not merged into final evaluation.
-## Recommendations
-| Recommendation | Model(s) | Why |
-|---|---|---|
-| Default (publish) | `airy-iq3_s.gguf` | Best quality for minimal size; safest drop-in.
-| Best compression/value | `airy-q2_k.gguf` | Great size reduction with minimal quality loss.
-| Aggressive small | `airy-q2_k_s.gguf` | Strong thinking mode; OK non-thinking.
-| Smallest still usable | `airy-iq2_s.gguf` | Lowest model that retains practical utility.
-| Archive / not recommended | `airy-iq2_xxs.gguf`, `airy-iq2_m.gguf`, `airy-tq1_0.gguf`, `airy-iq1_m.gguf`, `airy-iq1_s.gguf` | Included for transparency but not for deployment.
-## How To Run
-### 1. Install Python packages
-```bash
-pip install huggingface_hub llama-cpp-python
-```
-`huggingface_hub` is used to download the GGUF file from Hugging Face.
-`llama-cpp-python` is used to actually load and run the GGUF model in Python.
-### 2. Download a GGUF from Hugging Face in Python
-```python
-from huggingface_hub import hf_hub_download
-model_path = hf_hub_download(
-  repo_id="Acnoryx/Airy",
-  filename="airy-iq3_s.gguf",
-  local_dir="gguf",
-)
-print(model_path)
-```
-### 3. Run inference in Python
-```python
-from huggingface_hub import hf_hub_download
-from llama_cpp import Llama
-model_path = hf_hub_download(
-  repo_id="Acnoryx/Airy",
-  filename="airy-iq3_s.gguf",
-  local_dir="gguf",
-)
-llm = Llama(
-  model_path=model_path,
-  n_ctx=4096,
-  n_gpu_layers=-1,
-  verbose=False,
-)
-result = llm.create_chat_completion(
-  messages=[
-    {"role": "system", "content": "You are Acnoryx AI, a dermatology assistant focused on acne and skincare."},
-    {"role": "user", "content": "What are blackheads?"},
-  ],
-  temperature=0.2,
-)
-print(result["choices"][0]["message"]["content"])
-```
-### 4. Swap model files quickly
-Just change the `filename` value:
-- `airy-iq3_s.gguf`
-- `airy-q2_k.gguf`
-- `airy-iq2_s.gguf`
-### 5. Quick pick guide
-- Choose `airy-iq3_s.gguf` for the strongest overall result.
-- Choose `airy-q2_k.gguf` for the best compression/value balance.
-- Choose `airy-iq2_s.gguf` only if you need a much smaller file and can accept a visible quality drop.
-## Final Conclusion
-The current Airy lineup proves that the model can go below the old release-floor size while still keeping strong acne-domain quality.
-The best files in this repo are:
-- `airy-iq3_s.gguf`
-- `airy-iq3_xs.gguf`
-- `airy-q2_k.gguf`
-- `airy-q2_k_s.gguf`
-Among them, `airy-iq3_s.gguf` is the safest default publish choice, while `airy-q2_k.gguf` is the best efficiency result.

+---
+license: apache-2.0
+language:
+  - vi
+  - en
+tags:
+  - acne
+  - skincare
+  - dermatology
+  - gguf
+  - qwen3
+base_model: Qwen/Qwen3.5-0.8B
+pipeline_tag: text-generation
+---
+# Acnoryx/Airy — Research GGUF Bundle
+Research & evaluation companion to the main release. Contains sub-4-bit quantizations
+(<4-bit) for low-memory benchmarking on the **0.8B** model.
+## Model Details
+| | |
+|-|-|
+| **Base model** | Qwen/Qwen3.5-0.8B (752M params, FLA + Triton) |
+| **Fine-tune** | SFT on 30,007 acne/skincare/dermatology samples |
+| **Training** | 4 epochs, batch=2, grad_acc=8, lr=5e-5 |
+| **Training loss** | 0.1217 (eval_loss: 0.0876) |
+| **Languages** | Vietnamese, English |
+| **Domain** | Acne analysis, skincare routines, scan interpretation |
+| **Identity** | Acnoryx AI — in-app dermatology assistant |
+## Research Quantization Results
+Tested with 100 domain-specific questions × 2 modes (thinking / non-thinking).
+All quantizations in this bundle are **sub-4-bit (<4-bit)**, ordered high-to-low bit depth.
+| Quant | Size | Thinking | Non-Think | Avg | Status |
+|-------|------|----------|-----------|-----|--------|
+| **Q3_K_M** | 549 MB | **92%** | **94%** | **93.0%** | ✅ Excellent |
+| **IQ3_M** | 537 MB | **76%** | **75%** | **75.5%** | ⚠️ Usable but degraded |
+| **Q2_K** | 482 MB | 54% | 58% | 56.0% | ⚠️ Marginal |
+| **IQ2_M** | 439 MB | 10% | 12% | 11.0% | ❌ Not usable |
+| **IQ2_XS** | 397 MB | 0% | 0% | 0.0% | ❌ Skipped (early-stop) |
+| **IQ2_XXS** | 383 MB | 0% | 0% | 0.0% | ❌ Skipped (early-stop) |
+| **IQ1_M** | 365 MB | 0% | 0% | 0.0% | ❌ Skipped (early-stop) |
+| **IQ1_S** | 355 MB | 0% | 0% | 0.0% | ❌ Skipped (early-stop) |
+### imatrix
+IQ2/IQ1 quants were generated with importance matrix (imatrix) calibration from
+a 44KB domain-specific corpus. The 0.8B model holds at Q3_K_M (93%) and IQ3_M (75.5%),
+but falls sharply at IQ2 and below.
+## Full Quantization Map (Release + Research)
+Combined view across all quantizations for the 0.8B model, ordered by bit depth (high → low):
+| Quant | Size | Thinking | Non-Think | Avg | Bundle |
+|-------|------|----------|-----------|-----|--------|
+| F16 | 1932 MB | 94% | 93% | 93.5% | Release |
+| Q8_0 | 1032 MB | 89% | 90% | 89.5% | Release |
+| Q5_K_M | 718 MB | 90% | 86% | 88.0% | Release |
+| Q4_K_M | 641 MB | 89% | 92% | 90.5% | Release |
+| Q4_0 | 615 MB | 91% | 90% | 90.5% | Release |
+| IQ4_NL | 630 MB | 90% | 90% | 90.0% | Release |
+| IQ4_XS | 611 MB | 89% | 92% | 90.5% | Release |
+| **Q3_K_M** | **549 MB** | **92%** | **94%** | **93.0%** | **Research** |
+| **IQ3_M** | **537 MB** | **76%** | **75%** | **75.5%** | **Research** |
+| **Q2_K** | **482 MB** | **54%** | **58%** | **56.0%** | **Research** |
+| **IQ2_M** | **439 MB** | **10%** | **12%** | **11.0%** | **Research** |
+| **IQ2_XS** | **397 MB** | **0%** | **0%** | **0.0%** | **Research** |
+| **IQ2_XXS** | **383 MB** | **0%** | **0%** | **0.0%** | **Research** |
+| **IQ1_M** | **365 MB** | **0%** | **0%** | **0.0%** | **Research** |
+| **IQ1_S** | **355 MB** | **0%** | **0%** | **0.0%** | **Research** |
+### Key findings
+- **Standout research quant: Q3_K_M (549 MB, 93%)** — ties F16 quality at 28% the size
+- **IQ3_M (537 MB, 75.5%)** still usable — proof of 0.8B's superior quantization resilience
+- **Q2_K (56%)** marginal but non-zero — the 0.8B handles 2-bit far better than 0.6B (0%)
+- IQ2 and below are not viable despite imatrix calibration
+- **0.8B vs 0.6B:** At each quant level, 0.8B scores significantly higher
+## Usage
+```bash
+# llama.cpp — Q3_K_M is the research highlight
+./llama-cli -m acnoryx-0.8b-q3_k_m.gguf -cnv -p "Xin chào"
+```
+## Related
+- **Release bundle:** Production quantizations (F16 → IQ4_XS, ≥4-bit)
+- **0.6B research:** [Acnoryx/Airy-Lite](https://huggingface.co/Acnoryx/Airy-Lite) — smaller model with compact footprint

gguf/acnoryx-0.8b-iq1_m.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:bc41e7f81876d15a5f037c132695e5f3e8062fa8fd1c50cb9d98b00b0704ea3a
+size 382832064

gguf/acnoryx-0.8b-iq1_s.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:98d6475f5b4ed6f89274e05757ae901bbd08059fef006dfb233b2fe929ab8785
+size 371795904

gguf/acnoryx-0.8b-iq2_m.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:3c0da92dea1fd8d75312a9583fab13c0048a49ee23e917e672ac67656391de67
+size 459761600

gguf/acnoryx-0.8b-iq2_xs.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:7ef3524cdf46e85211ab39291b8d6d47000d468ee9c38e15bbd75f9225010c02
+size 416333760

gguf/acnoryx-0.8b-iq2_xxs.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:449bc10952982f127a074496d5ad3ae430392bdab70feddebffc119e1bcadbf0
+size 401225664

gguf/acnoryx-0.8b-iq3_m.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:9739c3cb690a42ac38673db9c8a97d1fc96078980967dc8f089f42ac6355c843
+size 563208896

gguf/acnoryx-0.8b-q2_k.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:54e6af18be0acf65bfaed296bacc7d6b7cf6e9bada977a393c29e3305790110d
+size 505755840

gguf/acnoryx-0.8b-q3_k_m.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:c0d14483d825fea21cad63b294d582f98d294e813d13b99d21c1caffae9d5fd9
+size 575476416