cyberknine/bau-qwen

🚀 BAU-Small-1.5B (ADTC 2026 Official Submission)

BAU-Small-1.5B is an edge-optimized, fine-tuned, and calibrated GGUF language model built for the Africa Deep Tech Challenge (ADTC) 2026 Laptop LLM track.

It is designed to run 100% offline with zero cloud dependencies on commodity 8 GB RAM laptops, serving as an offline business co-pilot for African Small and Medium Enterprises (SMEs), retail pharmacies, and local warehouses.

---

📌 Model Overview

• Base Model: Qwen/Qwen2.5-1.5B-Instruct
• Fine-Tuning: LoRA SFT on 10,000 synthetic SME enterprise interaction pairs (retail operations, inventory discrepancy logging, cashier audit trails, rota scheduling).
• Quantization: IQ3_XS (3.3 bits per weight) calibrated via llama-imatrix.
• Model File Size: ~698 MB (bau-small-1.5b.gguf)
• Peak Operational Memory: < 895 MB RAM (Leaves > 6.1 GB free on an 8 GB budget laptop).
• Embedded Prompt Engine: Custom Jinja chat template baked directly into the GGUF metadata for intelligent routing between plain business chat and structured JSON schemas.

---

🎯 Supported Capabilities & Schemas

The model automatically routes between natural business text and structured JSON payloads based on user query intent:

Schema / Mode	Description	Example Trigger
CONVERSATIONAL_CHAT	Natural language business advice, calculations, memos, and operational strategy.	"How can our retail store reduce cashier discrepancy losses during peak hours?"
GENERATIVE_CHART	Autonomous data extraction into Chart.js-compatible JSON schemas (pie, bar, line).	"Generate a chart of pallet SKU distribution across warehouse zones: Zone A has 400..."
DEEP_RESEARCH	Forensic multi-source investigation across local DBs and audit records.	"Investigate the audit logs to find why Q3 software license costs exceeded forecast by 25%."
SHIFT_SCHEDULE	Automated staff rota generation for pharmacists and warehouse technicians.	"Create a shift schedule for next week for 3 pharmacists: Dr. Sarah, Mr. David..."
RED_FLAG_ALERT	Anomaly detection and security alerts for unauthorized discounts and inventory loss.	"Alert management: Cashier #104 processed an unauthorized 80% discount without approval."
AUTO_TASK	Action item and ticket dispatching for store managers.	"Create a high-priority follow-up task to review cashier refund overrides by tomorrow."
DOCUMENT_OUTPUT	Formal markdown reports, SLA breach logs, and compliance checklists.	"Prepare a detailed SLA breach report for the network downtime incident."

---

⚡ Quickstart & Local Inference

1. Run with llama.cpp (CLI)

# Clone and build llama.cpp
git clone https://github.com/ggml-org/llama.cpp
cd llama.cpp && cmake -B build && cmake --build build -j4

# Download model
curl -L -o bau-small-1.5b.gguf "https://huggingface.co/cyberknine/bau-qwen/resolve/main/bau-small-1.5b.gguf"

# Run interactive inference (4 threads for standard laptops)
./build/bin/llama-cli \
  -m ./bau-small-1.5b.gguf \
  -t 4 \
  -c 2048 \
  -n 384 \
  --temp 0.2 \
  -p "How can our retail store reduce cashier discrepancy losses during peak hours without slowing down checkout queues?"

2. Python (llama-cpp-python)

from llama_cpp import Llama

llm = Llama(
    model_path="bau-small-1.5b.gguf",
    n_ctx=2048,
    n_threads=4,
    verbose=False
)

response = llm.create_chat_completion(
    messages=[
        {"role": "user", "content": "Generate a chart showing the distribution of pallet SKU types across our warehouse zones: Zone A has 400 pallets, Zone B has 350 pallets, Zone C has 150 pallets, and Zone D has 100 pallets."}
    ],
    temperature=0.2,
    max_tokens=384
)

print(response["choices"][0]["message"]["content"])

---

📊 ADTC 2026 Reference Hardware Benchmarks

Evaluated against the ADTC Standard Laptop Profile (Intel Core i5 / AMD Ryzen 5, 8 GB RAM, CPU-only, Ubuntu Linux):

Benchmark Metric	Measured Result	Reference Target
Peak RAM (RSS)	892.4 MB	< 7.0 GB budget
Time to First Token (TTFT)	~510 ms	< 1,000 ms
Generation Throughput	3.76 t/s (4 threads CPU)	15.0 t/s reference
Prompt Processing Speed	9.10 t/s	—
Thermal Throttling	None observed (Temp < 55°C)	Zero penalty
Schema Accuracy Score ($S_{acc}$)	95.20 / 100	> 90 / 100

---

📜 License & Acknowledgements

• License: Apache 2.0
• Submission Track: Africa Deep Tech Challenge 2026 — Laptop LLM Track
• Team ID: cyberknine (Akhimien Clement)