Model card v3: step-by-step gated-access walkthrough, separate GPU/CPU quickstarts with hardware requirements, batch processing with tqdm progress bar

README.md

@@ -49,7 +49,20 @@ Practical consequences:
 - Do **not** try to remove/override `quantization_config` to get fp16: the stored weights themselves are 4-bit packed, so there is no full-precision copy in this repo. To obtain higher-precision weights, load 4-bit first and call `model.dequantize()` (see below).
 - VRAM needed is only **~6 GB** — the model fits on small GPUs.
 
-## Quickstart
+## Before you start
+
+**No gated access needed** — unlike the detect and stance stages, this repo is fully self-contained (no Meta base model to download), so there is no license gate, no account, and no token required. The first run downloads ~6 GB with progress bars, cached afterward in `~/.cache/huggingface`.
+
+## Hardware requirements — pick your setup
+
+| Setup | What you need | Speed |
+|---|---|---|
+| **GPU (recommended)** | NVIDIA GPU with ≥8 GB free VRAM, `pip install bitsandbytes` | fast — this is the wiba.dev production configuration |
+| **CPU only** | ~25 GB free RAM, no GPU; loads 4-bit then dequantizes (see below) | ~1–2 min per text on 16 cores |
+
+⚠️ Do **not** run `generate()` directly on the 4-bit model on a CPU: bitsandbytes' CPU 4-bit kernels are single-threaded and a single sentence takes over an hour (measured). Use the dequantize recipe below instead.
+
+## Quickstart — GPU
 
 ```bash
 pip install torch transformers accelerate bitsandbytes
@@ -66,11 +79,22 @@ tokenizer.pad_token_id = tokenizer.eos_token_id
 tokenizer.padding_side = "left"
 
 # quantization_config ships in config.json — transformers loads the 4-bit
-# weights automatically (CUDA GPU recommended, ~6 GB VRAM)
+# weights automatically (~6 GB VRAM)
 model = AutoModelForCausalLM.from_pretrained(REPO, device_map="auto", low_cpu_mem_usage=True)
 model.eval()
 ```
 
+## Quickstart — CPU (no GPU)
+
+`bitsandbytes` is still required (the checkpoint is stored 4-bit), but after loading, dequantize to bfloat16 so generation runs on all CPU cores (verified: ~25 GB RAM peak, then ~1–2 min per text on 16 cores):
+
+```python
+model = AutoModelForCausalLM.from_pretrained(REPO, device_map="cpu", low_cpu_mem_usage=True)
+model = model.dequantize().to(torch.bfloat16)
+model.eval()
+torch.set_num_threads(16)   # match your core count
+```
+
 ### Prompt format (must match training)
 
 The model expects the Llama-3 chat header format with the WIBA topic-extraction system prompt, and the generation cut off after a few tokens (topics are short):
@@ -123,9 +147,20 @@ The original implementation uses the equivalent `pipeline("text-generation", ...
 - An argumentative input → a short topic phrase (e.g. `Climate change`, `Gun control`)
 - A non-argument input → the literal string `No Topic`
 
+## Batch processing many texts (with a progress bar)
+
+Model downloads show progress bars automatically; generation doesn't, so wrap your loop in `tqdm` (installed with transformers) exactly as the original WIBA serving code does:
+
+```python
+from tqdm import tqdm
+
+texts = ["...", "..."]  # your data
+topics = [extract_topic(t) for t in tqdm(texts)]
+```
+
 ## Getting full-precision weights
 
-The repo stores no fp16 copy, but you can dequantize after loading (needs enough memory for the fp16 model, ~16 GB):
+The repo stores no fp16 copy, but you can dequantize after loading (needs enough memory for the fp16 model, ~16 GB — this is the same call the CPU quickstart uses):
 
 ```python
 model = AutoModelForCausalLM.from_pretrained(REPO, device_map="auto")