Model card v3: step-by-step gated-access walkthrough, separate GPU/CPU quickstarts with hardware requirements, batch processing with tqdm progress bar

README.md

@@ -47,11 +47,39 @@ Because the trained `score` head ships inside the adapter file, loading this ada
 
 > **Checkpoint format note:** the adapter was originally trained and saved with PEFT 0.7.1, whose `score`-head layout cannot be loaded by modern PEFT (≥0.10 raises `KeyError: 'base_model.model.score.weight'`). The files on `main` were converted to the modern format (trained head merged as `base_layer + (alpha/r)·B·A`) and verified **logit-equivalent to the original, on both the modern stack (peft 0.19.1) and the original stack (peft 0.7.1)** — `main` works everywhere. The original-format files are preserved at `revision="937b9babeb146587b5a9463b239ae4ca6ad26e18"`.
 
-## Quickstart
+## Before you start: get access to the gated Meta base model (one-time, ~10 minutes)
+
+This adapter repo is freely downloadable, but the Meta base model it sits on is **gated** — Meta requires you to accept their license before you can download it. Step by step:
+
+1. **Create a Hugging Face account** (free): go to [huggingface.co/join](https://huggingface.co/join), sign up, and verify your email.
+2. **Request access to the base model**: while logged in, open [meta-llama/Llama-2-7b-hf](https://huggingface.co/meta-llama/Llama-2-7b-hf). At the top of the page is a box saying you need to share your contact information to access the model. Fill in the short form, accept the license, and submit.
+3. **Wait for the approval email** — usually minutes to a few hours. When the box on the model page changes to "You have been granted access", you're in.
+4. **Create an access token**: click your avatar (top right) → **Settings** → **Access Tokens** → **Create new token** → type **Read** → create, and **copy the token** (it looks like `hf_...`). Treat it like a password.
+5. **Log in on your computer**: in a terminal run
+
+   ```bash
+   pip install -U "huggingface_hub[cli]"
+   huggingface-cli login
+   ```
+
+   and paste the token when prompted (nothing is shown as you paste — that's normal). Verify with `huggingface-cli whoami`, which should print your username.
+
+This is once per computer. From then on, the code below downloads everything it needs automatically — you'll see progress bars for each file on the first run (~13.6 GB total), after which everything is cached in `~/.cache/huggingface` and loads from disk.
+
+## Hardware requirements — pick your setup
+
+| Setup | What you need | Speed |
+|---|---|---|
+| **GPU, fp16** | NVIDIA GPU with ≥15 GB free VRAM (e.g. RTX 4090, A100; 16 GB cards work) | sub-second per text |
+| **GPU, 4-bit** | NVIDIA GPU with ≥6 GB free VRAM, plus `pip install bitsandbytes` | fast — this is the wiba.dev production configuration |
+| **CPU only** | ~30 GB free RAM, no GPU | ~15–25 s per text on 16 cores — fine for trying it out, slow for bulk work |
+
+One-time download for any setup: ~13.6 GB (base model + adapter).
+
+## Quickstart — GPU
 
 ```bash
 pip install torch transformers peft accelerate sentencepiece
-huggingface-cli login   # base model is gated
 ```
 
 ```python
@@ -71,6 +99,34 @@ model = PeftModel.from_pretrained(base, ADAPTER)
 model.eval()
 ```
 
+**Low VRAM? Load the base 4-bit instead** (≈5 GB VRAM, the production setting — needs `pip install bitsandbytes`):
+
+```python
+from transformers import BitsAndBytesConfig
+
+bnb_config = BitsAndBytesConfig(
+    load_in_4bit=True,
+    bnb_4bit_quant_type="nf4",
+    bnb_4bit_use_double_quant=False,
+    bnb_4bit_compute_dtype=torch.float16,
+)
+base = AutoModelForSequenceClassification.from_pretrained(
+    BASE, num_labels=3, device_map="auto", quantization_config=bnb_config
+)
+```
+
+## Quickstart — CPU (no GPU)
+
+Identical to the GPU code, except load the base in float32 on the CPU:
+
+```python
+base = AutoModelForSequenceClassification.from_pretrained(
+    BASE, num_labels=3, dtype=torch.float32, device_map="cpu"
+)
+```
+
+Expect ~15–25 s per prediction on a 16-core machine (verified). Make sure you have ~30 GB of free RAM before starting — on machines without swap, overshooting RAM can freeze the system.
+
 ### Prompt format (must match training)
 
 The model uses the Llama-2 instruction wrapper with the WIBA argument-definition system prompt (the same system prompt as the detect stage), and takes **both the target topic and the text**:
@@ -127,25 +183,23 @@ print(classify_stance("climate change", "The weather is nice today."))
 
 The repo tokenizer's `<unk>` pad token (id 0) is in-vocabulary, so batched inference with `padding=True` works as-is.
 
-## Memory-efficient loading (production configuration)
+## Batch processing many texts (with a progress bar)
 
-The WIBA platform serves the 7B base 4-bit quantized (~4–5 GB VRAM instead of ~14 GB fp16). Requires a CUDA GPU and `pip install bitsandbytes`:
+Model downloads show progress bars automatically; inference doesn't, so wrap batches in `tqdm` (installed with transformers) exactly as the original WIBA serving code does. The repo's `<unk>` pad token works for batching as-is:
 
 ```python
-from transformers import BitsAndBytesConfig
+from tqdm import tqdm
+from transformers import pipeline
 
-bnb_config = BitsAndBytesConfig(
-    load_in_4bit=True,
-    bnb_4bit_quant_type="nf4",
-    bnb_4bit_use_double_quant=False,
-    bnb_4bit_compute_dtype=torch.float16,
-)
-base = AutoModelForSequenceClassification.from_pretrained(
-    BASE, num_labels=3, device_map="auto", quantization_config=bnb_config
-)
-```
+clf = pipeline("text-classification", model=model, tokenizer=tokenizer,
+               padding=True, truncation=True, max_length=2048)
 
-For batched inference, use a `text-classification` pipeline exactly as the original implementation does: `pipeline(task="text-classification", model=model, tokenizer=tokenizer, padding=True, truncation=True, max_length=2048)`.
+pairs = [("climate change", "..."), ("gun control", "...")]  # (topic, text) pairs
+prompts = [f"[INST] <<SYS>>\n{SYSTEM_PROMPT}\n<</SYS>>\n\nTarget: '{topic}' Text: '{text}' [/INST] "
+           for topic, text in pairs]
+idx = {"LABEL_0": "No Argument", "LABEL_1": "Argument in Favor", "LABEL_2": "Argument Against"}
+labels = [idx[out["label"]] for out in tqdm(clf(prompts, batch_size=4), total=len(prompts))]
+```
 
 ## Tested configurations