Initial release: swik-heuristic-v1 keyword classifier with benchmarks

README.md

@@ -0,0 +1,166 @@
+---
+license: cc-by-4.0
+language:
+- en
+tags:
+- text-classification
+- sentiment-analysis
+- financial-sentiment
+- finance
+- commodities
+- domain-specific
+- rule-based
+- interpretable
+pretty_name: swik Heuristic Sentiment v1
+---
+
+# swik-heuristic-v1 (v0.1)
+
+**Deterministic keyword-based financial sentiment classifier.**
+Fast, interpretable, no GPU, no API key. A baseline for domain-specific financial news sentiment.
+
+This is the Layer 1 model in swik's two-layer inference pipeline. It processes every request before
+any LLM call — both as a fast path for high-confidence cases and as a fallback when the API is unavailable.
+
+## What it does
+
+Two-pass classification:
+1. **Inversion check** — matches asset-specific inversion phrases (e.g., "production cut" → BULLISH for OIL)
+2. **Keyword scan** — matches generic bullish/bearish keyword lists
+
+If neither pass fires, the label is `neutral`.
+
+## Keyword Lists
+
+**Bullish (14 terms):** cut, surge, rally, record high, growth, beat, upgrade, rise, gain, boost, strong, exceed, recovery, rebound
+
+**Bearish (13 terms):** crash, plunge, drop, fall, miss, downgrade, warning, decline, loss, weak, below, cut guidance, layoff
+
+**Inversions:** Asset-specific phrase overrides from the [swik inversion catalog](https://swik.io/inversions) (125 active entries). Published separately as a dataset.
+
+## Usage
+
+```python
+from inference import SwikHeuristicV1
+
+model = SwikHeuristicV1()
+
+# Basic usage
+result = model.predict("Oil surges after OPEC production cut")
+# {'label': 'bullish', 'magnitude': 0.72, 'confidence': 0.45, 'method': 'keyword'}
+
+# With inversion catalog
+inversions = [
+    {"phrase": "coal power", "direction": "BULLISH", "variants": ["coal-fired power"]},
+    {"phrase": "production cut", "direction": "BULLISH"},
+]
+model_with_inv = SwikHeuristicV1(known_inversions=inversions)
+result = model_with_inv.predict("Coal power demand rises as gas prices surge", security="NATGAS")
+# {'label': 'bullish', ..., 'inversion_applied': 'coal power', 'method': 'inversion'}
+```
+
+## Benchmark Results
+
+Evaluated on matched corpus: inference_log vs community_labels_legacy (text_hash join), 2026-03-08 to 2026-03-29.
+
+| Metric | heuristic-v1 | haiku-4-5 (baseline) | haiku-4-5 (variant B) |
+|--------|-------------|---------------------|----------------------|
+| **Accuracy** | **98.88%** | 39.6% | 46.0% |
+| **F1 macro** | **0.981** | 0.309 | 0.456 |
+| Neutral F1 | 0.992 | 0.506 | — |
+| Bullish F1 | 0.970 | 0.231 | — |
+| Bearish F1 | 0.981 | 0.189 | — |
+| n (pairs) | 13,966 | 16,141 | 200 (test set) |
+
+> ⚠️ **Important:** These benchmarks are measured against AI-generated labels (Claude Haiku), not
+> human ground truth. The high heuristic accuracy reflects agreement with the labeling model, not
+> necessarily alignment with human judgment. Human-label benchmarks are pending.
+>
+> ⚠️ **Known dataset bias:** The companion labeled dataset is OIL-dominant — OIL accounts for ~56% of all
+> labeled records. Model performance on other securities (especially low-volume ones) may be significantly
+> lower than the aggregate numbers suggest. Evaluate per-security before deploying.
+
+## Confidence Calibration
+
+The heuristic outputs a fixed confidence of `0.45` for all predictions. This is intentional —
+unlike the Haiku baseline (which is anti-calibrated: higher confidence → higher error rate),
+the heuristic makes no claim about certainty. Use it as a deterministic rule engine, not a
+probabilistic model.
+
+## Known Failure Modes
+
+1. **Ambiguous generic terms**: Words like "cut" appear in both bullish (supply cuts → oil bullish)
+   and neutral contexts (budget cuts, interest rate cuts). Without the inversion catalog, these
+   will be mis-labeled.
+
+2. **Multi-entity headlines**: "Oil falls as dollar rises" — the heuristic detects "falls" (bearish)
+   but may assign it to the wrong security if entity filtering is weak.
+
+3. **Negation blindness**: "Oil did NOT surge" → misclassified as bullish. No negation handling.
+
+4. **Language and spelling**: English only. Abbreviations and misspellings not handled.
+
+5. **Context window**: Heuristic has no memory of prior sentences. Each text is classified in
+   isolation.
+
+## Model Weights
+
+**This model has no neural network weights.** It is a deterministic rule-based system (keyword lists + inversion catalog).
+
+- No fine-tuning. No LoRA adapter. No PyTorch/TensorFlow required.
+- Labels in the companion dataset were generated by Claude Haiku (claude-haiku-4-5 via API) — not by a local model.
+- A LoRA fine-tuned adapter is planned once the community label corpus reaches sufficient size and multi-labeler consensus.
+
+## Architecture Context
+
+This model is Layer 1 in swik's inference pipeline:
+
+```
+Text Input
+    ↓
+[heuristic-v1]  ← this model
+    ↓ layer1_score
+if security ∈ [OIL, NATGAS, LNG, GOLD, EURUSD]: use heuristic output
+else if relevance < threshold: use heuristic output
+else:
+    ↓
+[claude-haiku-4-5 + inversion catalog]  ← Layer 2
+    ↓
+Final prediction
+```
+
+For OIL, NATGAS, LNG, GOLD, EURUSD: the heuristic is the final model (accuracy ~99% on these).
+For other securities: heuristic pre-screens, Haiku runs if relevance passes.
+
+## Training Data
+
+Not trained. Deterministic rule-based system. Keyword lists were derived from:
+- Manual curation of financial news vocabulary
+- Error analysis on the swik inference corpus
+- Cross-validated against community labels
+
+## Dataset
+
+Labels used for benchmarking: [polibert/swik-sentiment-labels](https://huggingface.co/datasets/polibert/swik-sentiment-labels)
+
+## License
+
+CC BY 4.0
+
+## Citation
+
+```bibtex
+@misc{swik_heuristic_v1_2026,
+  title={swik-heuristic-v1: Domain-Specific Financial Sentiment Classifier},
+  author={swik Community},
+  year={2026},
+  url={https://huggingface.co/polibert/swik-heuristic-v1},
+  license={CC BY 4.0}
+}
+```
+
+## Links
+
+- Platform: [swik.io](https://swik.io)
+- Dataset: [polibert/swik-sentiment-labels](https://huggingface.co/datasets/polibert/swik-sentiment-labels)
+- Contribute labels: [swik.io/contribute/label](https://swik.io/contribute/label)

inference.py

@@ -0,0 +1,107 @@
+#!/usr/bin/env python3
+"""
+swik-heuristic-v1 — deterministic keyword-based financial sentiment classifier.
+
+A fast, interpretable baseline for domain-specific financial news sentiment.
+No GPU required. No API calls. Runs in microseconds.
+
+Usage:
+    from inference import SwikHeuristicV1, KNOWN_INVERSIONS
+    model = SwikHeuristicV1()
+    result = model.predict("OPEC agrees to production cuts", security="OIL")
+    # {"label": "bullish", "magnitude": 0.72, "confidence": 0.45, "method": "keyword"}
+
+For inversion-aware inference, pass known_inversions (list of dicts with phrase/direction).
+"""
+
+BULLISH_KEYWORDS = [
+    "cut", "surge", "rally", "record high", "growth", "beat", "upgrade",
+    "rise", "gain", "boost", "strong", "exceed", "recovery", "rebound"
+]
+BEARISH_KEYWORDS = [
+    "crash", "plunge", "drop", "fall", "miss", "downgrade", "warning",
+    "decline", "loss", "weak", "below", "cut guidance", "layoff"
+]
+
+LABEL_MAP = {"bullish": 0, "bearish": 1, "neutral": 2, "irrelevant": 3}
+LABEL_NAMES = ["bullish", "bearish", "neutral", "irrelevant"]
+
+
+class SwikHeuristicV1:
+    """
+    Two-pass keyword classifier:
+      Pass 1: Check known inversions (asset-specific phrase overrides)
+      Pass 2: Check generic bullish/bearish keyword lists
+      Default: neutral
+
+    Accuracy: 98.88% on matched inference corpus vs AI labels (n=13,966).
+    Note: measured against AI-generated labels, not human ground truth.
+    """
+
+    def __init__(self, known_inversions=None):
+        """
+        known_inversions: list of dicts with keys:
+            phrase (str), direction (str: BULLISH|BEARISH|NEUTRAL),
+            variants (list[str], optional), confidence (float, optional)
+        """
+        self.known_inversions = known_inversions or []
+
+    def predict(self, text: str, security: str = None, key_entities: list = None) -> dict:
+        text_lower = text.lower()
+        direction = "neutral"
+        magnitude = 0.4
+        relevance = 0.5
+        inversion_applied = None
+
+        # Pass 1: known inversions (highest priority)
+        for inv in self.known_inversions:
+            phrase = inv["phrase"].lower()
+            variants = [v.lower() for v in inv.get("variants", [])]
+            if phrase in text_lower or any(v in text_lower for v in variants):
+                direction = inv["direction"].lower()
+                magnitude = float(inv.get("confidence", 0.7))
+                relevance = 0.85
+                inversion_applied = inv["phrase"]
+                break
+
+        # Pass 2: generic keywords
+        if not inversion_applied:
+            if any(kw in text_lower for kw in BULLISH_KEYWORDS):
+                direction = "bullish"
+                magnitude = 0.72
+                relevance = 0.75
+            elif any(kw in text_lower for kw in BEARISH_KEYWORDS):
+                direction = "bearish"
+                magnitude = 0.68
+                relevance = 0.75
+
+        # Boost relevance if key entities mentioned
+        if key_entities:
+            for entity in key_entities:
+                if entity.lower() in text_lower:
+                    relevance = min(1.0, relevance + 0.15)
+                    break
+
+        return {
+            "label": direction,
+            "label_id": LABEL_MAP.get(direction, 2),
+            "magnitude": round(magnitude, 2),
+            "relevance": round(relevance, 2),
+            "confidence": 0.45,  # heuristic confidence is always 0.45
+            "inversion_applied": inversion_applied,
+            "method": "inversion" if inversion_applied else ("keyword" if direction != "neutral" else "default"),
+        }
+
+    def predict_batch(self, texts: list, security: str = None, key_entities: list = None) -> list:
+        return [self.predict(t, security, key_entities) for t in texts]
+
+
+if __name__ == "__main__":
+    import sys
+    model = SwikHeuristicV1()
+    text = " ".join(sys.argv[1:]) if len(sys.argv) > 1 else "OPEC agrees to production cuts, oil surges"
+    result = model.predict(text)
+    print(f"Text:   {text}")
+    print(f"Label:  {result['label']} (id={result['label_id']})")
+    print(f"Magnitude: {result['magnitude']} | Relevance: {result['relevance']} | Confidence: {result['confidence']}")
+    print(f"Method: {result['method']}")