latent-lab/larger-than-truth-bitnet-2b

lmprobe: Linear Probe on bitnet-b1.58-2B-4T

Truth probe for 'X is larger than Y' statements. Near-perfect accuracy (99.5%) — structural/relational knowledge survives ternary quantization.

Classes

• 0: false_statement
• 1: true_statement

Usage

from lmprobe import LinearProbe

probe = LinearProbe.from_hub("latent-lab/larger-than-truth-bitnet-2b", trust_classifier=True)
predictions = probe.predict(["your text here"])

Probe Details

• Base model: microsoft/bitnet-b1.58-2B-4T
• Model revision: 04c3b9ad9361b824064a1f25ea60a8be9599b127
• Layers: all (0–29, 30 layers)
• Pooling: last_token
• Classifier: logistic_regression
• Task: classification
• Random state: 42

Evaluation

Metric	Value
accuracy	0.9949
auroc	1.0000
f1	0.9949
precision	1.0000
recall	0.9899

Training Data

• Positive examples: 792

• Negative examples: 792

• Positive hash: sha256:4037705894f0a698a994bea48700aec726e8a67ae753295bf6fbf59339f3e4b1

• Negative hash: sha256:5212981b2ba107e1d5e4cc77e851ba81f91b56433b6f30fa6f02902b0d23399a

• Evaluation samples: 396

• Evaluation hash: sha256:dfff0237de05b45c8239ea3db1737da926b3b46da0969b663ae370d85b0611cc

Reproducibility

• lmprobe version: 0.5.8
• Python: 3.12.3
• PyTorch: 2.10.0+cu128
• scikit-learn: 1.8.0
• transformers: 5.3.0