README.md

---
license: mit
tags:
- text-classification
- multi-label-classification
- crypto
- technology
- twitter
- x
- roberta
pipeline_tag: text-classification
---

# Techpto Transformer

RoBERTa multi-label classifier for detecting whether X/Twitter posts or profile text are crypto-related, tech-related, both, or neither.

This is the V5B transformer checkpoint trained for the Techpto classifier project. It was later distilled into the faster `techpto-classifier` hashed linear model for full-corpus scanning.

## Labels

- `crypto`
- `tech`

The model uses sigmoid probabilities, not softmax. A text can match neither label, one label, or both labels.

## Files

- `model.safetensors`: transformer weights.
- `config.json`: `RobertaForSequenceClassification` config with `problem_type = multi_label_classification`.
- `tokenizer.json` and `tokenizer_config.json`: tokenizer files.
- `metrics.json`: full training/eval metrics.
- `classification_thresholds.json`: recommended threshold sets.

## Recommended Thresholds

For high precision on the test split:

```json
{
  "crypto": 0.80,
  "tech": 0.86
}
```

For best F1 on the test split:

```json
{
  "crypto": 0.41,
  "tech": 0.57
}
```

For higher recall / F2:

```json
{
  "crypto": 0.12,
  "tech": 0.16
}
```

## Test Metrics

At the higher-recall thresholds stored in `metrics.json`:

- Exact match accuracy: `0.9071`
- Micro F1: `0.9200`
- Macro F1: `0.9099`

At the high-precision threshold set:

- Exact match accuracy: `0.9305`
- Micro precision: `0.9704`
- Micro recall: `0.8873`
- Micro F1: `0.9270`
- Macro F1: `0.9186`

## Usage

```python
import torch
from transformers import AutoModelForSequenceClassification, AutoTokenizer

repo_id = "pompompur-in/techpto-transformer"
tokenizer = AutoTokenizer.from_pretrained(repo_id)
model = AutoModelForSequenceClassification.from_pretrained(repo_id)
model.eval()

text = "Building a new AI agent workflow for crypto wallet monitoring."
inputs = tokenizer(text, return_tensors="pt", truncation=True, max_length=192)

with torch.no_grad():
    logits = model(**inputs).logits[0]
    probs = torch.sigmoid(logits)

labels = ["crypto", "tech"]
thresholds = {"crypto": 0.80, "tech": 0.86}
predictions = {
    label: float(prob) >= thresholds[label]
    for label, prob in zip(labels, probs)
}

print(dict(zip(labels, map(float, probs))))
print(predictions)
```

## Notes

This checkpoint is intended for classification/research workflows over social text. It is not a general-purpose language model.