Text Classification
Transformers
Safetensors
English
bert
finance
ai-detector
sequence-classification
text-embeddings-inference
Instructions to use msperlin/finbert-ai-detector with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use msperlin/finbert-ai-detector with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-classification", model="msperlin/finbert-ai-detector")# Load model directly from transformers import AutoTokenizer, AutoModelForSequenceClassification tokenizer = AutoTokenizer.from_pretrained("msperlin/finbert-ai-detector") model = AutoModelForSequenceClassification.from_pretrained("msperlin/finbert-ai-detector") - Notebooks
- Google Colab
- Kaggle
Enhance README with usage instructions for finbert-ai-detector package and transformers library
Browse files
README.md
CHANGED
|
@@ -42,13 +42,42 @@ This model is intended for researchers, financial analysts, and auditors who wan
|
|
| 42 |
|
| 43 |
## How to Get Started with the Model
|
| 44 |
|
| 45 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 46 |
|
| 47 |
```python
|
| 48 |
import torch
|
| 49 |
from transformers import AutoTokenizer, AutoModelForSequenceClassification
|
| 50 |
|
| 51 |
-
# Load the model and tokenizer
|
| 52 |
model_id = "msperlin/finbert-ai-detector"
|
| 53 |
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
|
| 54 |
|
|
@@ -79,33 +108,6 @@ prob_ai = probabilities[0][1].item()
|
|
| 79 |
print(f"Probability of being AI-generated: {prob_ai:.2%}")
|
| 80 |
```
|
| 81 |
|
| 82 |
-
### Batch Processing Example
|
| 83 |
-
|
| 84 |
-
For larger text datasets, you can generate predictions in batches to properly utilize GPU acceleration:
|
| 85 |
-
|
| 86 |
-
```python
|
| 87 |
-
texts = [
|
| 88 |
-
"Company revenue grew by 15% due to increased demand in the European market.",
|
| 89 |
-
"A machine learning model generated this text based on recent financial statements."
|
| 90 |
-
]
|
| 91 |
-
|
| 92 |
-
inputs = tokenizer(
|
| 93 |
-
texts,
|
| 94 |
-
return_tensors="pt",
|
| 95 |
-
truncation=True,
|
| 96 |
-
max_length=512,
|
| 97 |
-
padding=True
|
| 98 |
-
).to(device)
|
| 99 |
-
|
| 100 |
-
with torch.no_grad():
|
| 101 |
-
outputs = model(**inputs)
|
| 102 |
-
probabilities = torch.nn.functional.softmax(outputs.logits, dim=-1)
|
| 103 |
-
|
| 104 |
-
probs_ai = probabilities[:, 1].tolist()
|
| 105 |
-
for text, prob in zip(texts, probs_ai):
|
| 106 |
-
print(f"AI Probability: {prob:.2%} - Text: {text}")
|
| 107 |
-
```
|
| 108 |
-
|
| 109 |
## Intended Use & Limitations
|
| 110 |
|
| 111 |
- **Intended Usage:** Analyzing formal financial reports, press releases, corporate filings, and similar structured financial disclosures.
|
|
|
|
| 42 |
|
| 43 |
## How to Get Started with the Model
|
| 44 |
|
| 45 |
+
There are two ways to use this model, you can either use a [python package](https://pypi.org/project/finbert-ai-detector/) that handles the model internaly, or the standard transformers library (default for hugging face).
|
| 46 |
+
|
| 47 |
+
### Using Python package `finbert-ai-detector`
|
| 48 |
+
|
| 49 |
+
Install locally using pip:
|
| 50 |
+
|
| 51 |
+
```
|
| 52 |
+
pip install finbert-ai-detector
|
| 53 |
+
```
|
| 54 |
+
|
| 55 |
+
Run it with the following code:
|
| 56 |
+
|
| 57 |
+
```python
|
| 58 |
+
from finbert_ai_detector import FinbertAIDetector
|
| 59 |
+
|
| 60 |
+
# Initialize the detector (downloads the model if not cached)
|
| 61 |
+
detector = FinbertAIDetector()
|
| 62 |
+
|
| 63 |
+
# Example text
|
| 64 |
+
text = "The Tax Cuts and Jobs Act enacted in 2017 in the United States, significantly changed the tax rules applicable to U.S.-domiciled corporations. Changes such as lower corporate tax rates, full expensing for qualified property, taxation of offshore earnings, limitations on interest expense deductions, and changes to the municipal bond tax exemption may impact demand for our products and services."
|
| 65 |
+
|
| 66 |
+
# Predict a single text
|
| 67 |
+
result = detector.predict(text)
|
| 68 |
+
print(f"Prediction: {result['label']}")
|
| 69 |
+
print(f"AI Probability: {result['ai_probability']:.2%}")
|
| 70 |
+
```
|
| 71 |
+
|
| 72 |
+
### Using `transformers` library.
|
| 73 |
+
|
| 74 |
+
Here is a quick example of how to make predictions using Python and PyTorch:
|
| 75 |
|
| 76 |
```python
|
| 77 |
import torch
|
| 78 |
from transformers import AutoTokenizer, AutoModelForSequenceClassification
|
| 79 |
|
| 80 |
+
# Load the model and tokenizer
|
| 81 |
model_id = "msperlin/finbert-ai-detector"
|
| 82 |
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
|
| 83 |
|
|
|
|
| 108 |
print(f"Probability of being AI-generated: {prob_ai:.2%}")
|
| 109 |
```
|
| 110 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 111 |
## Intended Use & Limitations
|
| 112 |
|
| 113 |
- **Intended Usage:** Analyzing formal financial reports, press releases, corporate filings, and similar structured financial disclosures.
|