Feature Extraction
Transformers
Safetensors
English
distilbert
question-answering
extractive-qa
squad
fine-tuned
text-embeddings-inference
Instructions to use harpertoken/clue with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use harpertoken/clue with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("feature-extraction", model="harpertoken/clue")# pip install -U transformers accelerate # Load model directly from transformers import AutoTokenizer, AutoModelForQuestionAnswering tokenizer = AutoTokenizer.from_pretrained("harpertoken/clue") model = AutoModelForQuestionAnswering.from_pretrained("harpertoken/clue", device_map="auto") - Notebooks
- Google Colab
- Kaggle
Delete files clue.md with huggingface_hub
Browse files
clue.md
DELETED
|
@@ -1,52 +0,0 @@
|
|
| 1 |
-
---
|
| 2 |
-
license: mit
|
| 3 |
-
language: en
|
| 4 |
-
library_name: transformers
|
| 5 |
-
pipeline_tag: feature-extraction
|
| 6 |
-
base_model: distilbert-base-uncased
|
| 7 |
-
datasets:
|
| 8 |
-
- squad
|
| 9 |
-
tags:
|
| 10 |
-
- distilbert
|
| 11 |
-
- extractive-qa
|
| 12 |
-
- question-answering
|
| 13 |
-
- squad
|
| 14 |
-
- fine-tuned
|
| 15 |
-
---
|
| 16 |
-
|
| 17 |
-
# clue
|
| 18 |
-
|
| 19 |
-
A short continued-fine-tuning run of [`quiz`](https://huggingface.co/harpertoken/quiz), itself a DistilBERT encoder adapted for extractive question answering on SQuAD. The architecture and tokenizer are identical; only the weights differ. Where `quiz` reflects a full training pass, this checkpoint reflects roughly a thousand SQuAD examples seen once, which makes it a useful small-scale reference point and a poor substitute for a properly trained model.
|
| 20 |
-
|
| 21 |
-
Training used a learning rate of 2e-5 at batch size one for a single epoch, in float32. The published weights are `model.safetensors`. The `config.json` previously carried a key `tie_weights_`, which no version of Transformers reads; it has been removed, and nothing else in the config was altered.
|
| 22 |
-
|
| 23 |
-
## Usage
|
| 24 |
-
|
| 25 |
-
Transformers 5 removed the `question-answering` pipeline, so load the model directly:
|
| 26 |
-
|
| 27 |
-
```python
|
| 28 |
-
import torch
|
| 29 |
-
from transformers import AutoModelForQuestionAnswering, AutoTokenizer
|
| 30 |
-
|
| 31 |
-
tok = AutoTokenizer.from_pretrained("harpertoken/clue")
|
| 32 |
-
model = AutoModelForQuestionAnswering.from_pretrained("harpertoken/clue")
|
| 33 |
-
|
| 34 |
-
question = "Who wrote Hamlet?"
|
| 35 |
-
context = "Hamlet is a tragedy written by William Shakespeare around 1600."
|
| 36 |
-
inputs = tok(question, context, return_tensors="pt", truncation=True, max_length=512)
|
| 37 |
-
|
| 38 |
-
with torch.inference_mode():
|
| 39 |
-
out = model(**inputs)
|
| 40 |
-
start, end = int(out.start_logits.argmax()), int(out.end_logits.argmax())
|
| 41 |
-
print(tok.decode(inputs.input_ids[0][start : end + 1]))
|
| 42 |
-
```
|
| 43 |
-
|
| 44 |
-
On the three questions used to check `quiz` — the capital of France, the author of Hamlet, and the year the Eiffel Tower was completed — this checkpoint returns `paris`, `william shakespeare` and `1889`, the same answers. A thousand examples has not visibly degraded it, which is itself a reason to doubt that the fine-tuning taught much.
|
| 45 |
-
|
| 46 |
-
## Limitations
|
| 47 |
-
|
| 48 |
-
A thousand examples is a demonstration of the fine-tuning mechanics rather than a trained model, and the documentation this replaced claimed SQuAD exact-match and F1 figures that were never produced by an evaluation. Treat this as a low-fidelity copy of `quiz`. It is English-only, inherits the same uncased tokenisation and SQuAD domain bias described in the `quiz` card, and shares its 512-token limit. Compare the two directly before assuming the fine-tuning helped.
|
| 49 |
-
|
| 50 |
-
## Attribution
|
| 51 |
-
|
| 52 |
-
DistilBERT follows Sanh et al. (2019); SQuAD follows Rajpurkar et al. (2016).
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|