harpertoken commited on
Commit
bb5698f
·
verified ·
1 Parent(s): 19c7b68

Delete files clue.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. clue.md +0 -52
clue.md DELETED
@@ -1,52 +0,0 @@
1
- ---
2
- license: mit
3
- language: en
4
- library_name: transformers
5
- pipeline_tag: feature-extraction
6
- base_model: distilbert-base-uncased
7
- datasets:
8
- - squad
9
- tags:
10
- - distilbert
11
- - extractive-qa
12
- - question-answering
13
- - squad
14
- - fine-tuned
15
- ---
16
-
17
- # clue
18
-
19
- A short continued-fine-tuning run of [`quiz`](https://huggingface.co/harpertoken/quiz), itself a DistilBERT encoder adapted for extractive question answering on SQuAD. The architecture and tokenizer are identical; only the weights differ. Where `quiz` reflects a full training pass, this checkpoint reflects roughly a thousand SQuAD examples seen once, which makes it a useful small-scale reference point and a poor substitute for a properly trained model.
20
-
21
- Training used a learning rate of 2e-5 at batch size one for a single epoch, in float32. The published weights are `model.safetensors`. The `config.json` previously carried a key `tie_weights_`, which no version of Transformers reads; it has been removed, and nothing else in the config was altered.
22
-
23
- ## Usage
24
-
25
- Transformers 5 removed the `question-answering` pipeline, so load the model directly:
26
-
27
- ```python
28
- import torch
29
- from transformers import AutoModelForQuestionAnswering, AutoTokenizer
30
-
31
- tok = AutoTokenizer.from_pretrained("harpertoken/clue")
32
- model = AutoModelForQuestionAnswering.from_pretrained("harpertoken/clue")
33
-
34
- question = "Who wrote Hamlet?"
35
- context = "Hamlet is a tragedy written by William Shakespeare around 1600."
36
- inputs = tok(question, context, return_tensors="pt", truncation=True, max_length=512)
37
-
38
- with torch.inference_mode():
39
- out = model(**inputs)
40
- start, end = int(out.start_logits.argmax()), int(out.end_logits.argmax())
41
- print(tok.decode(inputs.input_ids[0][start : end + 1]))
42
- ```
43
-
44
- On the three questions used to check `quiz` — the capital of France, the author of Hamlet, and the year the Eiffel Tower was completed — this checkpoint returns `paris`, `william shakespeare` and `1889`, the same answers. A thousand examples has not visibly degraded it, which is itself a reason to doubt that the fine-tuning taught much.
45
-
46
- ## Limitations
47
-
48
- A thousand examples is a demonstration of the fine-tuning mechanics rather than a trained model, and the documentation this replaced claimed SQuAD exact-match and F1 figures that were never produced by an evaluation. Treat this as a low-fidelity copy of `quiz`. It is English-only, inherits the same uncased tokenisation and SQuAD domain bias described in the `quiz` card, and shares its 512-token limit. Compare the two directly before assuming the fine-tuning helped.
49
-
50
- ## Attribution
51
-
52
- DistilBERT follows Sanh et al. (2019); SQuAD follows Rajpurkar et al. (2016).