Bimaldo commited on
Commit
099b7ee
·
verified ·
1 Parent(s): ed2140a

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +67 -0
README.md ADDED
@@ -0,0 +1,67 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ language: en
3
+ license: mit
4
+ tags:
5
+ - diffusion
6
+ - text-generation
7
+ - riddles
8
+ - diffusion-lm
9
+ datasets:
10
+ - prajwaldongre/riddles-a-synthetic-riddle-dataset-for-nlp
11
+ metrics:
12
+ - exact-match
13
+ - token-f1
14
+ ---
15
+
16
+ # Diffusion-LM Riddle Solver — Phase 3 Reconstructed
17
+
18
+ A Diffusion-LM-style (Li et al., 2022) text generation model trained on 232 synthetic riddles. Held-out exact match: **47.0%** (K=1 and K=10).
19
+
20
+ ## Model description
21
+
22
+ Continuous embedding diffusion with a Transformer encoder/decoder. The model takes a riddle as context and iteratively denoises Gaussian noise into answer word embeddings via a learned reverse process.
23
+
24
+ ## Intended use
25
+
26
+ Research and diagnostics. Not a production-ready riddle solver.
27
+
28
+ ## Training data
29
+
30
+ [Riddles — A Synthetic Riddle Dataset for NLP](https://www.kaggle.com/datasets/prajwaldongre/riddles-a-synthetic-riddle-dataset-for-nlp) (CC0). 232 training examples after deduplication.
31
+
32
+ ## Architecture
33
+
34
+ | Parameter | Value |
35
+ |-----------|-------|
36
+ | Parameters | 8,024,576 |
37
+ | Timesteps (T) | 200 |
38
+ | d_model | 256 |
39
+ | Layers | 4 |
40
+ | d_ff | 1024 |
41
+ | Heads | 4 |
42
+ | Answer length | 4 |
43
+ | Noise schedule | sqrt power-law |
44
+
45
+ ## Performance
46
+
47
+ | Split | Exact match (K=1) | Token F1 |
48
+ |-------|-------------------|----------|
49
+ | Train (n=192) | 87.5% | 0.960 |
50
+ | Held-out (n=66) | 47.0% | 0.523 |
51
+
52
+ ## Limitations
53
+
54
+ - Trained on 232 examples only. Does not generalize broadly.
55
+ - Uses continuous embedding diffusion with Euclidean clamping. Discrete formulations may differ.
56
+ - Exact-match metric penalizes semantically equivalent answers.
57
+
58
+ ## Files
59
+
60
+ - `model.safetensors`: Model weights
61
+ - `config.json`: Architecture hyperparameters
62
+ - `vocab.json`: Vocabulary mapping
63
+ - `inference.py`: Standalone prediction script
64
+
65
+ ## Source
66
+
67
+ Full source code, diagnostics, and reproduction configs: [github.com/beme08/riddle-diffusion-lm](https://github.com/beme08/riddle-diffusion-lm)