Hugging Face's logo

anarmorarm
/
WaveletLM

Text Generation
English
wavelet
attention-free
no-attention
attentionfree
FWHT
walsh-hadamard
PKM
product-key-memory
causal-lm
sub-quadratic
Model card Files
xet
 Community
Ramon Gougis commited on
Commit
4ccd8d7
·
verified ·
1 Parent(s): 3c744ab

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +2 -2
README.md CHANGED
@@ -45,11 +45,11 @@ from huggingface_hub import hf_hub_download
45
  ckpt_path = hf_hub_download(repo_id="ragou19/WaveletLM", filename="best_model.pt")
46
 
47
  # Then follow the instructions in the GitHub repo to load and run:
48
- # https://github.com/ramongougis/WaveletLM
49
  ```
 
50
 
51
  ## Architecture
52
- See the full <a href="https://github.com/ramongougis/WaveletLM#architecture">architecture documentation</a> on GitHub.
53
 
54
  ## Training
55
  Trained on a single RTX 5090 for 5 epochs on WikiText-103 (best of 3 seeds: 1337, 42, 7). Best validation loss: 3.16. PG-19 weights also included (1-epoch run; longer training planned post-release).
 
45
  ckpt_path = hf_hub_download(repo_id="ragou19/WaveletLM", filename="best_model.pt")
46
 
47
  # Then follow the instructions in the GitHub repo to load and run:
 
48
  ```
49
+ https://github.com/ramongougis/WaveletLM
50
 
51
  ## Architecture
52
+ ![WaveletLM architecture](https://raw.githubusercontent.com/ramongougis/WaveletLM/main/assets/waveletlm-architecture.svg)
53
 
54
  ## Training
55
  Trained on a single RTX 5090 for 5 epochs on WikiText-103 (best of 3 seeds: 1337, 42, 7). Best validation loss: 3.16. PG-19 weights also included (1-epoch run; longer training planned post-release).