| --- |
| language: |
| - en |
| metrics: |
| - f1 |
| base_model: |
| - FacebookAI/roberta-base |
| tags: |
| - parsing |
| - hashing |
| - unsupervised |
| --- |
| |
| ## On Eliciting Syntax from Language Models via Hashing |
|
|
| ## Model Details |
|
|
| This repository contains the implementation of [**Parserker v2**](https://aclanthology.org/2024.emnlp-main.479/), a |
| hashing-based unsupervised parser trained on the Penn Treebank dataset using only the raw text (no syntactic annotations |
| or tree labels). |
|
|
| ## Usage |
|
|
| ### Requirements |
|
|
| `pip install transformers torch nltk torchrua` |
|
|
| ### Demo |
|
|
| ```python |
| from nltk import TreePrettyPrinter |
| from transformers import AutoModel, AutoTokenizer |
| |
| model = AutoModel.from_pretrained("yehzw/parserker", trust_remote_code=True) |
| tokenizer = AutoTokenizer.from_pretrained("yehzw/parserker", trust_remote_code=True) |
| |
| model.eval() |
| |
| words, input_ids, duration = tokenizer([ |
| "The quick brown fox jumps over the lazy dog", |
| "The man who you met yesterday is my teacher", |
| "The boy saw the girl with a telescope", |
| "The dog on the hill barked at the man who laughed", |
| ]) |
| |
| for w, s in zip(words, model.parse(input_ids, duration).tolist()): |
| t = model.to_tree(w, s) |
| t = TreePrettyPrinter(t).text() |
| print(t) |
| ``` |
|
|
| ### Ouput Examples |
|
|
| The quick brown fox jumps over the lazy dog |
|
|
| ``` |
| 854D |
| _____________________|_________ |
| | DF41 |
| | _________|____ |
| 955F | DD59 |
| __________|_____ | _________|____ |
| | DC45 | | D457 |
| | __________|____ | | _________|____ |
| | | DE45 | | | DECD |
| | | ____|____ | | | ____|____ |
| 103B C404 DC05 D60D 9300 C995 D0B7 DC8D DE8D |
| | | | | | | | | | |
| The quick brown fox jumps over the lazy dog |
| ``` |
|
|
| The man who you met yesterday is my teacher |
|
|
| ``` |
| C50D |
| ________________________|____________ |
| | D558 |
| | _________________|___________ |
| | 9718 | |
| | _________|____ | |
| | | C718 5D52 |
| | | _________|____ ____|____ |
| 965F | | DF00 | DF5D |
| ____|____ | | ____|_______ | ____|______ |
| 103B C60D 1799 4719 D300 47BC 7192 5895 CE0D |
| | | | | | | | | | |
| The man who you met yesterday is my teacher |
| ``` |
|
|
| The boy saw the girl with a telescope |
|
|
| ``` |
| C14C |
| ______________|_________ |
| | DF41 |
| | ______________|____ |
| | | C54D |
| | | _________|____ |
| | | | 9D59 |
| | | | ____|____ |
| 165F | D657 | 9E55 |
| ____|____ | ____|____ | ____|_______ |
| 103B C20D D100 D0B7 C60D 3991 9817 CE8D |
| | | | | | | | | |
| The boy saw the girl with a telescope |
| ``` |
|
|
| The dog on the hill barked at the man who laughed |
|
|
| ``` |
| C50D |
| ____________________|__________ |
| | DF40 |
| | __________|_________ |
| C54D | DD19 |
| _________|____ | ______________|____ |
| | 9D09 | | C5CD |
| | ____|____ | | _________|_________ |
| 965F | D65F | | D657 97D8 |
| ____|____ | ____|____ | | ____|____ ____|______ |
| 103B C20D 2F99 D0B3 C60D D300 6395 D0B7 C60D 1799 CF89 |
| | | | | | | | | | | | |
| The dog on the hill barked at the man who laughed |
| ``` |
|
|
| ## Citation |
|
|
| ```bib |
| @inproceedings{wang-utiyama-2024-eliciting, |
| title = "On Eliciting Syntax from Language Models via Hashing", |
| author = "Wang, Yiran and |
| Utiyama, Masao", |
| editor = "Al-Onaizan, Yaser and |
| Bansal, Mohit and |
| Chen, Yun-Nung", |
| booktitle = "Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing", |
| month = nov, |
| year = "2024", |
| address = "Miami, Florida, USA", |
| publisher = "Association for Computational Linguistics", |
| url = "https://aclanthology.org/2024.emnlp-main.479/", |
| doi = "10.18653/v1/2024.emnlp-main.479", |
| pages = "8412--8427" |
| } |
| ``` |
