| --- |
| library_name: tokenizers |
| tags: [Danish, Morphological Tokenization, LLaMA] |
| --- |
| ``` |
| _______ ___ .___ ___. ______ .______ .______ __ __ |
| | \ / \ | \/ | / __ \ | _ \ | _ \ | | | | |
| | .--. | / ^ \ | \ / | | | | | | |_) | | |_) | | |__| | |
| | | | | / /_\ \ | |\/| | | | | | | / | ___/ | __ | |
| | '--' | / _____ \ | | | | | `--' | | |\ \----.| | | | | | |
| |_______/ /__/ \__\ |__| |__| \______/ | _| `._____|| _| |__| |__| |
| |
| ``` |
| ### DA-MORPH-LLAMA3.2-TOKEN |
|
|
| A morphological tokenizer tailored for the LLaMA architecture, designed to enhance segmentation of Danish text by leveraging morphology-aware strategies. |
