Feature Extraction
MLX
English
hrr
vsa
holographic-reduced-representations
tokenizer
morphemes
compositional
Instructions to use thebasedcapital/morph-hrr with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- MLX
How to use thebasedcapital/morph-hrr with MLX:
# Download the model from the Hub pip install huggingface_hub[hf_xet] huggingface-cli download --local-dir morph-hrr thebasedcapital/morph-hrr
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- LM Studio
| import pytest | |
| from morph_hrr import segment | |
| cases = [ | |
| ("unhappy", ("un", "happy", "")), | |
| ("unkind", ("un", "kind", "")), | |
| ("running", ("", "runn", "ing")), # no undoubling (documented) | |
| ("walking", ("", "walk", "ing")), | |
| ("happiness", ("", "happ", "iness")), # longest suffix "iness" | |
| ("kingdom", ("", "king", "dom")), | |
| ("walks", ("", "walk", "s")), | |
| # Rule-based longest-match can't know "order" is a root: after "pre" it | |
| # strips "er" too. Documents the (deliberate) imperfectness. | |
| ("preorder", ("pre", "ord", "er")), | |
| ("disconnected", ("dis", "connect", "ed")), | |
| ("happy", ("", "happy", "")), # no bare-"y" suffix | |
| ("KING", ("", "king", "")), # case-insensitive | |
| ("a", ("", "a", "")), # too short -> root only | |
| ] | |
| def test_segment(word, expected): | |
| assert segment(word) == expected | |
| def test_non_alpha_passthrough(): | |
| p, r, s = segment("123") | |
| assert r == "123" and p == "" and s == "" | |
| def test_empty(): | |
| assert segment("") == ("", "", "") | |