| --- |
| license: apache-2.0 |
| base_model_relation: finetune |
| base_model: |
| - sbintuitions/modernbert-ja-310m |
| tags: |
| - modernbert |
| --- |
| |
| BERT |
|
|
|  |
|
|
| *2girls, two girls are sitting at the table, the left girl has pink hair, the right girl has black hair. The left girl is wearing a yellow hair ribbon* |
|
|
| This modern architecture combines the LLM adapter with existing architectures. |
|
|
| Although the tokenization process is still done at the BERT level, the T5 ids are not entirely discarded. It can be called a text encoder with a dual tokenizer. |
|
|
| This implementation is also leaner than the fat, transformer-based adapters on the top of LLMs. |
|
|
| While ModernBERT's entire multilingual vocabulary can still be used, this is backward compatible with Anima's T5. |
|
|
| This repo also converges faster than [her lil sister](https://huggingface.co/nebulette/berry). |
|
|
| Source data |
|
|
| - anime themed, long natural sentences |
| - general tags |
| - character count tags |
| - colors |
| - negatives |
| - safety levels |
| - spatial relationships |
|
|
| No credits are provided; this is original work. |
