YAML Metadata Warning:The pipeline tag "text2text-generation" is not in the official list: text-classification, token-classification, table-question-answering, question-answering, zero-shot-classification, translation, summarization, feature-extraction, text-generation, fill-mask, sentence-similarity, text-to-speech, text-to-audio, automatic-speech-recognition, audio-to-audio, audio-classification, audio-text-to-text, voice-activity-detection, depth-estimation, image-classification, object-detection, image-segmentation, text-to-image, image-to-text, image-to-image, image-to-video, unconditional-image-generation, video-classification, reinforcement-learning, robotics, tabular-classification, tabular-regression, tabular-to-text, table-to-text, multiple-choice, text-ranking, text-retrieval, time-series-forecasting, text-to-video, image-text-to-text, image-text-to-image, image-text-to-video, visual-question-answering, document-question-answering, zero-shot-image-classification, graph-ml, mask-generation, zero-shot-object-detection, text-to-3d, image-to-3d, image-feature-extraction, video-text-to-text, keypoint-detection, visual-document-retrieval, any-to-any, video-to-video, other
Okinawa Dialect Assistant v2
An experimental Japanese-to-Shuri dictionary lookup model. Given a Japanese word or short prompt, it proposes the corresponding Shuri form found in the training dictionary. This is a larger, prompt-augmented continuation of LoNebula/okinawa-dialect-assistant.
Training data
The v2 dataset contains 18,062 prompted rows derived from 9,031 source dictionary entries. Each source entry is rendered with two fixed Japanese prompt forms. The additional rows increase prompt coverage; they do not add vocabulary, independent attestations, or new linguistic claims. Both prompt variants for a headword remain in the same train or test split. The source is attributed and distributed under CC BY 4.0; see the dataset card for source and attribution details.
Use
from transformers import AutoModelForCausalLM, AutoTokenizer
repo = "LoNebula/okinawa-dialect-assistant-v2"
tokenizer = AutoTokenizer.from_pretrained(repo)
model = AutoModelForCausalLM.from_pretrained(repo)
prompt = "日本語の「ありがとう」に対応する首里方言を教えてください。"
messages = [
{"role": "system", "content": "辞書に基づき、根拠のない語を作らずに回答してください。"},
{"role": "user", "content": prompt},
]
inputs = tokenizer.apply_chat_template(messages, add_generation_prompt=True,
return_tensors="pt")
generated = model.generate(inputs, max_new_tokens=64)
print(tokenizer.decode(generated[0][inputs.shape[1]:], skip_special_tokens=True))
Limitations
This is a small experimental model trained on dictionary-form examples, not a fluent conversation model, comprehensive Okinawan language model, speech model, or validated translator. The source focuses on Shuri and does not represent all Okinawan/Ryukyuan varieties. Verify outputs with fluent speakers and the cited dictionary. Generated responses may be wrong or incomplete.
License
The fine-tuned weights are published under Apache-2.0, subject to the base model's terms. The source-derived dataset is separately licensed CC BY 4.0 with attribution; consult its dataset card.
- Downloads last month
- 18