# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM
tokenizer = AutoTokenizer.from_pretrained("bluolightning/bert-tiny-japanese-char")
model = AutoModelForCausalLM.from_pretrained("bluolightning/bert-tiny-japanese-char")Quick Links
Custom Japanese BERT (4-layer)
This model is a tiny Japanese BERT model with 4 layers, optimized for speed.
Model Background
- Architecture: BERT (4 layers, 256 hidden size, 4 heads, 1024 FFN)
- Distillation: Distilled from a fine-tuned version of
tohoku-nlp/bert-base-japanese-char-v2. - Initialization: The student model was randomly initialized.
- Tokenizer: Japanese Character-level tokenizer, shared with the teacher.
- Downloads last month
- 7
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("fill-mask", model="bluolightning/bert-tiny-japanese-char")