Update README.md

README.md

@@ -1,199 +1,202 @@
 ---
-library_name: transformers
-tags: []
+language:
+- ja
+license: mit
+base_model: tohoku-nlp/bert-base-japanese-v3
+tags:
+- japanese
+- keigo
+- text-classification
+- omotenashi
+- hospitality
+- bert
+pipeline_tag: text-classification
 ---
 
-# Model Card for Model ID
+# Keigo Evaluator — 敬語レベル分類モデル
 
-<!-- Provide a quick summary of what the model is/does. -->
+A fine-tuned Japanese BERT model that classifies the politeness level (敬語レベル) of Japanese speech into four levels. Designed to evaluate whether an employee is speaking with appropriate **Keigo (敬語)** and **Omotenashi (おもてなし)** standards in a hospitality or service context.
 
+---
 
+## Intended Use
 
-## Model Details
-
-### Model Description
-
-<!-- Provide a longer summary of what this model is. -->
-
-This is the model card of a 🤗 transformers model that has been pushed on the Hub. This model card has been automatically generated.
-
-- **Developed by:** [More Information Needed]
-- **Funded by [optional]:** [More Information Needed]
-- **Shared by [optional]:** [More Information Needed]
-- **Model type:** [More Information Needed]
-- **Language(s) (NLP):** [More Information Needed]
-- **License:** [More Information Needed]
-- **Finetuned from model [optional]:** [More Information Needed]
-
-### Model Sources [optional]
-
-<!-- Provide the basic links for the model. -->
-
-- **Repository:** [More Information Needed]
-- **Paper [optional]:** [More Information Needed]
-- **Demo [optional]:** [More Information Needed]
-
-## Uses
-
-<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
-
-### Direct Use
-
-<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
-
-[More Information Needed]
-
-### Downstream Use [optional]
-
-<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
-
-[More Information Needed]
-
-### Out-of-Scope Use
-
-<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
-
-[More Information Needed]
+This model is the NLP component of an AI-powered service quality evaluation pipeline:
 
-## Bias, Risks, and Limitations
+```
+Voice Recording → Whisper ASR → Transcribed Text → This Model → Keigo Verdict
+```
 
-<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+It is intended for:
+- Evaluating employee speech quality in hospitality and customer service settings
+- Automated Keigo compliance checking in call centres or hotel/restaurant environments
+- Quality assurance systems for Japanese service staff training
 
-[More Information Needed]
+---
 
-### Recommendations
+## Labels
 
-<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+The model predicts one of four classes:
 
-Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+| Label | Level | Name | Description | Service Verdict |
+|-------|-------|------|-------------|-----------------|
+| LABEL_0 | 1 | 最高敬語 | Highest honorific — sonkeigo dominant | ✅ Pass |
+| LABEL_1 | 2 | 敬語 | Standard honorific — appropriate for most service contexts | ✅ Pass |
+| LABEL_2 | 3 | 丁寧語 | Polite but not honorific — insufficient for hospitality | ❌ Fail |
+| LABEL_3 | 4 | 普通語 | Casual / plain speech — inappropriate in service contexts | ❌ Fail |
 
-## How to Get Started with the Model
+---
 
-Use the code below to get started with the model.
+## How to Use
+
+### Installation
+
+```bash
+pip install transformers torch fugashi unidic-lite
+```
+
+> **Note**: `unidic-lite` is required (not `ipadic`) — this model uses the UniDic dictionary for MeCab tokenization.
+
+### Basic Usage
+
+```python
+from transformers import pipeline
+import torch
+
+classifier = pipeline(
+    'text-classification',
+    model='ishraq/keigo-evaluator',
+    device=0 if torch.cuda.is_available() else -1
+)
+
+LEVEL_MAP = {
+    'LABEL_0': {'level': 1, 'name': '最高敬語', 'passed': True},
+    'LABEL_1': {'level': 2, 'name': '敬語',     'passed': True},
+    'LABEL_2': {'level': 3, 'name': '丁寧語',   'passed': False},
+    'LABEL_3': {'level': 4, 'name': '普通語',   'passed': False},
+}
+
+def evaluate_keigo(text: str) -> dict:
+    result = classifier(text)[0]
+    info   = LEVEL_MAP[result['label']]
+    return {
+        'text':       text,
+        'level':      info['level'],
+        'level_name': info['name'],
+        'confidence': round(result['score'], 3),
+        'passed':     info['passed'],
+        'verdict':    '✅ 適切な敬語です' if info['passed'] else '❌ 敬語レベルが不足しています'
+    }
+
+print(evaluate_keigo('いらっしゃいませ。本日はどのようなご用件でございましょうか？'))
+# {'level': 1, 'level_name': '最高敬語', 'confidence': 0.91, 'passed': True, 'verdict': '✅ 適切な敬語です'}
+
+print(evaluate_keigo('ちょっと待って。'))
+# {'level': 4, 'level_name': '普通語', 'confidence': 0.99, 'passed': False, 'verdict': '❌ 敬語レベルが不足しています'}
+```
+
+### Full Voice Pipeline (Whisper + Keigo Evaluator)
+
+```python
+import whisper
+from transformers import pipeline
+import torch
+
+asr        = whisper.load_model('medium')
+classifier = pipeline(
+    'text-classification',
+    model='ishraq/keigo-evaluator',
+    device=0 if torch.cuda.is_available() else -1
+)
+
+def evaluate_recording(audio_path: str) -> dict:
+    transcript = asr.transcribe(audio_path, language='ja')['text']
+    result     = classifier(transcript)[0]
+    info       = LEVEL_MAP[result['label']]
+    return {
+        'transcript': transcript,
+        'level':      info['level'],
+        'level_name': info['name'],
+        'confidence': round(result['score'], 3),
+        'passed':     info['passed'],
+        'verdict':    '✅ 適切な敬語です' if info['passed'] else '❌ 敬語レベルが不足しています'
+    }
+
+result = evaluate_recording('employee_call.mp3')
+print(result)
+```
 
-[More Information Needed]
+---
 
 ## Training Details
 
-### Training Data
-
-<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
-
-[More Information Needed]
-
-### Training Procedure
-
-<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
-
-#### Preprocessing [optional]
-
-[More Information Needed]
-
-
-#### Training Hyperparameters
-
-- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
-
-#### Speeds, Sizes, Times [optional]
-
-<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+### Dataset
 
-[More Information Needed]
+**KeiCO Corpus** — a Japanese keigo classification corpus of 10,002 sentences labelled by politeness level and keigo type (sonkeigo / kenjōgo / teineigo) across a wide range of service situations including greetings (挨拶), apologies (謝る), meetings (会う), and seasonal expressions (季節).
 
-## Evaluation
+| Level | Count | % |
+|-------|-------|---|
+| 1 — 最高敬語 | 2,584 | 25.8% |
+| 2 — 敬語     | 2,044 | 20.4% |
+| 3 — 丁寧語   | 2,692 | 26.9% |
+| 4 — 普通語   | 2,682 | 26.8% |
 
-<!-- This section describes the evaluation protocols and provides the results. -->
+The dataset is well-balanced. No oversampling or class weighting was applied.
 
-### Testing Data, Factors & Metrics
+### Training Hyperparameters
 
-#### Testing Data
+| Parameter | Value |
+|-----------|-------|
+| Epochs | 5 |
+| Batch size | 32 |
+| Learning rate | 2e-5 |
+| Weight decay | 0.01 |
+| Warmup ratio | 10% |
+| Max sequence length | 128 |
+| Optimizer | AdamW |
+| Scheduler | Linear warmup + decay |
+| Gradient clipping | 1.0 |
+| Loss | Cross-entropy |
 
-<!-- This should link to a Dataset Card if possible. -->
+### Training Infrastructure
 
-[More Information Needed]
+- **Hardware**: NVIDIA T4 GPU (Google Colab)
+- **Framework**: PyTorch + Hugging Face Transformers
+- **Train / Val split**: 85% / 15% stratified by label
 
-#### Factors
-
-<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
-
-[More Information Needed]
-
-#### Metrics
-
-<!-- These are the evaluation metrics being used, ideally with a description of why. -->
-
-[More Information Needed]
-
-### Results
-
-[More Information Needed]
-
-#### Summary
-
-
-
-## Model Examination [optional]
-
-<!-- Relevant interpretability work for the model goes here -->
-
-[More Information Needed]
-
-## Environmental Impact
-
-<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
-
-Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
-
-- **Hardware Type:** [More Information Needed]
-- **Hours used:** [More Information Needed]
-- **Cloud Provider:** [More Information Needed]
-- **Compute Region:** [More Information Needed]
-- **Carbon Emitted:** [More Information Needed]
-
-## Technical Specifications [optional]
-
-### Model Architecture and Objective
-
-[More Information Needed]
-
-### Compute Infrastructure
-
-[More Information Needed]
-
-#### Hardware
-
-[More Information Needed]
-
-#### Software
-
-[More Information Needed]
-
-## Citation [optional]
-
-<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
-
-**BibTeX:**
-
-[More Information Needed]
+---
 
-**APA:**
+## Evaluation Results
 
-[More Information Needed]
+Sample inference results on held-out test sentences:
 
-## Glossary [optional]
+| Input | Predicted Level | Confidence | Verdict |
+|-------|----------------|------------|---------|
+| 本日はお早いのですね、お散歩ですか？ | 2 — 敬語 | 0.598 | ✅ Pass |
+| ご多用中にもかかわらず、よくお出くださいました。 | 2 — 敬語 | 0.557 | ✅ Pass |
+| お問い合わせをいただいた商品が、本日入荷しました。 | 3 — 丁寧語 | 0.740 | ❌ Fail |
+| 今日はうどんにする。 | 4 — 普通語 | 0.993 | ❌ Fail |
+| 忙しいのに、よく来たね。 | 4 — 普通語 | 0.996 | ❌ Fail |
 
-<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+Casual speech (Level 4) is detected with near-perfect confidence. Borderline honorific sentences show appropriately lower confidence scores.
 
-[More Information Needed]
+---
 
-## More Information [optional]
+## Limitations
 
-[More Information Needed]
+- The model evaluates **transcribed text**, not raw audio. Whisper transcription quality directly affects evaluation accuracy — `whisper medium` or `whisper large` is recommended for Japanese.
+- Confidence scores below **0.60** on a passing result indicate borderline speech — consider flagging for human review.
+- The model classifies overall politeness level and does **not** identify specific keigo errors (e.g. incorrect verb conjugation).
+- Accuracy may be lower for highly domain-specific speech such as medical or legal Japanese.
 
-## Model Card Authors [optional]
+---
 
-[More Information Needed]
+## Citation
 
-## Model Card Contact
+If you use this model, please cite the KeiCO corpus and the base model:
 
-[More Information Needed]
+```
+Base model: Tohoku NLP Lab, BERT-base Japanese v3
+Dataset: KeiCO Corpus — Japanese Keigo Classification Corpus
+Fine-tuned by: Ishraq (B-JET Ideathon 2026 — Smart Service Evaluator)
+```