|
|
--- |
|
|
license: apache-2.0 |
|
|
base_model: Helsinki-NLP/opus-mt-en-zh |
|
|
tags: |
|
|
- generated_from_trainer |
|
|
- translation |
|
|
- machine-translation |
|
|
- english |
|
|
- traditional-chinese |
|
|
- transformer |
|
|
- fine-tuned |
|
|
datasets: |
|
|
- agentlans/en-zhtw-google-translate |
|
|
language: |
|
|
- en |
|
|
- zh |
|
|
pipeline_tag: translation |
|
|
--- |
|
|
<details> |
|
|
<summary>English-to-Traditional Chinese Translator</summary> |
|
|
|
|
|
This model is a fine-tuned version of [Helsinki-NLP/opus-mt-en-zh](https://huggingface.co/Helsinki-NLP/opus-mt-en-zh), trained on the [agentlans/en-zhtw-google-translate](https://huggingface.co/datasets/agentlans/en-zhtw-google-translate) dataset. |
|
|
|
|
|
It is optimized to produce **Traditional Chinese translations by default**, enhancing the naturalness and fluency of the output. |
|
|
|
|
|
## Model Description |
|
|
|
|
|
- **Input:** English text only |
|
|
- **Output:** Traditional Chinese translation |
|
|
|
|
|
</details> |
|
|
|
|
|
<details> |
|
|
<summary>英文至繁體中文翻譯模型</summary> |
|
|
|
|
|
本模型為 [Helsinki-NLP/opus-mt-en-zh](https://huggingface.co/Helsinki-NLP/opus-mt-en-zh) 的微調版本，使用 [agentlans/en-zhtw-google-translate](https://huggingface.co/datasets/agentlans/en-zhtw-google-translate) 資料集進行訓練。 |
|
|
|
|
|
模型已針對輸出繁體中文進行最佳化，提升了翻譯結果的自然度與流暢性。 |
|
|
|
|
|
## 模型說明 |
|
|
|
|
|
- **輸入：** 僅支援英文文本 |
|
|
- **輸出：** 繁體中文翻譯 |
|
|
</details> |
|
|
|
|
|
## How to use / 如何使用 |
|
|
|
|
|
```python |
|
|
from transformers import pipeline |
|
|
|
|
|
# Load the translation model |
|
|
# 載入翻譯模型 |
|
|
model_checkpoint = "agentlans/en-zhtw" |
|
|
translator = pipeline("translation", model=model_checkpoint) |
|
|
|
|
|
# This is for correcting English punctuation marks to Traditional Chinese. |
|
|
# 這是為了將英語標點符號校正為繁體中文。 |
|
|
def en_to_zh_punct(text): |
|
|
punct = { |
|
|
'!': '！', '?': '？', ',': '，', '.': '。', |
|
|
':': '：', ';': '；', '(': '（', ')': '）', |
|
|
'[': '【', ']': '】', '{': '｛', '}': '｝' |
|
|
} |
|
|
result, in_dq, in_sq = [], False, False |
|
|
for ch in text: |
|
|
if ch == '"': |
|
|
result.append("」" if in_dq else "「") |
|
|
in_dq = not in_dq |
|
|
elif ch == "'": |
|
|
result.append("』" if in_sq else "『") |
|
|
in_sq = not in_sq |
|
|
else: |
|
|
result.append(punct.get(ch, ch)) |
|
|
return "".join(result) |
|
|
|
|
|
# The main function for translating English to Traditional Chinese |
|
|
# 將英語翻譯成繁體中文的主要功能 |
|
|
def translate(en_text): |
|
|
return [en_to_zh_punct(x["translation_text"]) for x in translator(en_text)] |
|
|
|
|
|
# Example |
|
|
# 範例 |
|
|
translate( |
|
|
[ |
|
|
"Trump announces new tariffs on penguin islands. The penguins plan to tax U.S. imports in retaliation.", |
|
|
"We now return to the White House for the latest developments on the trade war.", |
|
|
] |
|
|
) |
|
|
# ['川普宣佈對企鵝島徵收新關稅，企鵝打算對美國進口產品徵稅報復。', '我們現在回到白宮尋找貿易戰的最新發展。'] |
|
|
``` |
|
|
|
|
|
## Limitations / 限制 |
|
|
|
|
|
<details> |
|
|
<summary>Limitations</summary> |
|
|
|
|
|
- Handles only one- or two-sentence inputs in English effectively. |
|
|
- Struggles with English spelling, names, abbreviations, and especially technical terminology. |
|
|
- Uses unusual punctuation like the English comma instead of the Chinese comma. |
|
|
- Has difficulty understanding context. |
|
|
- As a result, may generate inaccurate information or omit important details. |
|
|
- Sometimes uses incorrect words due to the base model being primarily trained on Simplified Chinese, which does not always correspond directly to Traditional Chinese. |
|
|
</details> |
|
|
|
|
|
<details> |
|
|
<summary>限制</summary> |
|
|
|
|
|
- 僅適用於處理一至兩句英文句子的輸入，處理較長段落時效果有限。 |
|
|
- 難以準確掌握英語拼字、專有名詞及縮寫，尤其在處理技術術語時表現不佳。 |
|
|
- 常出現標點符號使用不當的情況，例如以英文逗號取代中文逗號。 |
|
|
- 對語境的理解能力有限。 |
|
|
- 可能導致資訊不準確或遺漏重要細節。 |
|
|
- 由於基礎模型主要以簡體中文語料訓練，有時會使用不自然或錯誤的詞語，簡體與繁體用語之間也未必能精確對應。 |
|
|
</details> |
|
|
|
|
|
## Training procedure / 訓練過程 |
|
|
|
|
|
<details> |
|
|
<summary>Click here / 點這裡</summary> |
|
|
|
|
|
### Training hyperparameters |
|
|
|
|
|
The following hyperparameters were used during training: |
|
|
- learning_rate: 5e-05 |
|
|
- train_batch_size: 8 |
|
|
- eval_batch_size: 8 |
|
|
- seed: 42 |
|
|
- optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments |
|
|
- lr_scheduler_type: linear |
|
|
- num_epochs: 5.0 |
|
|
|
|
|
### Training results |
|
|
|
|
|
| Training Loss | Epoch | Step | Validation Loss | Input Tokens Seen | |
|
|
|:-------------:|:-----:|:------:|:---------------:|:-----------------:| |
|
|
| 1.3993 | 1.0 | 99952 | 1.2487 | 54454616 | |
|
|
| 1.2801 | 2.0 | 199904 | 1.1701 | 108935048 | |
|
|
| 1.1728 | 3.0 | 299856 | 1.1232 | 163424808 | |
|
|
| 1.1001 | 4.0 | 399808 | 1.0871 | 217911400 | |
|
|
| 1.0243 | 5.0 | 499760 | 1.0584 | 272407288 | |
|
|
|
|
|
|
|
|
### Framework versions |
|
|
|
|
|
- Transformers 4.51.3 |
|
|
- Pytorch 2.6.0+cu124 |
|
|
- Datasets 3.2.0 |
|
|
- Tokenizers 0.21.0 |
|
|
|
|
|
</details> |
