alexantonov/ru-chv-marian

model

This model is a fine-tuned version of on an unknown dataset. It achieves the following results on the evaluation set:

• eval_loss: 3.3948
• eval_bleu: 20.1644
• eval_gen_len: 54.7876
• eval_runtime: 36.1129
• eval_samples_per_second: 55.271
• eval_steps_per_second: 0.886
• epoch: 7.9305
• step: 45148

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0005
• train_batch_size: 64
• eval_batch_size: 64
• seed: 42
• gradient_accumulation_steps: 4
• total_train_batch_size: 256
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: inverse_sqrt
• lr_scheduler_warmup_steps: 8000
• num_epochs: 10
• mixed_precision_training: Native AMP
• label_smoothing_factor: 0.1

Framework versions

• Transformers 5.6.2
• Pytorch 2.5.1+cu121
• Datasets 4.8.5
• Tokenizers 0.22.2

Translation procedure

import torch
from transformers import AutoTokenizer, AutoModelForSeq2SeqLM

# Путь к локальной папке или Hugging Face checkpoint
# checkpoint = r"path\to\your\chv_ru_marian\model"
# Например:
checkpoint = "alexantonov/ru-chv-marian"

tokenizer = AutoTokenizer.from_pretrained(checkpoint)
model = AutoModelForSeq2SeqLM.from_pretrained(checkpoint)

device = "cuda" if torch.cuda.is_available() else "cpu"
model.to(device)
model.eval()

input_file =  r"\eval\til\til.cv-ru.ru"
output_file = r"\eval\til.cv-ru.marian.cv"

MAX_SOURCE_LENGTH = 128
MAX_TARGET_LENGTH = 128


def normalize_text(s: str) -> str:
    return " ".join(str(s).replace("\u00a0", " ").split())


def add_eos_and_truncate(ids, max_length):
    eos = tokenizer.eos_token_id

    if len(ids) > 0 and ids[-1] == eos:
        ids = ids[:-1]

    ids = ids[: max_length - 1]
    ids = ids + [eos]
    return ids


def translate_one(text: str) -> str:
    text = normalize_text(text)

    enc = tokenizer(
        text,
        add_special_tokens=False,
        padding=False,
        truncation=False,
        return_tensors=None,
    )

    input_ids = add_eos_and_truncate(enc["input_ids"], MAX_SOURCE_LENGTH)

    input_ids = torch.tensor([input_ids], dtype=torch.long, device=device)
    attention_mask = torch.ones_like(input_ids, device=device)

    with torch.no_grad():
        outputs = model.generate(
            input_ids=input_ids,
            attention_mask=attention_mask,

            max_length=MAX_TARGET_LENGTH,
            num_beams=5,
            early_stopping=True,

            # Полезно против повторов у недоученной модели
            no_repeat_ngram_size=3,
            repetition_penalty=1.15,

            pad_token_id=tokenizer.pad_token_id,
            eos_token_id=tokenizer.eos_token_id,
            decoder_start_token_id=tokenizer.pad_token_id,
        )

    decoded = tokenizer.decode(
        outputs[0],
        skip_special_tokens=True,
        clean_up_tokenization_spaces=True,
    )

    return decoded.strip()


with open(input_file, "r", encoding="utf-8") as f:
    texts = [line.rstrip("\n") for line in f]

counter = 0

with open(output_file, "w", encoding="utf-8") as out_f:
    for text in texts:
        counter += 1

        # Если пустая строка — сохраняем пустую строку
        if not text.strip():
            out_f.write("\n")
            continue

        decoded = translate_one(text)

        print(f"Num: {counter}")
        print(f"CHV: {text}")
        print(f"RU : {decoded}")
        print("-" * 80)

        out_f.write(decoded + "\n")

print(f"Результаты сохранены в файл: {output_file}")