NLPForUA/gpt2-large-uk-title-generation

📌 Educational release (Lab work)
This model was trained and published as part of an educational lab assignment on news title generation using GPT-2 + LoRA fine-tuning.
The full training notebook (data prep → fine-tuning → evaluation) is available here:

https://github.com/niksyromyatnikov/opnu-ml-assignments/blob/main/news-title-generation-gpt2-lora/news-title-generation-gpt2-lora.ipynb

NLPForUA/gpt2-large-uk-title-generation generates Ukrainian-style news headlines from article text.
It is designed for experimenting with prompting, decoding strategies, and parameter-efficient fine-tuning (LoRA).

---

What this model does

Given a Ukrainian news article (or its first paragraph), the model generates a short candidate title/headline.

Recommended prompt pattern:

• article text
• blank line
• Назва:
• generation

---

Model details

• Architecture: GPT-2 Large (decoder-only causal LM)
• Parameters: ~774M (GPT-2 Large class)
• Language: Ukrainian (uk)
• Task: title / headline generation
• Fine-tuning: PEFT LoRA (parameter-efficient adaptation)

---

Training data

The model was fine-tuned on Ukrainian news articles with associated headlines from:

• Dataset: FIdo-AI/ua-news

Typical categories include politics, economy, sports, tech, society, etc.

---

Intended use

✅ Good for

• Headline suggestions for Ukrainian text
• Generating multiple candidate titles for editorial review
• Educational demos of LoRA fine-tuning and decoding
• Dataset prototyping / augmentation (with human filtering)

❌ Not intended for

• Fully automated publishing without a human editor
• High-stakes applications requiring guaranteed factual accuracy
• Legal/medical/safety-critical content generation

---

Usage (Transformers)

from transformers import AutoTokenizer, AutoModelForCausalLM, pipeline

model_id = "NLPForUA/gpt2-large-uk-title-generation"

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id)

articles = [
    "Уряд України розглядає нові зміни до податкового законодавства. Експерти прогнозують вплив на малий бізнес та ІТ-сектор..."
]

def predict_title(model, inputs: list, postprocess=False, temperature=0.6, max_new_tokens=48) -> list:
    outputs = []
    with torch.no_grad():
        for idx, row in enumerate(inputs):
            if (idx+1) % 100 == 0:
                print(f"Generated {idx+1} titles\n")
            prompt = row + "\n Назва:"
            batch = tokenizer([prompt], return_tensors='pt').to(device)
            output_tokens = model.generate(
                **batch,
                max_new_tokens=max_new_tokens,
                do_sample=True,
                temperature=temperature,
                pad_token_id=tokenizer.eos_token_id,
                bos_token_id=tokenizer.bos_token_id,
                eos_token_id=tokenizer.eos_token_id,
            )
            output = tokenizer.decode(output_tokens[0], skip_special_tokens=True)
            if postprocess:
                output = output.split("\n Назва:")[1]
            outputs.append(output.strip())
    return outputs

predicted_titles = predict_title(model, articles, postprocess=True)

for row in predicted_titles:
    print(row, '\n\n')