README.md

---
license: apache-2.0
base_model: Qwen/Qwen2.5-1.5B-Instruct
tags:
  - text-generation
  - title-generation
  - headline-generation
  - lora
  - qwen2
  - fine-tuned
language:
  - en
pipeline_tag: text-generation
---

# HeadlineGPT

**HeadlineGPT** is a fine-tuned version of
[Qwen/Qwen2.5-1.5B-Instruct](https://huggingface.co/Qwen/Qwen2.5-1.5B-Instruct)
specialized for generating concise, engaging titles from source content.

It is designed for:

- News and article headlines
- Research and academic content
- Talks and presentations
- Social media posts
- Other short-form content that needs an attention-grabbing title

## Model Details

| Property | Value |
|---|---|
| Base model | Qwen2.5-1.5B-Instruct |
| Fine-tuning | LoRA / PEFT |
| LoRA rank | 16 |
| LoRA alpha | 32 |
| LoRA dropout | 0.05 |
| Language | English |
| Training objective | Reward-weighted supervised fine-tuning |

### Training

The model was trained on content–title pairs.

Instead of treating every training example equally, examples were weighted according to their associated engagement score. Higher-scoring examples therefore contribute more strongly to the training loss.

This approach is intended to encourage title characteristics associated with higher engagement while retaining the broader writing patterns present in the training data.

The model was trained for **2 epochs on a randomly sampled 25,000-example subset** of the larger dataset.

## Usage

```python
from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

model = AutoModelForCausalLM.from_pretrained(
    "csankalp21/headlinegpt",
    torch_dtype=torch.float16,
    device_map="auto"
)

tokenizer = AutoTokenizer.from_pretrained(
    "csankalp21/headlinegpt"
)

messages = [
    {
        "role": "system",
        "content": "You are an expert at writing highly engaging titles."
    },
    {
        "role": "user",
        "content": (
            "Generate a high-engagement title for the following content:\n\n"
            "<your content here>"
        )
    }
]

text = tokenizer.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True
)

inputs = tokenizer(
    text,
    return_tensors="pt"
).to(model.device)

with torch.no_grad():
    output = model.generate(
        **inputs,
        max_new_tokens=40,
        temperature=0.7,
        do_sample=True,
        top_p=0.9,
        repetition_penalty=1.1
    )

generated_tokens = output[0][inputs["input_ids"].shape[1]:]

print(
    tokenizer.decode(
        generated_tokens,
        skip_special_tokens=True
    )
)