README.md

---
license: apache-2.0
base_model: Qwen/Qwen2.5-1.5B-Instruct
tags:
  - text-generation
  - title-generation
  - headline-generation
  - onnx
  - transformers.js
  - webgpu
language:
  - en
pipeline_tag: text-generation
---

# HeadlineGPT — ONNX / WebGPU

Browser-ready ONNX export of [csankalp21/headlinegpt](https://huggingface.co/csankalp21/headlinegpt), a Qwen2.5-1.5B-Instruct model fine-tuned for generating concise, engaging titles.

This version is designed for **local, in-browser inference using WebGPU** with [Transformers.js](https://huggingface.co/docs/transformers.js).

Your content can be processed entirely on the user's device without sending it to a remote inference server.

See the [main HeadlineGPT model card](https://huggingface.co/csankalp21/headlinegpt) for training details and model limitations.

## Model Format

| Property | Value |
|---|---|
| Base model | Qwen2.5-1.5B-Instruct |
| Format | ONNX |
| Precision | FP16 |
| Export task | `text-generation-with-past` |
| Approximate size | 3.7 GB |
| Intended runtime | Transformers.js / ONNX Runtime Web |
| Hardware acceleration | WebGPU |

The current release uses FP16 weights. A smaller quantized version may be released separately.

## Browser Usage

Install Transformers.js:

```bash
npm install @huggingface/transformers


```javascript
import { pipeline } from "@huggingface/transformers";

const generator = await pipeline(
  "text-generation",
  "csankalp21/headlinegpt-onnx",
  {
    device: "webgpu",
    dtype: "fp16",
  }
);

const messages = [
  {
    role: "system",
    content: "You are an expert at writing highly engaging titles."
  },
  {
    role: "user",
    content:
      "Generate a high-engagement title for the following content:\n\n" +
      "<your content here>"
  }
];

const output = await generator(messages, {
  max_new_tokens: 40,
  temperature: 0.7,
  do_sample: true,
  top_p: 0.9,
  repetition_penalty: 1.1,
});

console.log(output);
```

## Local Inference

Inference runs in the browser using the user's GPU through WebGPU.

No API key or inference server is required for browser-local inference.

After the model files have been downloaded, they can be cached locally by the browser for subsequent use.

## Python / ONNX Runtime

The model can also be loaded with Optimum and ONNX Runtime:

```python
from optimum.onnxruntime import ORTModelForCausalLM
from transformers import AutoTokenizer

model_id = "csankalp21/headlinegpt-onnx"

tokenizer = AutoTokenizer.from_pretrained(model_id)

model = ORTModelForCausalLM.from_pretrained(
    model_id
)
```

## Example

**Input:**

> Apple is developing a new generation of artificial intelligence tools designed to make its devices more useful and personalized.

**Generated title:**

> **Apple's New AI Tools Will Make Your Devices More Useful**

## Performance & Requirements

WebGPU support and performance depend on the user's browser, operating system, GPU, available VRAM, and browser implementation.

The FP16 model is approximately **3.7 GB**, so the initial download can be substantial. Model files may be cached locally by the browser after the first download.

## Limitations

* English-focused; performance may vary substantially on other languages.
* Quality can vary outside the content distribution used during training.
* The model may occasionally introduce details that are not explicitly present in the source content.
* Engagement-oriented titles are not guaranteed to achieve higher engagement.
* Browser performance varies significantly across devices.
* The current FP16 release requires a relatively large initial download.
* WebGPU availability depends on browser and hardware support.

## Related Models and Resources

* **Main model:** [csankalp21/headlinegpt](https://huggingface.co/csankalp21/headlinegpt)
* **Transformers.js:** [Documentation](https://huggingface.co/docs/transformers.js)
* **Qwen2.5-1.5B-Instruct:** [Base model](https://huggingface.co/Qwen/Qwen2.5-1.5B-Instruct)

## License

Apache 2.0.

Please also review the license and usage terms of the underlying [Qwen2.5-1.5B-Instruct](https://huggingface.co/Qwen/Qwen2.5-1.5B-Instruct) model.

```
```