Vino1502/scihigh-2026-task2-t5-large

Fine-Tuned FLAN-T5-Large for SciHigh 2026 (Task 2)

This repository contains a Parameter-Efficient Fine-Tuning (PEFT) LoRA adapter built on top of google/flan-t5-large. It was trained specifically for the SciHigh 2026 (Subtask 2) competition to automatically generate concise and accurate scientific titles given research paper abstracts.

Model Details

• Developed by: Vino1502
• Model Type: Sequence-to-Sequence (Seq2Seq) Transformer with LoRA
• Language: English
• Base Model: google/flan-t5-large
• Task: Scientific Abstract-to-Title Generation (SciHigh 2026 - Subtask 2)

Uses

Direct Use

This model is intended for scientific title generation. Given a research paper abstract, the model generates a concise scientific title.

How to Get Started with the Model

You can load and run inference with this model using the following code:

import torch
from transformers import AutoTokenizer, AutoModelForSeq2SeqLM
from peft import PeftModel

# 1. Configuration
BASE_MODEL_NAME = "google/flan-t5-large"
ADAPTER_REPO_ID = "Vino1502/scihigh-2026-task2-t5-large"

# 2. Load Tokenizer & Base Model
tokenizer = AutoTokenizer.from_pretrained(BASE_MODEL_NAME)
base_model = AutoModelForSeq2SeqLM.from_pretrained(
    BASE_MODEL_NAME,
    dtype=torch.float16 if torch.cuda.is_available() else torch.float32,
    device_map="auto"
)

# 3. Load LoRA Adapter from your HF Repo
model = PeftModel.from_pretrained(base_model, ADAPTER_REPO_ID)
model.eval()

# 4. Sample Inference
abstract_text = "Your scientific abstract goes here..."
prompt = f"Generate a concise scientific title for this abstract: {abstract_text}"

# Tokenize and place tensors on model's device
inputs = tokenizer(prompt, return_tensors="pt", max_length=512, truncation=True).to(model.device)

with torch.no_grad():
    outputs = model.generate(
        **inputs,
        max_length=64,
        num_beams=2,
        early_stopping=True
    )

predicted_title = tokenizer.decode(outputs[0], skip_special_tokens=True)
print("Predicted Title:", predicted_title)