README.md

---
license: gemma
base_model: google/gemma-4-31b-it
tags:
- relation-extraction
- romanian
- cross-lingual
- qlora
- peft
- lora
- end-to-end
language:
- ro
- en
library_name: peft
pipeline_tag: text-generation
---

# Gemma 4 31B QLoRA adapter — Romanian/English End-to-End Relation Extraction

[Dragoș Mitruț Vasile](https://scholar.google.com/citations?user=eD-SutAAAAAJ) · [Elena-Simona Apostol](https://scholar.google.com/citations?user=XUZcjpEAAAAJ) · [Stefan-Adrian Toma](https://scholar.google.com/citations?user=wsz8cUgAAAAJ) · [Adrian Paschke](https://scholar.google.com/citations?user=D_ZARycAAAAJ) · [Ciprian-Octavian Truică](https://scholar.google.com/citations?user=ZOKqr-QAAAAJ)

[![Paper](https://img.shields.io/badge/Paper-InProgress-blue)](https://When-Paper-Appears-it-Will-Work.com)
[![arXiv](https://img.shields.io/badge/arXiv-WIP-b31b1b)](https://arxiv.org/abs/WIP)
[![Website](https://img.shields.io/badge/Project-Website-green)](https://github.com/DS4AI-UPB/crosslingual-romanian-re)
[![GitHub](https://img.shields.io/badge/GitHub-Code-black?logo=github)](https://github.com/DS4AI-UPB/crosslingual-romanian-re)
[![License](https://img.shields.io/badge/License-CC%20BY--NC--SA%204.0-lightgrey)](https://creativecommons.org/licenses/by-nc-sa/4.0/)

QLoRA adapter for `google/gemma-4-31b-it`, fine-tuned for **End-to-End Relation Extraction** on a Romanian translation of SemEval-2010 Task 8 plus the original English data. Unlike the classification adapter, entity tags are not given: the model reads a plain sentence and generates both entities and the relation between them as a single structured output.

<!-- This adapter accompanies the SYNASC 2026 paper *"Cross-lingual Relation Extraction with Large Language Models: Zero-Shot, Few-Shot, and Fine-Tuned Evaluation on Romanian"*. -->

## Results (SemEval-2010 Task 8 test set)

| Language | Exact match | Relation match | Entity match |
|----------|-------------|----------------|--------------|
| English  | 0.719       | 0.816          | 0.796        |
| Romanian | 0.674       | 0.809          | 0.751        |

QLoRA raises exact match by about 39pp over zero-shot in both languages. This is the setting where the LLM clearly outperforms encoder classifiers, which require a separate entity recognition stage and do not apply directly.

## Training

- Base: `google/gemma-4-31b-it`, loaded in 4-bit
- LoRA: rank 32, alpha 64, dropout 0.05, all attention and MLP projections
- 3 epochs, effective batch size 16, peak LR 2e-4, cosine decay, 5% warmup
- Combined English + Romanian training data
- Single NVIDIA A100 40GB

## Usage

```python
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

base = AutoModelForCausalLM.from_pretrained("google/gemma-4-31b-it", load_in_4bit=True, device_map="auto")
model = PeftModel.from_pretrained(base, "<HF_REPO_E2E>")
tok = AutoTokenizer.from_pretrained("<HF_REPO_E2E>")
```

The model outputs a JSON object with the two entities and the relation. The exact prompt and parsing logic are in `run_inference.py` in the [code repository](<GITHUB_REPO>).

## Limitations

The Romanian training data is machine-translated with automatic post-validation. Because end-to-end extraction scores predictions against the gold entity spans taken from the markers, untranslated or misplaced gold spans penalize the Romanian numbers, which should be read as a lower bound. See the paper for the translation quality analysis.

## Citation

```bibtex
@misc{vasile2026crosslingual,
  title  = {Cross-lingual Relation Extraction with Large Language Models: Zero-Shot, Few-Shot, and Fine-Tuned Evaluation on Romanian},
  author = {Vasile, Drago\c{s}-Mitru\c{t} and Apostol, Elena-Simona and Toma, \c{S}tefan-Adrian and Paschke, Adrian and Truic\u{a}, Ciprian-Octavian},
  year   = {2026},
  note   = {Preprint}
}
```