README.md

---
library_name: transformers
tags:
- generated_from_trainer
datasets:
- Helsinki-NLP/opus-100
model-index:
- name: string-repetition-tiny
  results: []
license: mit
language:
- en
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# WeLT String Repetition

This model is traained using [this](https://github.com/sign/WeLT/blob/eab950ace0322f3299997dd5c9ff34f179ecc6a4/training/experiments/easy-tasks/string-repetition.yaml) config.
It is designed to take in English strings, and repeat them.

It is published here, so that it can be used in tests.

## Usage
```python
from pathlib import Path

import torch
from transformers import GenerationConfig
from transformers.trainer_utils import get_last_checkpoint

from welt.model import WordLatentTransformerForCausalLM
from welt.processor import TextImageProcessor


with torch.no_grad():
    model = WordLatentTransformerForCausalLM.from_pretrained("sign/WeLT-string-repetition")
    processor = TextImageProcessor.from_pretrained(model_path)
    model.eval()

    texts = [
        # Texts from validation set
        "<text>\x0EWouldn't it be more cruel for society to let people die... - ... when with some effort it could save them?\x0F<repeat> ",
        "<text>\x0ESuperman's exact opposite who lives in the backwards Bizarro World.\x0F<repeat> ",
        "<text>\x0EYOu dOn't know the half Of it.\x0F<repeat> ",
    ]

    inputs = processor(texts, collated=True, packed=False)

    outputs = model.generate(
        **inputs,
        processor=processor,
        max_generated_words=32,
    )
    for text, output in zip(texts, outputs, strict=False):
        print(f"Generated for '{text}': {output}")
```


### Framework versions

- Transformers 4.57.3
- Pytorch 2.9.1+cu130
- Datasets 4.4.1
- Tokenizers 0.22.1