Create README.md

README.md

@@ -0,0 +1,41 @@
+---
+language:
+- de
+license: mit
+library_name: transformers
+pipeline_tag: text-generation
+tags:
+- t5
+- german
+- wechsel
+- cross-lingual
+datasets:
+- unpaywall-scientific
+---
+
+# DE-T5-Sci-Transfer-Init
+
+WECHSEL-initialized checkpoint: English EN-T5-Sci weights + German tokenizer (`GermanT5/t5-efficient-gc4-german-base-nl36`) aligned using WECHSEL (static embeddings + bilingual dictionary). **No additional German training** after transfer. Folder includes `transfer_metadata.pt` with alignment diagnostics.
+
+## Model Details
+- Embedding init: Orthogonal Procrustes map (fastText n-gram embeddings) + temperature-weighted mixtures (k-nearest neighbors)
+- Special tokens: `<extra_id_0..99>` aligned, sentinel behavior preserved
+- Tokenizer: GermanT5 SentencePiece (files bundled here)
+
+## Evaluation (Global-MMLU, zero-shot)
+| Metric | EN | DE |
+| --- | --- | --- |
+| Overall accuracy | 0.2434 | 0.2463 |
+| Humanities | 0.2485 | 0.2559 |
+| STEM | 0.2391 | 0.2445 |
+| Social Sciences | 0.2317 | 0.2307 |
+| Other | 0.2517 | 0.2491 |
+
+This demonstrates immediate cross-lingual transfer without any German gradient steps.
+
+## Intended Use
+Starting point for German continued pretraining or fine-tuning where English scientific knowledge should be retained but a German tokenizer is required.
+
+## Limitations
+- No German data exposure beyond embedding alignment; you should run additional continued pretraining (see next model) for best performance.
+- Still limited to 512-token context.