Update README.md
Browse files
README.md
CHANGED
|
@@ -17,7 +17,7 @@ library_name: pytorch
|
|
| 17 |
pipeline_tag: text-generation
|
| 18 |
---
|
| 19 |
|
| 20 |
-
#
|
| 21 |
|
| 22 |
This repository hosts the **weights, data, and results** of the architectural-introspection
|
| 23 |
experiments ("meta-transformers"). The idea: instead of text-based reflection, give a model
|
|
@@ -29,7 +29,7 @@ modules. At inference the model runs two passes — it reads its own per-layer a
|
|
| 29 |
encodes them into *cognitive tokens*, and injects them back via gated cross-attention. This
|
| 30 |
yields calibrated refusal and self-correction.
|
| 31 |
|
| 32 |
-
**Code, training and evaluation scripts:** https://
|
| 33 |
|
| 34 |
---
|
| 35 |
|
|
@@ -89,7 +89,7 @@ Details in the code repository's `docs/`.
|
|
| 89 |
record:
|
| 90 |
|
| 91 |
```python
|
| 92 |
-
# from the
|
| 93 |
# see src/phase2_selective_llama8b/04_evaluate.py for the full loading + two-pass example
|
| 94 |
from src.phase2_selective_llama8b.reflexion_model_selective import ReflexionModelSelective
|
| 95 |
# build frozen Llama-3.1-8B-Instruct, wrap it, load the encoder + cross-attention weights
|
|
@@ -133,8 +133,8 @@ Checkpoint → code mapping:
|
|
| 133 |
## Citation
|
| 134 |
|
| 135 |
```bibtex
|
| 136 |
-
@software{
|
| 137 |
-
title = {
|
| 138 |
year = {2026}
|
| 139 |
}
|
| 140 |
```
|
|
|
|
| 17 |
pipeline_tag: text-generation
|
| 18 |
---
|
| 19 |
|
| 20 |
+
# Meta-transformers — Architectural Introspection for LLMs
|
| 21 |
|
| 22 |
This repository hosts the **weights, data, and results** of the architectural-introspection
|
| 23 |
experiments ("meta-transformers"). The idea: instead of text-based reflection, give a model
|
|
|
|
| 29 |
encodes them into *cognitive tokens*, and injects them back via gated cross-attention. This
|
| 30 |
yields calibrated refusal and self-correction.
|
| 31 |
|
| 32 |
+
**Code, training and evaluation scripts:** https://codeberg.org/imperius/meta-transformers-ENG.git
|
| 33 |
|
| 34 |
---
|
| 35 |
|
|
|
|
| 89 |
record:
|
| 90 |
|
| 91 |
```python
|
| 92 |
+
# from the meta-transformers repository
|
| 93 |
# see src/phase2_selective_llama8b/04_evaluate.py for the full loading + two-pass example
|
| 94 |
from src.phase2_selective_llama8b.reflexion_model_selective import ReflexionModelSelective
|
| 95 |
# build frozen Llama-3.1-8B-Instruct, wrap it, load the encoder + cross-attention weights
|
|
|
|
| 133 |
## Citation
|
| 134 |
|
| 135 |
```bibtex
|
| 136 |
+
@software{meta_transformers_core_2026,
|
| 137 |
+
title = {Meta-transformers: Architectural Introspection for Large Language Models},
|
| 138 |
year = {2026}
|
| 139 |
}
|
| 140 |
```
|