Improve model card: Add abstract and full paper title to link

README.md

@@ -1,15 +1,15 @@
 ---
-language: en
-license: apache-2.0
-library_name: transformers
-tags:
-  - tptt
-  - peft
-  - trust_remote_code
-pipeline_tag: text-generation
 base_model: allenai/OLMo-1B-hf
 datasets:
 - yahma/alpaca-cleaned
+language: en
+library_name: transformers
+license: apache-2.0
+pipeline_tag: text-generation
+tags:
+- tptt
+- peft
+- trust_remote_code
 ---
 
 # Titanesque-OLMo-1B-hf
@@ -34,8 +34,10 @@ datasets:
 
 Titanesque version of `allenai/OLMo-1B-hf` with parallel linearized attention (TPTT 😊) and PEFT.
 
-The architecture was presented in the paper [TPTT](https://huggingface.co/papers/2506.17671).
+The architecture was presented in the paper [TPTT: Transforming Pretrained Transformers into Titans](https://huggingface.co/papers/2506.17671).
 
+## Abstract
+Transformer-based large language models (LLMs) have achieved strong performance across many natural language processing tasks. Nonetheless, their quadratic computational and memory requirements, particularly in self-attention layers, pose challenges for efficient inference on long contexts and for deployment in resource-limited environments. We present TPTT (Transforming Pretrained Transformers into Titans), a framework designed to augment pretrained Transformers with linearized attention (LiZA) and internal memory gating via Memory as Gate (MaG), applied without full retraining. TPTT supports parameter-efficient fine-tuning (LoRA) and integrates with standard toolkits such as Hugging Face Transformers. We evaluated TPTT on several pretrained models, including Llama-1B, OlMoE-1B-7B, Qwen2.5-1.5B, Gemma3-270m, OpenELM-1.3B, and Mistral-7B, in order to assess applicability across architectures of different scales. Experiments on models with approximately 1 billion parameters, evaluated primarily on the MMLU benchmark, suggest potential improvements in both efficiency and accuracy compared to baseline models. For example, Titans-Llama-1B exhibited up to a 20% relative increase in Exact Match scores in one-shot evaluation. An additional finding is that it is possible to convert a quadratic-attention model into a purely linear-attention model using the DeltaProduct mechanism. All training runs were carried out with modest computational resources. These preliminary findings indicate that TPTT may help adapt pretrained LLMs for long-context tasks with limited overhead. Further studies on larger models and a broader set of benchmarks will be necessary to evaluate the generality and robustness of the framework. Code is available at this https URL . Python package at this https URL .
 
 ## Model list
 
@@ -45,7 +47,7 @@ Classic model parameter with LiZA injection :
 |-------------------------------|----------------------|------------|------------|----------------|---------------|------|-------------------------------------------------------|
 | delta_rule  | 8192 (default)       | 0.5        | False      | 64             | False         | Yes  | Parallel linearized attention with delta_rule operator|
 | delta_rule_gelu | 8192 (default) | 0.5        | False      | 64             | False         | Yes  | Non-linear operator with gelu activation              |
-| delta_product    | 8192 (default) | 0.5        | False      | 64             | False         | Yes  | Second order operator with derivative trick              |
+| delta_product    | 8192 (default) | 0.5        | False         | 64             | False         | Yes  | Second order operator with derivative trick              |
 | delta_product_r  | 8192 (default) | 0.5        | False      | 64             | False         | Yes  | Second order operator with rotative trick             |
 | delta_product_c  | 8192 (default) | 0.5        | False      | 64             | False         | Yes  | Second order operator with combined trick             |
 
@@ -73,5 +75,4 @@ print(tokenizer.decode(outputs, skip_special_tokens=True))
 
 If you use TPTT in your academic work, please cite [Furfaro](https://huggingface.co/ffurfaro). For questions or support, please open an issue on the [GitHub repository](https://github.com/fabienfrfr/tptt) or contact the maintainer.
 
-
 ---