Update README.md

README.md

@@ -1,3 +1,94 @@
 ---
 license: mit
+language:
+  - en
+pipeline_tag: text2text-generation
+library_name: pytorch
+tags:
+  - transformer
+  - pytorch
+  - from-scratch
+  - linux
+  - seq2seq
 ---
+
+# Linux Command Transformer
+
+A small Transformer Encoder-Decoder trained from scratch with PyTorch to generate Linux commands from natural language descriptions.
+
+This model was developed as a personal learning project to better understand the Transformer architecture, its training process, and its limitations.
+
+The Transformer implementation was guided by [Umar Jamil's Transformer from Scratch walkthrough](https://www.youtube.com/watch?v=ISNdQcPhsts).
+
+## Model Architecture
+
+The model follows the original Transformer Encoder-Decoder architecture introduced in *Attention Is All You Need*.
+
+| Parameter | Value |
+|---|---|
+| Encoder Layers | 2 |
+| Decoder Layers | 2 |
+| Model Dimension | 64 |
+| Attention Heads | 4 |
+| Feed Forward Dimension | 128 |
+| Tokenizer | GPT-2 |
+| Maximum Source Length | 64 |
+| Maximum Target Length | 96 |
+
+The model was trained with randomly initialized weights. No pretrained language model weights were used.
+
+## Training
+
+The model was trained on approximately 8,000 natural language–Linux command pairs, using an NVIDIA RTX 4060 Laptop GPU with 8 GB of VRAM.
+
+Training configuration:
+
+- Optimizer: AdamW
+- Learning rate: 0.001
+- Loss: Cross-Entropy
+- Gradient clipping: 1.0
+- Maximum epochs: 10
+
+The best checkpoint was selected at epoch 6, with a validation loss of **2.0178**.
+
+## Evaluation Results
+
+The model was evaluated on 796 test examples.
+
+| Metric | Accuracy |
+|---|---:|
+| Exact Match | 11.68% |
+| Command Accuracy | 81.03% |
+| First Argument Accuracy | 36.35% |
+| Command + First Argument Accuracy | 35.45% |
+
+The model identifies the correct main Linux utility in many cases, but generating complete commands with accurate arguments remains challenging.
+
+The argument-related metrics were calculated on 773 examples containing at least one argument. Metrics use whitespace-based command splitting rather than full shell parsing.
+
+## Usage
+
+This is a custom PyTorch architecture and cannot be loaded directly using `AutoModel.from_pretrained()`.
+
+The repository provides:
+
+- `model.pt` — Trained model weights
+- `config.json` — Architecture configuration
+- `tokenizer/` — GPT-2 tokenizer configuration with additional special tokens
+
+To run inference, use the implementation provided in the associated GitHub repository.
+
+The inference script downloads the model from Hugging Face, reconstructs the Transformer architecture, loads the weights, and generates Linux commands.
+
+## Limitations
+
+The model was trained on a relatively small dataset, and training experiments were constrained by the available GPU resources.
+
+Although it achieves 81.03% Command Accuracy, its Exact Match Accuracy is only 11.68%.
+
+This model is intended for educational and experimental purposes. Generated commands should be reviewed before execution, especially commands that modify or delete files.
+
+## References
+
+- [Attention Is All You Need](https://arxiv.org/abs/1706.03762)
+- [Umar Jamil — Coding a Transformer from Scratch](https://www.youtube.com/watch?v=ISNdQcPhsts)