Create README.md

README.md

@@ -0,0 +1,54 @@
+---
+language:
+- en
+metrics:
+- accuracy
+---
+# Custom Transformer for Amazon Sentiment Analysis
+
+This repository contains a **custom-built Transformer Encoder** model for binary sentiment classification, trained on the **Amazon Polarity** dataset.
+
+## 🚀 Model Overview
+Unlike standard pre-trained models (like BERT), this architecture was built from scratch to demonstrate the implementation of **Self-Attention** and **Positional Encodings** in PyTorch.
+
+* **Architecture**: 4-Layer Transformer Encoder
+* **Task**: Binary Sentiment Analysis (Positive/Negative)
+* **Accuracy**: 89.67% on Test Set
+* **Parameters**: Optimized for efficient inference on edge devices
+
+## 🛠️ Technical Specifications
+* **Embedding Dimension**: 128
+* **Attention Heads**: 8
+* **Feed-Forward Dimension**: 512
+* **Sequence Length**: 300 tokens
+* **Optimizer**: AdamW with Linear Learning Rate Warmup
+
+## 💻 Training Environment
+This model was trained locally on an **Apple Mac mini M4** with **24GB of Unified Memory**.
+* **Accelerator**: Metal Performance Shaders (MPS)
+* **Dataset**: Subset of 500,000 samples from Amazon Polarity
+
+## 📈 Performance & Insights
+During development, the model was benchmarked against a Bidirectional LSTM. The Transformer architecture achieved a **~5% improvement in accuracy**, demonstrating its superior ability to capture long-range dependencies in product reviews.
+
+## 📖 How to Use
+To use this model, ensure you have `torch` and `transformers` installed.
+
+```python
+from transformers import DistilBertTokenizer
+import torch
+
+# 1. Initialize Tokenizer
+tokenizer = DistilBertTokenizer.from_pretrained('Nefflymicn/amazon-sentiment-transformer')
+
+# 2. Load Model (Architecture must match)
+model = TransformerSentimentModel(
+            vocab_size=tokenizer.vocab_size,
+            embed_dim=128,
+            num_heads=8,
+            ff_dim=512,
+            num_layers=4,
+            output_dim=2
+        ) 
+model.load_state_dict(torch.load("pytorch_model.bin", map_location='cpu'))
+model.eval()