Update README.md

README.md

@@ -21,35 +21,32 @@ metrics:
 - recall
 - f1
 model-index:
-  - name: SGD Logistic Regression
-    results:
-      - task:
-          type: text-classification
-          name: Programming Language Classification
-        dataset:
-          type: custom
-          name: Code Language Classification Dataset
-        metrics:
-          - type: accuracy
-            value: 91.1
-            name: SGD Test Accuracy
-
-  - name: FastText
-    results:
-      - task:
-          type: text-classification
-          name: Programming Language Classification
-        dataset:
-          type: custom
-          name: Code Language Classification Dataset
-        metrics:
-          - type: accuracy
-            value: 95.5
-            name: FastText Test Accuracy
+- name: SGD Logistic Regression
+  results:
+  - task:
+      type: text-classification
+      name: Programming Language Classification
+    dataset:
+      type: custom
+      name: Code Language Classification Dataset
+    metrics:
+    - type: accuracy
+      value: 91.1
+      name: SGD Test Accuracy
+- name: FastText
+  results:
+  - task:
+      type: text-classification
+      name: Programming Language Classification
+    dataset:
+      type: custom
+      name: Code Language Classification Dataset
+    metrics:
+    - type: accuracy
+      value: 95.5
+      name: FastText Test Accuracy
 datasets:
 - kaushik-harsh-99/Code-Language-Classification
-base_model:
-- answerdotai/ModernBERT-base
 ---
 # Experiment Timeline
 
@@ -277,16 +274,16 @@ FastText proved to be one of the strongest accuracy-to-compute trade-offs observ
 
 ## Motivation
 
-While FastText achieved strong results, it still relies primarily on local token and character patterns.
+After achieving strong results with FastText, the next stage of the project explored whether transformer architectures could further improve programming language classification performance.
 
-Modern transformer architectures can model:
+Unlike FastText, transformer models can learn:
 
 - Long-range dependencies
+- Global context
 - Structural relationships
-- Contextual representations
-- Semantic information
+- Context-aware representations
 
-The next phase aims to determine the maximum achievable accuracy on the dataset.
+The goal was to determine whether additional model capacity translates into meaningful real-world gains for source code language identification.
 
 ---
 
@@ -300,82 +297,82 @@ The next phase aims to determine the maximum achievable accuracy on the dataset.
 
 - Sequence Classification
 
-### Training Features
+## Results
 
-- Mixed Precision Training
-- Gradient Checkpointing
-- Dynamic Padding
-- Large Effective Batch Size
-- Validation Tracking Throughout Training
-- Automatic Best Checkpoint Selection
+### Approximate Test Accuracy
 
----
+**~97–98%**
 
-## Current Status
+### Improvement Over FastText
 
-**Training In Progress**
+**~2–3 percentage points**
 
-The dataset contains approximately:
+---
 
-```text
-1.6 million training samples
-```
+## Observations
 
-Validation metrics are evaluated multiple times per epoch and checkpoints are saved throughout training to enable detailed learning curve analysis.
+ModernBERT achieved the highest overall accuracy among all models tested.
 
----
+However, experimentation revealed that the improvement over FastText was relatively small considering the large increase in computational requirements.
 
-## Objectives
+Compared with FastText:
 
-The ModernBERT experiments aim to answer:
+- Training time increased dramatically
+- GPU memory usage increased significantly
+- Inference became substantially slower
+- Model size increased considerably
+- Deployment became more complex
 
-1. What is the maximum achievable accuracy on this dataset?
-2. Which language pairs remain difficult after FastText?
-3. How much improvement does contextual modeling provide?
-4. Is the improvement sufficient to justify the additional compute cost?
+Although ModernBERT achieved higher accuracy, the gain remained limited relative to the increase in compute.
 
 ---
 
-# Planned Future Work
+## Key Finding
 
-## Knowledge Distillation
+For programming language classification specifically:
 
-After training the ModernBERT teacher model:
+> Transformer-based neural networks do not appear to be the most efficient solution for this task.
 
-```text
-ModernBERT Teacher
-        ↓
-Student Model
-```
+Programming languages contain strong lexical and structural signals that can already be captured extremely effectively using lightweight approaches.
+
+FastText achieved performance surprisingly close to ModernBERT while requiring only a fraction of:
 
-The goal is to transfer knowledge from the transformer into smaller models.
+- Compute
+- Training time
+- Memory
+- Storage
+- Inference cost
+
+---
 
-### Potential Student Architectures
+# Current Benchmark Summary
 
-- Distilled ModernBERT variants
-- Compact transformer models
-- FastText students
-- Lightweight deployment models
+| Model | Test Accuracy | Relative Compute |
+|--------|--------------:|-----------------:|
+| SGD Logistic Regression | ~91.1% | Very Low |
+| FastText | ~95.5% | Low |
+| ModernBERT-base | ~97–98% | Extremely High |
 
 ---
 
+# Current Conclusions
 
-# Current Benchmark Summary
+## 1. Classical machine learning remains surprisingly competitive
 
-| Model | Accuracy |
-|---------|---------:|
-| SGD Logistic Regression | ~91.1% |
-| FastText | ~95.5% |
-| ModernBERT-base | Training |
+Character-level linear models establish a strong baseline even at large scale.
 
 ---
 
-# Key Takeaways So Far
+## 2. FastText provides the strongest accuracy-to-compute ratio
+
+Current experiments indicate FastText delivers the best balance of:
+
+- Accuracy
+- Training speed
+- Inference speed
+- Memory efficiency
+- Deployment simplicity
 
-- Character n-gram features provide a surprisingly strong baseline for programming language classification.
-- FastText delivers a substantial performance improvement while maintaining practical training and inference costs.
-- Careful preprocessing is critical, particularly when using FastText label prefixes.
-- Source code classification benefits heavily from character-level information.
-- Larger neural models should be evaluated not only on accuracy but also on deployment cost, memory footprint, and inference speed.
+while remaining within only a few percentage points of transformer performance.
 
-The project continues to evolve toward a high-accuracy, deployment-friendly code language classifier capable of operating efficiently at large scale.
+---