Update README.md

README.md

@@ -1,3 +1,4 @@
+
 ---
 license: apache-2.0
 language:
@@ -10,6 +11,7 @@ tags:
 - pytorch
 - text-generation
 - openwebtext
+- custom_code
 ---
 
 # Q-MoE-400
@@ -18,6 +20,39 @@ tags:
 
 This model serves as a research artifact for studying the compute efficiency of sparse architectures compared to dense transformers. It demonstrates how routing mechanisms can enable high-capacity models with lower inference costs.
 
+## 💻 Usage
+
+You can use this model directly with the Hugging Face `transformers` library. Since this model uses a custom architecture, `trust_remote_code=True` is required.
+
+```python
+from transformers import AutoTokenizer, AutoModelForCausalLM
+import torch
+
+path = "QuarkML/Q-MoE-400"
+
+# Load tokenizer and model
+tok = AutoTokenizer.from_pretrained(path, trust_remote_code=True)
+model = AutoModelForCausalLM.from_pretrained(
+    path,
+    trust_remote_code=True,
+    dtype=torch.float16,  # optional but recommended for GPU
+    device_map="auto"     # automatically maps to available device (CUDA/CPU)
+)
+
+# Generate text
+inputs = tok("Artificial Neural network are ", return_tensors="pt")
+inputs = {k: v.to(model.device) for k, v in inputs.items()}
+
+out = model.generate(
+    **inputs,
+    max_new_tokens=50,
+    do_sample=True,
+    temperature=0.8
+)
+
+print(tok.decode(out[0], skip_special_tokens=True))
+```
+
 ## 🎯 Project Goal
 
 The primary goal of the Q-MoE project is to investigate:
@@ -31,7 +66,7 @@ The model was evaluated at step **79,100**. The final validation metrics indicat
 
 | Metric | Value | Description |
 | :--- | :--- | :--- |
-| **Step** | 90,000 | Total training steps |
+| **Step** | 79,100 | Total training steps |
 | **Train Loss** | 3.2190 | Total training loss (CE + Aux) |
 | **Train CE** | 3.0987 | Cross-Entropy loss on training data |
 | **Val Loss** | 3.2028 | Total validation loss |
@@ -41,7 +76,7 @@ The model was evaluated at step **79,100**. The final validation metrics indicat
 
 ### Training Progress
 
-![image](https://cdn-uploads.huggingface.co/production/uploads/64054e5e0ab5e22719fc179f/CALqiEjv1HahbLnZrbLPi.png)
+![Training Curve](https://cdn-uploads.huggingface.co/production/uploads/64054e5e0ab5e22719fc179f/CALqiEjv1HahbLnZrbLPi.png)
 
 ## 📝 Generation Example
 
@@ -57,24 +92,6 @@ The following example demonstrates the model's generation capabilities after tra
 > While this is an obvious disadvantage for software development, the reality is that there are many aspects of software that are highly important to a programmer's day-to-day life. This is why even a moderately experienced programmer should never be concerned about this.
 >
 > The best way to learn about your code is by going through its source code. That way, it's always safe to do something new when writing code. This gives your programmer freedom and confidence.
->
-> One of the most popular techniques for coding small functions in your computer is "code reuse." The same technique can be used by programmers in any number of different ways. Some programmers might write code to get the job done, and others develop it to get it to the end. They use the same tools as most programmers to get the job done.
->
-> The best way to learn about your code is by going through its source code. That way, it's always safe to do something new when writing code. That technique can help you get the job done. It provides a way to write code that is easy to understand and maintain, and makes debugging easier.
-
-## 🛠️ Repository Contents
-
-This repository contains checkpoints compatible with both major frameworks:
-- **JAX/Flax:** The original training checkpoints (Orbit/Orbax format).
-- **PyTorch:** Converted weights for easier integration with the Hugging Face ecosystem (Safetensors).
-
-## 💻 Inference & Usage
-
-For inference code, architectural details, training pipeline and conversion scripts, please visit the official GitHub repository:
-
-👉 **[https://github.com/sidharth72/Q-MoE-400 ]**
-
-To run the model, you will likely need the custom modeling code provided in the GitHub repo, as this uses a specialized sparse MoE architecture.
 
 ## ⚙️ Training Details
 
@@ -95,5 +112,6 @@ If you find this model or the associated research useful, please cite:
   year = {2025},
   publisher = {Hugging Face},
   journal = {Hugging Face Repository},
-  howpublished = {\url{[https://huggingface.co/QuarkML/Q-MoE-400](https://huggingface.co/QuarkML/Q-MoE-400)}}
-}
+  howpublished = {\url{https://huggingface.co/QuarkML/Q-MoE-400}}
+}
+```