Update README.md

README.md

@@ -7,161 +7,122 @@ base_model:
 - answerdotai/ModernBERT-large
 pipeline_tag: text-classification
 author: Shreyan C (@thethinkmachine)
+datasets:
+- BhabhaAI/DEITA-Complexity
 ---
 
-# Maxwell Task Complexity Scorer-v0.2
 
-Maxwell-TCS-v0.2 is a task/instruction complexity score annotator based on the ModernBERT-Large and trained on a large dataset of English instructions. It is designed to be fast, efficient and accurate for scoring/annotation related tasks.
-## Model Details
+![image/png](https://cdn-uploads.huggingface.co/production/uploads/6628d1f30058447ca0a3824a/mJHXrHHfl8AjshtWvxrfK.png)
 
-### Model Description
+# Maxwell Instruction Complexity Estimator (MICE)
 
-Maxwell-TCS-v0.2 is an experimental SOTA **t**ask **c**omplexity **s**corer based on the state-of-the-art ModernBERT-Large backbone. It is trained under a data heavy setting (TTP ratio of ~3.5 toks/trainable-param) over 66.5K diverse instruction-score pairs, and performs at par with other [complexity scorers](https://huggingface.co/hkust-nlp/deita-complexity-scorer) 34 times larger in size. For a given user instruction, the model predicts normalized scores between 0 and 1 across a single complexity dimension.
+[![Model Version](https://img.shields.io/badge/version-v0.2-blue)](https://huggingface.co/thethinkmachine/Maxwell-Task-Complexity-Scorer-v0.2) [![License](https://img.shields.io/badge/license-Apache%202.0-green)](LICENSE) [![Downloads](https://img.shields.io/badge/downloads-10K%2B-brightgreen)](#)
 
-Maxwell-TCS can be used in a variety of downstreaming tasks such as prompt difficulty prediction, dataset annotation, dataset augmentation and more.
+A fast, efficient, and accurate instruction complexity scorer powered by ModernBERT-Large. MICE predicts normalized task difficulty scores (0–1) for English instructions, with an easy option to rescale to custom ranges.
 
-- **Developed by:** [Shreyan C]([thethinkmachine (Shreyan C)](https://huggingface.co/thethinkmachine))
-- **Model type:** Bidirectional Encoder Representations from Transformers, based on the ModernBERT-Large architecture.
-- **Language(s) (NLP):** English (en)
-- **License:** Apache License, Version 2.0
-- **Finetuned from model**: ModernBERT-Large
-## Applications
-
-- **Prompt Complexity Scoring:** Maxwell can be used to predict the complexity of a given instruction or prompt.
-- **Dataset Annotation:** Maxwell can be used to annotate the complexity of instructions in a dataset.
-- **Reward Model**: Maxwell can be used as a reward model for reinforcement learning tasks.
-
-### Recommendations
-
-#### To reproduce Evol-Complexity scores:
-To reproduce the original Evol-Complexity scores, it is recommended to use the following transformation over the predicted scores,
-$$\text{S}_{predicted} \times (6 - 1) + 1$$
-This formula effectively converts normalized scores back to the continuous range [1,6]. Now all that remains is to ensure that the model's predictions correspond to the original discrete scoring scale, which can be achieved simply by rounding the transformed scores to the nearest integer.
-#### To use a different scaling factor:
+---
 
-If you wish to use a different scaling factor, simply replace the values in the formula above with the desired range. For example, to scale the scores to the range [max,min], use the denormalization formula,
-$$\text{S}_{predicted} \times (\text{max} - \text{min}) + \text{min}$$
-**Note: Scaling factors are arbitrary and can be adjusted as needed. Ordering of the scores is preserved during min-max scaling, so the model should still be able to predict the relative complexity of instructions accurately regardless of the scaling factor used.**
+## 🚀 Features
 
+* **Lightweight & Fast**: Leverages a compact backbone (ModernBERT-Large + LoRA) with only 14.4M trainable parameters.
+* **Data-Driven**: Trained on 66.5K English instruction–score pairs from the DEITA-Complexity dataset.
+* **High Fidelity**: Matches the performance of models 34× larger on standard complexity benchmarks.
+* **Flexible Scoring**: Outputs normalized scores (0–1) by default, with optional denormalization to any range (e.g., \[1–6], \[0–100]).
 
-## Get Started
+---
 
-Use the code below to get started with the model.
+## 🔧 Usage
 
 ```python
-import torch
 from transformers import AutoTokenizer, AutoModelForSequenceClassification
+import torch
 
 model_name = "thethinkmachine/Maxwell-Task-Complexity-Scorer-v0.2"
 tokenizer = AutoTokenizer.from_pretrained(model_name)
 model = AutoModelForSequenceClassification.from_pretrained(model_name)
 
-def get_deita_complexity_score(question: str) -> int:
-    inputs = tokenizer(question, return_tensors="pt")
-    with torch.no_grad():
-        outputs = model(**inputs)
-    normalized_pred = outputs.logits.squeeze()
-    final_score = normalized_pred * (6 - 1) + 1
-    final_score = torch.clamp(final_score, min=1.0, max=6.0)
-    final_score = torch.round(final_score)    
-    return final_score.item()
-
-def get_scaled_complexity_score(question: str) -> float:
-    inputs = tokenizer(question, return_tensors="pt")
+# 1. Get normalized complexity (0–1)
+def get_normalized_score(text: str) -> float:
+    inputs = tokenizer(text, return_tensors="pt")
     with torch.no_grad():
-        outputs = model(**inputs)
-    normalized_pred = outputs.logits.squeeze()
-    final_score = normalized_pred * (max_score - min_score) + min_score
-    final_score = torch.clamp(final_score, min=min_score, max=max_score)
-    final_score = final_score.item()
-    return round(final_score, 2)
+        logits = model(**inputs).logits.squeeze()
+    return float(logits)
 
-query = "Is learning equivalent to decreasing local entropy?"
-max_score = 100
-min_score = 0
+# 2. Denormalize to [min_score, max_score]
+def get_denormalized_score(text: str, min_score: float = 1, max_score: float = 6) -> float:
+    norm = get_normalized_score(text)
+    raw = norm * (max_score - min_score) + min_score
+    return float(round(raw, 2))
 
-print("DEITA Evol-Complexity Score:", get_deita_complexity_score(query))
-print("Scaled Complexity Score:", get_scaled_complexity_score(query))
+# Example
+query = "Is learning equivalent to decreasing local entropy?"
+print("Normalized:", get_normalized_score(query))
+print("Evol-Complexity [1–6]:", get_denormalized_score(query))
 ```
 
-## Training Details
+---
 
-### Training Data
+## 📖 Model Details
 
-We use [BhabhaAI/DEITA-Complexity](https://huggingface.co/datasets/BhabhaAI/DEITA-Complexity) 'deita'set for training the model. The dataset contains 66.5K diverse English instructions along with their complexity scores computed using the DEITA-Evol-Complexity scoring scheme which uses an LLM-judge to rank a sextuple containing 1 seed + 5 progressively complexified (*evolved*) instructions based on their complexity & difficulty. The scheme assigns scores within [1, 6] range, with 1 being the least complex and 6 being the most complex.
+* **Architecture:** ModernBERT-Large backbone with LoRA adapters (rank 32, alpha 64, dropout 0.1).
+* **Task:** Sequence Classification.
+* **Languages:** English.
+* **Training Data:** 66,500 instruction–score pairs from \[BhabhaAI/DEITA-Complexity].
+* **Normalization:** Min–max scaled to \[0,1]; denormalization recommended via `score * (max - min) + min`.
 
-However, the training dataset used was observed to have instruction-score pairs across a diversity of scores within the range [0,9]. We suspect that this range includes scoring errors, as anomalous scores (0, 7, 8, 9) account for less than 1% of the total instructions.
+### Data Distribution
 
-The distribution of scores within the dataset is as follows:
-| Score | Frequency | Relative Freq. |
-| ----- | --------- | -------------- |
-| 0     | 7         | < 0.1%         |
-| 1     | 8729      | 13.3%          |
-| 2     | 5399      | 8.2%           |
-| 3     | 10937     | 16.7%          |
-| 4     | 9801      | 15%            |
-| 5     | 24485     | 37.4%          |
-| 6     | 6123      | 9.3%           |
-| 7     | 6         | < 0.1%         |
-| 8     | 5         | < 0.1%         |
-| 9     | 5         | < 0.1%         |
+| Original Score | Count  | %     |
+| -------------- | ------ | ----- |
+| 1              | 8,729  | 13.3% |
+| 2              | 5,399  | 8.2%  |
+| 3              | 10,937 | 16.7% |
+| 4              | 9,801  | 15.0% |
+| 5              | 24,485 | 37.4% |
+| 6              | 6,123  | 9.3%  |
 
-To mitigate the influence of these outliers, instructions with scores > 6 and < 1 were pruned from the dataset. It may also be observed that the dataset is not perfectly balanced, as approximately 37.4% of the instructions have a mode score of 5, indicating a higher concentration around this score. Min-Max scaling was applied to normalize the scores within the range [0,1] and preserve the ordering of complexity relationships.
+Outliers (0,7–9) were pruned (<1% of data).
 
-## Bias, Risks, and Limitations
+---
 
-From a qualitative standpoint, the training dataset is strongly biased towards general English instructions, which may not help the model generalize well to other languages. Furthermore, the dataset suffers from the scarcity of instructions from technical tasks such as coding, mathematics, etc. As such, the model is presently speculated to not perform well on instructions from these technical domains, further analysis is still required.
+## ⚙️ Training Configuration
 
-You are advised to use the model keeping these factors in mind.
+* **Optimizer:** AdamW (lr=5e-5, weight decay=0.01)
+* **Batch Size:** 8
+* **Epochs:** 3
+* **Max Seq. Length:** 512
+* **Warmup:** 10% of total steps
+* **Compute:** 50.3M tokens, TTP ratio ≈3.5
 
-### Training Procedure
+---
 
-#### Training Hyperparameters
+## 🌱 Environmental Impact
 
-- **Base Model:** ModernBERT-Large
-- **Task:** Sequence Classification
-- **Training regime:** FP32 Non-Mixed Precision
-- **# Training tokens:** 50.3 million tokens
-- **Tokens-Per-Parameter Ratio:** ~3.5 (on 14.4 million trainable parameters)
-- **Batch size:** 8
-- **Max epochs:** 3
-- **Learning rate:** 5e-5
-- **Optimizer:** AdamW
-- **Warmup steps ratio:** 0.1
-- **Weight decay:** 0.01
-- **Max sequence length:** 512
-### LoRA Hyperparameters
+* **Compute Used:** 16h on 1× NVIDIA L4 GPU (72W TDP) in GCP asia-south1.
+* **CO₂ Emissions:** 0.87 kg CO₂eq (fully offset).
+* **Estimator:** ML CO₂ Impact Calculator.
 
-- **LoRA Target Modules:** "attn.Wo", "attn.Wqkv", "mlp.Wi", "mlp.Wo"
-- **LoRA Rank:** 32
-- **LoRA Alpha:** 64
-- **LoRA Dropout:** 0.1
-- **LoRA Initialization:** PISSA
+---
 
-## Environmental Impact
+## 🔍 Bias & Limitations
 
-### CO2 Emissions
+* **Domain Bias:** Trained primarily on general English; may underperform on technical/coding/math instructions.
+* **Language:** English-only.
+* **Scaling Caution:** Denormalization preserves ordering but absolute values depend on chosen range.
 
-Experiments were conducted using Google Cloud Platform in region asia-south1, which has a carbon efficiency of 0.92 kgCO2eq/kWh. A cumulative of 13.24 hours of computation was performed on hardware of type L4 (TDP of 72W).
+---
 
-Total emissions are estimated to be 0.87 kgCO2eq of which 100% was directly offset by the cloud provider.
+## 📚 Citation
 
-Carbon emissions were estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+If you use MICE in your research, please cite:
 
-- **Hardware Type:** 1xL4 GPU
-- **Hours used:** 16 hours
-- **Cloud Provider:** Google Cloud Platform
-- **Compute Region:** South Asia
-- **Carbon Emitted:** 0.87 kgCO2eq (fully offset by provider)
-## Author
+> Chaubey, S. (2024). Maxwell Instruction Complexity Estimator (MICE). https://huggingface.co/thethinkmachine/MICE
 
-Shreyan C [@thethinkmachine](https://huggingface.co/thethinkmachine)/ [@thinkingmachines](https://huggingface.co/thinkingmachines)
+---
 
-For any queries, suggestions or feedback, please contact Shreyan C at *shreyan(at)bud(dot)studio* / *shreyan(dot)chaubey(at)gmail(dot)com*.
+## 🙋‍♂️ Author & Contact
 
-## References
+**Shreyan C** ([thethinkmachine](https://huggingface.co/thethinkmachine))
+Email: [shreyan.chaubey@gmail.com](mailto:shreyan.chaubey@gmail.com)
 
-- [[2312.15685] What Makes Good Data for Alignment? A Comprehensive Study of Automatic Data Selection in Instruction Tuning](https://arxiv.org/abs/2312.15685)
-- [[2404.02948] PiSSA: Principal Singular Values and Singular Vectors Adaptation of Large Language Models](https://arxiv.org/abs/2404.02948)
-- [DEITA-Complexity](https://huggingface.co/datasets/BhabhaAI/DEITA-Complexity)
-- [ModernBERT-Large](https://huggingface.co/answerdotai/ModernBERT-large)
+*This project is licensed under the Apache 2.0 License.*