Update README.md

README.md

@@ -1,206 +1,165 @@
 ---
-base_model: bert-base-uncased
-library_name: peft
+language: en
+license: apache-2.0
 tags:
-- base_model:adapter:bert-base-uncased
+- text-classification
+- bert
 - lora
-- transformers
+- peft
+- 20-newsgroups
+datasets:
+- SetFit/20_newsgroups
+base_model: bert-base-uncased
+metrics:
+- accuracy
+model-index:
+- name: bert-lora-20newsgroups
+  results:
+  - task:
+      type: text-classification
+      name: Text Classification
+    dataset:
+      name: 20 Newsgroups
+      type: SetFit/20_newsgroups
+    metrics:
+    - type: accuracy
+      value: 0.82
+      name: Accuracy
 ---
 
-# Model Card for Model ID
-
-<!-- Provide a quick summary of what the model is/does. -->
-
-
-
-## Model Details
-
-### Model Description
-
-<!-- Provide a longer summary of what this model is. -->
-
-
+# BERT-LoRA for 20 Newsgroups Classification
 
-- **Developed by:** [More Information Needed]
-- **Funded by [optional]:** [More Information Needed]
-- **Shared by [optional]:** [More Information Needed]
-- **Model type:** [More Information Needed]
-- **Language(s) (NLP):** [More Information Needed]
-- **License:** [More Information Needed]
-- **Finetuned from model [optional]:** [More Information Needed]
+## Model Description
 
-### Model Sources [optional]
+This model is a **BERT-base-uncased** fine-tuned with **LoRA (Low-Rank Adaptation)** for multi-class text classification on the 20 Newsgroups dataset.
 
-<!-- Provide the basic links for the model. -->
+- **Base Model:** bert-base-uncased
+- **Method:** LoRA (Parameter-Efficient Fine-Tuning)
+- **Task:** Multi-class text classification (20 categories)
+- **Dataset:** 20 Newsgroups (~11K training, ~7K test samples)
+- **Trainable Parameters:** ~300K (0.3% of total)
+- **Adapter Size:** ~2 MB
 
-- **Repository:** [More Information Needed]
-- **Paper [optional]:** [More Information Needed]
-- **Demo [optional]:** [More Information Needed]
+## Categories
 
-## Uses
+The model classifies text into 20 newsgroup topics:
+- `alt.atheism`, `comp.graphics`, `comp.os.ms-windows.misc`, `comp.sys.ibm.pc.hardware`
+- `comp.sys.mac.hardware`, `comp.windows.x`, `misc.forsale`, `rec.autos`
+- `rec.motorcycles`, `rec.sport.baseball`, `rec.sport.hockey`, `sci.crypt`
+- `sci.electronics`, `sci.med`, `sci.space`, `soc.religion.christian`
+- `talk.politics.guns`, `talk.politics.mideast`, `talk.politics.misc`, `talk.religion.misc`
 
-<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+## Usage
 
-### Direct Use
+### Installation
 
-<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+```bash
+pip install transformers peft torch
+```
 
-[More Information Needed]
+### Load Model
 
-### Downstream Use [optional]
+```python
+from transformers import AutoTokenizer, AutoModelForSequenceClassification
+from peft import PeftModel
+import torch
 
-<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+# Load tokenizer
+tokenizer = AutoTokenizer.from_pretrained("bert-base-uncased")
 
-[More Information Needed]
+# Load base model
+base_model = AutoModelForSequenceClassification.from_pretrained(
+    "bert-base-uncased",
+    num_labels=20
+)
 
-### Out-of-Scope Use
+# Load LoRA adapters
+model = PeftModel.from_pretrained(base_model, "your-username/bert-lora-20newsgroups")
+model.eval()
+```
 
-<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+### Make Predictions
 
-[More Information Needed]
+```python
+text = "NASA announced a new mission to Mars with advanced rovers."
 
-## Bias, Risks, and Limitations
+inputs = tokenizer(text, return_tensors="pt", truncation=True, max_length=512)
 
-<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+with torch.no_grad():
+    outputs = model(**inputs)
+    prediction = outputs.logits.argmax(-1).item()
 
-[More Information Needed]
+categories = [
+    "alt.atheism", "comp.graphics", "comp.os.ms-windows.misc",
+    "comp.sys.ibm.pc.hardware", "comp.sys.mac.hardware", "comp.windows.x",
+    "misc.forsale", "rec.autos", "rec.motorcycles", "rec.sport.baseball",
+    "rec.sport.hockey", "sci.crypt", "sci.electronics", "sci.med",
+    "sci.space", "soc.religion.christian", "talk.politics.guns",
+    "talk.politics.mideast", "talk.politics.misc", "talk.religion.misc"
+]
 
-### Recommendations
-
-<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
-
-Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
-
-## How to Get Started with the Model
-
-Use the code below to get started with the model.
-
-[More Information Needed]
+print(f"Predicted category: {categories[prediction]}")
+# Output: sci.space
+```
 
 ## Training Details
 
-### Training Data
-
-<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
-
-[More Information Needed]
-
-### Training Procedure
-
-<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
-
-#### Preprocessing [optional]
-
-[More Information Needed]
-
-
-#### Training Hyperparameters
-
-- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
-
-#### Speeds, Sizes, Times [optional]
-
-<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
-
-[More Information Needed]
-
-## Evaluation
-
-<!-- This section describes the evaluation protocols and provides the results. -->
-
-### Testing Data, Factors & Metrics
-
-#### Testing Data
-
-<!-- This should link to a Dataset Card if possible. -->
-
-[More Information Needed]
-
-#### Factors
-
-<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
-
-[More Information Needed]
-
-#### Metrics
-
-<!-- These are the evaluation metrics being used, ideally with a description of why. -->
-
-[More Information Needed]
-
-### Results
-
-[More Information Needed]
-
-#### Summary
-
-
-
-## Model Examination [optional]
-
-<!-- Relevant interpretability work for the model goes here -->
-
-[More Information Needed]
-
-## Environmental Impact
-
-<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
-
-Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
-
-- **Hardware Type:** [More Information Needed]
-- **Hours used:** [More Information Needed]
-- **Cloud Provider:** [More Information Needed]
-- **Compute Region:** [More Information Needed]
-- **Carbon Emitted:** [More Information Needed]
-
-## Technical Specifications [optional]
-
-### Model Architecture and Objective
-
-[More Information Needed]
-
-### Compute Infrastructure
-
-[More Information Needed]
-
-#### Hardware
-
-[More Information Needed]
-
-#### Software
-
-[More Information Needed]
-
-## Citation [optional]
-
-<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
-
-**BibTeX:**
-
-[More Information Needed]
-
-**APA:**
-
-[More Information Needed]
-
-## Glossary [optional]
-
-<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
-
-[More Information Needed]
-
-## More Information [optional]
-
-[More Information Needed]
-
-## Model Card Authors [optional]
-
-[More Information Needed]
-
-## Model Card Contact
-
-[More Information Needed]
-### Framework versions
-
-- PEFT 0.18.0
+### LoRA Configuration
+- **Rank (r):** 8
+- **Alpha:** 16
+- **Dropout:** 0.1
+- **Target modules:** query, value (attention layers)
+
+### Training Hyperparameters
+- **Learning rate:** 2e-4
+- **Batch size:** 16
+- **Epochs:** 3
+- **Optimizer:** AdamW
+- **Max sequence length:** 512
+
+### Training Results
+- **Training time:** ~15 minutes on GPU
+- **Test accuracy:** ~82%
+- **Parameters trained:** 294,912 (0.27% of total)
+
+## Performance
+
+| Metric | Value |
+|--------|-------|
+| Accuracy | 82% |
+| Precision (macro) | 81% |
+| Recall (macro) | 81% |
+| F1-score (macro) | 81% |
+
+## Limitations
+
+- Trained only on newsgroup-style text from the 1990s
+- May not generalize well to modern social media or short texts
+- Performance varies across categories (some topics easier than others)
+- Max input length is 512 tokens (longer texts are truncated)
+
+## Why LoRA?
+
+LoRA provides:
+- **99% smaller model size** (2 MB vs 440 MB)
+- **100x fewer trainable parameters** (300K vs 110M)
+- **Faster training** (15 min vs 2+ hours)
+- **Same accuracy** as full fine-tuning (~82%)
+
+Perfect for deployment, experimentation, and resource-constrained environments.
+
+## Citation
+
+```bibtex
+@misc{bert-lora-20newsgroups,
+  author = {Your Name},
+  title = {BERT-LoRA for 20 Newsgroups Classification},
+  year = {2024},
+  publisher = {HuggingFace},
+  howpublished = {\url{https://huggingface.co/your-username/bert-lora-20newsgroups}}
+}
+```
+
+## License
+
+Apache 2.0 (following base BERT model license)