Update README.md

README.md

@@ -1,43 +1,36 @@
----
-language: en
-license: apache-2.0
-tags:
-- distilbert
-- fine-tuned
-- url-classification
-- text-classification
-- nlp
-- web-mining
-- seo
-datasets:
-- ruggsea/infini-news-corpus
-pipeline_tag: text-classification
----
 
-# 🌐 URL Content vs Section Classifier (Fine-tuned DistilBERT)
+---
 
-This model is a **fine-tuned version of DistilBERT** designed to classify web URLs into two categories:
+## 📊 Training Data
 
-- **content** → A specific page such as an article, blog post, or news story  
-- **section** → A category page, listing page, or homepage/navigation page  
+The model was trained on a large-scale news URL dataset:
 
-It is optimized for **web crawling, URL filtering, and content extraction pipelines**.
+- Source: Infini News Corpus
+- Dataset: ruggsea/infini-news-corpus
+- URLs extracted from real-world news websites
 
 ---
 
-# 🧠 Model Description
-
-This model is fine-tuned from:
+## 🏷️ Labeling Strategy
 
-👉 **:contentReference[oaicite:0]{index=0}**
+Since manual labeling is expensive, weak supervision rules were used:
 
-DistilBERT is a compressed version of BERT that retains strong language understanding capabilities while being lightweight and fast.
+### Content pages:
+- Deep URL paths
+- Article-like slugs
+- Presence of IDs or long titles
+- News/story patterns
 
-In this project, it is adapted to learn **structural patterns in URLs** rather than natural language sentences.
+### Section pages:
+- Short paths
+- Category URLs
+- Homepage or listing pages
+- Trailing slash URLs
 
 ---
 
-# 🚀 Task Definition
+## ⚙️ Usage
 
-## Input
-A raw URL string:
+### Install dependencies
+```bash
+pip install transformers torch