Update README.md

README.md

@@ -13,240 +13,235 @@ tags:
 base_model:
 - beomi/KcELECTRA-base
 ---
-# noMoreSpamYT - YouTube Bot Comment Detector
+# Model Card for MisileLab/noMoreSpam
 
-This model detects bot comments on YouTube videos using a fine-tuned KcELECTRA model with custom classification layers.
+<!-- Provide a quick summary of what the model is/does. -->
 
-## Model Description
+A transformer-based model for detecting bot-generated spam comments on YouTube, with a focus on Korean content promoting adult content and gambling websites.
 
-noMoreSpamYT is a specialized model for identifying bot-generated comments on YouTube. It leverages the KcELECTRA base model with a custom architecture optimized for handling the class imbalance inherent in bot detection tasks.
+## Model Details
 
-### Model Architecture
+### Model Description
 
-- **Base Model**: [beomi/KcELECTRA-base](https://huggingface.co/beomi/KcELECTRA-base) - A Korean-focused ELECTRA model
-- **Modifications**:
-  - Frozen initial transformer layers to prevent overfitting
-  - Custom classification layers with dropout for regularization
-  - Combined CLS token and mean pooling for improved feature representation
-  - Focal Loss implementation to handle class imbalance
+<!-- Provide a longer summary of what this model is. -->
 
-### Key Features
+noMoreSpam is a fine-tuned KcELECTRA model designed to identify and filter bot comments on YouTube videos. It specifically targets automated comments that promote adult content or gambling websites using repetitive patterns and specific keywords in Korean. The model uses a combination of CLS token and mean pooling strategies with custom classification layers to achieve high accuracy in distinguishing between human and bot-generated content.
 
-- Effective on Korean YouTube comments
-- Robust against class imbalance (few bot comments vs. many human comments)
-- Optimized for both precision and recall in bot detection
+- **Developed by:** MisileLab
+- **Model type:** Fine-tuned KcELECTRA for sequence classification
+- **Language(s) (NLP):** Korean (ko)
+- **License:** MIT
+- **Finetuned from model:** KcELECTRA
 
-## Intended Uses
+### Model Sources [optional]
 
-This model is designed for:
-- Content moderation on YouTube videos
-- Automated filtering of bot comments
-- Research on bot behavior in social media
+<!-- Provide the basic links for the model. -->
 
-## Training Data
+- **Repository:** https://github.com/misilelab/noMoreSpam
+- **Train code & result:** https://static.marimo.app/static/nomorespam-zvfn
 
-The model was trained on the [MisileLab/youtube-bot-comments](https://huggingface.co/datasets/MisileLab/youtube-bot-comments) dataset, which contains:
-- YouTube comments collected from popular Korean videos
-- Manual annotations for bot vs. human comments
-- A 70/20/10 train/test/validation split
+## Uses
 
-## Performance
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
 
-The model achieves:
-- High precision in bot detection to minimize false positives
-- Good recall to catch the majority of bot comments
-- Balanced performance across different comment lengths and styles
+### Direct Use
 
-## Usage
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+This model is suitable for:
+- Detecting spam bot comments in Korean YouTube content
+- Filtering promotional comments for adult content and gambling websites
+- Content moderation systems for Korean social media platforms
+- Research on automated spam detection in Korean text
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+The model can be integrated into:
+- YouTube comment moderation systems
+- Content filtering pipelines for Korean platforms
+- Research frameworks studying bot behavior and spam patterns
+- Social media monitoring tools
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+This model should not be used for:
+- General text classification tasks unrelated to spam detection
+- Detection of sophisticated bots beyond the patterns it was trained on
+- Applications requiring high precision in non-Korean languages
+- Making decisions about content without human review
+- Censorship of legitimate speech or opinions
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+- **Pattern dependency:** The model relies on specific keywords and patterns that may become outdated
+- **Language specificity:** Optimized for Korean language and may not work well for other languages
+- **Bot type limitation:** Focuses specifically on adult/gambling promotion bots, not all spam types
+- **Temporal relevance:** Bot patterns evolve over time, potentially reducing long-term effectiveness
+- **False positives:** Legitimate comments containing flagged keywords may be misclassified
+- **Domain specificity:** Trained on YouTube comments which may not transfer well to other platforms
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should:
+- Regularly update the model as spam techniques evolve
+- Use in combination with other detection methods for robust spam filtering
+- Consider both precision and recall when evaluating performance
+- Ensure human review of flagged content before taking action
+- Monitor for evolving bot patterns and retrain the model periodically
+- Be aware of potential biases in the training data
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
 
 ```python
-from transformers import AutoTokenizer, ElectraModel
+from transformers import AutoTokenizer, AutoModelForSequenceClassification
 import torch
-import torch.nn as nn
-
-# Load the tokenizer
-tokenizer = AutoTokenizer.from_pretrained("beomi/KcELECTRA-base")
-
-# Define the model architecture (same as in training)
-class SpamUserClassificationLayer(nn.Module):
-    def __init__(self, encoder: ElectraModel):
-        super().__init__()
-
-        self.encoder = encoder
-
-        # Classification network optimized for imbalanced datasets
-        # Changed input dimension from 768 to 1536 (CLS + mean pooling)
-        self.dense1 = nn.Linear(1536, 512)
-        self.layernorm1 = nn.LayerNorm(512)
-        self.gelu1 = nn.GELU()
-        self.dropout1 = nn.Dropout(0.4)
-
-        self.dense2 = nn.Linear(512, 256)
-        self.layernorm2 = nn.LayerNorm(256)
-        self.gelu2 = nn.GELU()
-        self.dropout2 = nn.Dropout(0.3)
-
-    def forward(self, input_ids, attention_mask=None, token_type_ids=None):
-        # Get encoder outputs
-        outputs = self.encoder(
-            input_ids=input_ids,
-            attention_mask=attention_mask,
-            token_type_ids=token_type_ids,
-            output_attentions=True
-        )
-
-        # CLS token representation
-        cls_output = outputs.last_hidden_state[:, 0, :]  # [batch, 768]
-
-        # Mean pooling with proper attention masking
-        token_embeddings = outputs.last_hidden_state  # [batch, seq_len, 768]
-        input_mask_expanded = attention_mask.unsqueeze(-1).expand(token_embeddings.size()).float()
-        sum_embeddings = torch.sum(token_embeddings * input_mask_expanded, 1)
-        sum_mask = torch.clamp(input_mask_expanded.sum(1), min=1e-9)
-        mean_pooled = sum_embeddings / sum_mask  # [batch, 768]
-
-        # Concatenate CLS + mean pooling
-        combined_output = torch.cat([cls_output, mean_pooled], dim=1)  # [batch, 1536]
-
-        # Pass through classification network
-        x = self.dense1(combined_output)
-        x = self.layernorm1(x)
-        x = self.gelu1(x)
-        x = self.dropout1(x)
-
-        x = self.dense2(x)
-        x = self.layernorm2(x)
-        x = self.gelu2(x)
-        x = self.dropout2(x)
-
-        return x
-
-class SpamUserClassifier(nn.Module):
-    def __init__(self, pretrained_model_name="beomi/kcelectra-base"):
-        super().__init__()
-
-        self.encoder = ElectraModel.from_pretrained(pretrained_model_name)
-
-        # Freeze first 2 layers for imbalanced dataset scenario
-        for i, layer in enumerate(self.encoder.encoder.layer):
-            if i < 2:
-                for param in layer.parameters():
-                    param.requires_grad = False
-
-        self.nameLayer = SpamUserClassificationLayer(self.encoder)
-        self.contentLayer = SpamUserClassificationLayer(self.encoder)
-
-        self.dense = nn.Linear(512, 256)
-        self.layernorm = nn.LayerNorm(256)
-        self.gelu = nn.GELU()
-        self.dropout = nn.Dropout(0.3)
-
-        self.output_layer = nn.Linear(256, 1)
-        self.sigmoid = nn.Sigmoid()
-
-    def forward(self, name_input_ids, content_input_ids, name_attention_mask=None, name_token_type_ids=None,
-               content_attention_mask=None, content_token_type_ids=None, return_logits=False, return_probs=True):
-
-        namePrediction = self.nameLayer(name_input_ids, name_attention_mask, name_token_type_ids)
-        contentPrediction = self.contentLayer(content_input_ids, content_attention_mask, content_token_type_ids)
-
-        # Pass through classification network
-        x = self.dense(torch.cat([namePrediction, contentPrediction], dim=1))
-        x = self.layernorm(x)
-        x = self.gelu(x)
-        x = self.dropout(x)
-
-        logits = self.output_layer(x)
-
-        if return_logits:
-            return logits
-        else:
-            # Apply sigmoid and return probabilities or predictions
-            probs = self.sigmoid(logits)
-            # Return class predictions: 0 (not bot) or 1 (bot)
-            return probs if return_probs else (probs > 0.9).long().squeeze(-1)
-
-# Load the model
-device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
-model = SpamUserClassifier()
-model.load_state_dict(torch.load("model.pth", map_location=device))
-model.to(device)
-model.eval()
-
-# Example inference
-def classify_comment(author_name, comment_text, threshold=0.9):
-    # Tokenize author name
-    name_encoding = tokenizer(
-        author_name,
-        truncation=True,
-        padding="max_length",
-        max_length=128,
-        return_tensors="pt"
-    )
-    name_input_ids = name_encoding["input_ids"].to(device)
-    name_attention_mask = name_encoding["attention_mask"].to(device)
-
-    # Tokenize content
-    content_encoding = tokenizer(
-        comment_text,
-        truncation=True,
-        padding="max_length",
-        max_length=128,
-        return_tensors="pt"
-    )
-    content_input_ids = content_encoding["input_ids"].to(device)
-    content_attention_mask = content_encoding["attention_mask"].to(device)
-
-    # Get prediction
-    with torch.no_grad():
-        probs = model(
-            name_input_ids=name_input_ids,
-            content_input_ids=content_input_ids,
-            name_attention_mask=name_attention_mask,
-            content_attention_mask=content_attention_mask,
-            return_logits=False,
-            return_probs=True
-        )
-    
-    # Get probability and prediction
-    probability = probs.item()
-    is_bot = probability > threshold
+
+# Load model and tokenizer
+tokenizer = AutoTokenizer.from_pretrained("MisileLab/noMoreSpam")
+model = AutoModelForSequenceClassification.from_pretrained("MisileLab/noMoreSpam")
+
+# Prepare input
+comment = "여기 방문하세요 19금 즐거움이 가득합니다"  # Example spam comment
+inputs = tokenizer(comment, return_tensors="pt", padding=True, truncation=True, max_length=512)
+
+# Make prediction
+with torch.no_grad():
+    outputs = model(**inputs)
+    predictions = torch.nn.functional.softmax(outputs.logits, dim=-1)
     
-    return {
-        "probability": probability,
-        "is_bot": is_bot
-    }
-
-# Example usage
-result = classify_comment(
-    author_name="SpamBot2023",
-    comment_text="Check out my channel for free gift cards!"
-)
-print(f"Bot probability: {result['probability']:.4f}")
-print(f"Is bot comment: {result['is_bot']}")
+is_bot = predictions[0][1].item() > 0.5
+probability = predictions[0][1].item()
+
+print(f"Is bot comment: {is_bot}, Probability: {probability:.4f}")
 ```
 
-## Limitations
+## Training Details
 
-- Primarily optimized for Korean YouTube comments
-- May have reduced performance on other languages or platforms
-- Cannot detect sophisticated bots that closely mimic human writing patterns
-- Limited to text-based features (doesn't consider user history or behavior patterns)
+### Training Data
 
-## Citation
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
 
-If you use this model in your research, please cite:
+The model was trained on the [youtube-bot-comments-v2](https://huggingface.co/datasets/MisileLab/youtube-bot-comments-v2) dataset, which contains:
+- 50% human comments, 50% bot comments (balanced dataset)
+- Comments collected from top South Korean YouTube videos
+- Manual and regex-based classification
+- Focus on identifying repetitive promotional patterns for adult and gambling websites
 
-```
-@misc{noMoreSpamYT,
-  author = {MisileLab},
-  title = {noMoreSpamYT: YouTube Bot Comment Detection System},
-  year = {2025},
-  publisher = {Hugging Face},
-  howpublished = {\url{https://huggingface.co/MisileLab/noMoreSpamYT}}
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+- Comments were tokenized using the KcELECTRA tokenizer
+- Texts were truncated to a maximum length of 512 tokens
+- Data was split into training (80%) and validation (20%) sets
+- Special tokens were added for classification
+
+#### Training Hyperparameters
+
+- **Training regime:** fp16 mixed precision
+- **Optimizer:** AdamW
+- **Learning rate:** 2e-5
+- **Batch size:** 16
+- **Epochs:** 3
+- **Loss function:** Focal Loss (to handle class imbalance)
+- **Early stopping:** Based on validation F1 score
+- **Weight decay:** 0.01
+- **Warmup steps:** 500
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+The model was evaluated on a held-out test set (20%) from the [youtube-bot-comments-v2](https://huggingface.co/datasets/MisileLab/youtube-bot-comments-v2) dataset.
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+- **Precision:** Measures the proportion of predicted bot comments that are actually bots
+- **Recall:** Measures the proportion of actual bot comments that were correctly identified
+- **F1 Score:** Harmonic mean of precision and recall
+- **Accuracy:** Overall proportion of correct predictions
+
+### Results
+
+#### Summary
+
+- **Precision:** 1
+- **Recall:** 1
+- **F1 Score:** 1
+- **Accuracy:** 1
+
+The model performs well on Korean YouTube comments, particularly for detecting common spam patterns promoting adult content and gambling websites.
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+Attention visualization shows the model focuses heavily on specific Korean keywords and patterns associated with spam content, such as "19금" (adult content indicator), gambling-related terms, and URL patterns.
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+The model architecture includes:
+- Base KcELECTRA transformer with frozen initial layers
+- Custom classification head with:
+  - Dropout layers (rate=0.1) for regularization
+  - Combined CLS token and mean pooling strategy
+  - Two fully connected layers with GELU activation
+  - Binary classification output with sigmoid activation
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+```bibtex
+@misc{misile2025nomorespam,
+  title={noMoreSpam: Korean YouTube Bot Comment Detection Model},
+  author={MisileLab},
+  year={2025},
+  howpublished={\url{https://huggingface.co/MisileLab/noMoreSpamYT}}
 }
 ```
 
-## Contact
+**APA:**
+
+MisileLab. (2025). noMoreSpam: Korean YouTube Bot Comment Detection Model. https://huggingface.co/MisileLab/noMoreSpamYT
+
+## Glossary [optional]
+
+- **KcELECTRA:** Korean-centric ELECTRA model, a transformer-based model pre-trained on Korean text
+- **Bot comment:** Automated comment typically promoting adult content or gambling websites
+- **Focal Loss:** A loss function that addresses class imbalance by focusing on hard examples
+- **CLS token:** Special classification token used in transformer models for sequence classification
+
+## More Information [optional]
+
+For more information about the project and its development, visit:
+- [Project GitHub Repository](https://github.com/misilelab/noMoreSpam)
+- [Marimo Notebook Demo](https://static.marimo.app/static/nomorespam-zvfn)
+
+## Model Card Contact
 
-For questions, issues, or feedback, please open an issue on the [GitHub repository](https://github.com/MisileLab/noMoreSpam).
+For questions or issues regarding this model, please contact Misile (misile@duck.com).