Update README.md

README.md

@@ -1,80 +1,64 @@
 ---
-language:
-- en
-- id
-license: mit
+library_name: pytorch
 tags:
 - multimodal
-- audio-classification
-- text-classification
+- music-classification
 - math-rock
-- midwest-emo
-- emotion-recognition
-- personality-profiling
 - pytorch
+datasets:
+- anggars/neural-mathrock
+language:
+- en
 metrics:
-- f1
-- precision
-- recall
 - accuracy
 ---
 
 # Neural Mathrock: Multimodal Emotion and Personality Analysis
 
-This repository hosts a multimodal deep learning framework specialized in the affective and psychological analysis of **Math Rock** and **Midwest Emo** music. By integrating lyrical semantics and acoustic patterns, the system provides a comprehensive profile of a track's emotional and personality-based characteristics.
+This repository hosts a multimodal deep learning framework specialized in the affective and psychological analysis of Math Rock and Midwest Emo music. By integrating lyrical semantics and acoustic patterns, the system extracts features to classify emotional and personality-based characteristics.
 
 ## Project Objectives
-
 The research is structured to prioritize emotional resonance and genre-specific complexities:
-
-1.  **Emotion Recognition:** Identifying affective states (e.g., Sadness, Tension, Joy) through the synergy of vocal delivery and lyrical themes.
-2.  **Personality (MBTI) Profiling:** Correlating complex musical arrangements and introspective lyrics with personality archetypes (e.g., INFP, INTJ, INTP).
-3.  **Acoustic Feature Extraction:** Analyzing technical attributes of Math Rock, including non-standard time signatures, syncopation, and clean guitar timbres.
+- **Emotion & Vibe Recognition:** Identifying affective states and general vibes (e.g., Melancholic, Aggressive) through the synergy of lyrical themes and audio arrays.
+- **Personality (MBTI) Profiling:** Correlating complex musical arrangements and introspective lyrics with personality archetypes.
+- **Acoustic Feature Extraction:** Analyzing technical attributes like syncopation and odd time signatures using robust audio signal processing.
 
 ## Technical Architecture: Late Fusion Multimodal
-
-The system utilizes a **Late Fusion** approach to process distinct data modalities:
+The system utilizes a custom Multimodal PyTorch architecture:
 
 ### 1. Lyrical Stream (NLP)
 - **Encoder:** `xlm-roberta-base`
-- **Logic:** Extracts high-level semantic embeddings from song lyrics. The encoder is frozen to maintain stable pre-trained representations given the specialized nature of the dataset.
+- **Logic:** Extracts high-level semantic embeddings from song lyrics (768-dim). The base model weights are frozen to maintain stable pre-trained representations.
 
 ### 2. Acoustic Stream (DSP)
-- **Model:** 1D-Convolutional Neural Network (CNN)
-- **Input:** 20-channel Mel-frequency cepstral coefficients (MFCC).
-- **Logic:** Captures the "twinkly" guitar textures and erratic drum patterns common in Midwest Emo and Math Rock.
+- **Model:** 2D-Convolutional Neural Network (CNN) followed by a Transformer Encoder.
+- **Input:** 3-channel stacked features (Mel-spectrogram, 13-coefficient MFCC, and Chroma).
+- **Logic:** Captures the complex guitar textures and erratic drum patterns common in the genre, pooling them into a 256-dim feature vector.
 
 ### 3. Fusion Layer
-- **Method:** Feature concatenation (768-dim Text + 256-dim Audio).
-- **Heads:** Multi-task fully connected layers for joint Emotion and Personality classification.
-
-## Performance Summary (Weighted Evaluation)
+- **Method:** Feature concatenation (768-dim Text + 256-dim Audio) into a 1024-dim representation, processed through a 512-dim feed-forward layer with dropout regularization.
+- **Heads:** Multi-task fully connected layers for joint classification of MBTI, Emotion, Vibe, Intensity, and Tempo.
 
-The model was optimized using **Weighted Cross-Entropy Loss** to mitigate significant class imbalances.
+## Performance Summary and Academic Context
+This project is an undergraduate thesis developed at Sekolah Tinggi Teknologi Cipasung (STTC), Informatics Department, Class of 2022. It explores the extreme class imbalances naturally found in niche music genres.
 
-| Metric | Score |
-| :--- | :--- |
-| **Accuracy** | 0.81 |
-| **Weighted Avg F1** | 0.76 |
-| **Macro Avg F1** | 0.45 |
-
-### Class Highlights:
-- **INTJ:** 1.00 F1-score (Highly distinctive acoustic-lyrical signatures).
-- **INFP:** 0.89 F1-score (Robust detection of the genre's majority archetype).
-- **ISTP:** 0.53 F1-score (Successful identification of niche instrumental patterns).
-
-## Dataset
-
-Trained on the [anggars/neural-mathrock](https://huggingface.co/datasets/anggars/neural-mathrock) dataset, containing specialized annotations for emotion, MBTI, and musical features.
-
-## Academic Context
-
-This project is an undergraduate thesis developed at **Sekolah Tinggi Teknologi Cipasung (STTC)**, Informatics Department, Class of 2022. It explores the intersection of Music Information Retrieval (MIR) and psychological profiling.
+**Key Findings:**
+- **High-Performance Metrics:** The architecture successfully identifies broad acoustic targets, achieving an **F1-Score of 0.77 for High Intensity** and **0.72 for Melancholic Vibe**, proving the viability of the multimodal feature extraction pipeline.
+- **Class Imbalance in Niche Genres:** For 16-class targets like MBTI and Emotion, the model highlights the natural bias of the dataset. Classes with sufficient samples (e.g., ESTJ, Desire, Pride) perform adequately, while minority classes struggle due to lack of representation (Macro F1 ~0.07). This serves as a realistic baseline for future research regarding data sparsity in highly subjective Music Information Retrieval (MIR) tasks.
 
 ## How to Use
+Since this is a custom PyTorch architecture, it cannot be loaded via the standard `pipeline` API. You must define the model classes locally and load the state dictionary.
 
 ```python
 import torch
-# Example: 
-# model.load_state_dict(torch.load("pytorch_model.bin"))
-# model.eval()
+
+# 1. Define the MultimodalMathRock architecture classes here first
+# (Refer to the training script for the AudioCNNTransformer, TextTransformer, and Fusion module)
+
+# 2. Load the weights
+model = MultimodalMathRock()
+state_dict = torch.load("model.pt", map_location="cpu")
+model.load_state_dict(state_dict['model_state'])
+model.eval()
+
+# Model is ready for inference