FutureMa commited on
Commit
d12bc5f
·
verified ·
1 Parent(s): 8bafb01

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +131 -3
README.md CHANGED
@@ -1,3 +1,131 @@
1
- ---
2
- license: apache-2.0
3
- ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Model Card for MatroidNN
2
+
3
+ ## Model Details
4
+
5
+ ### Model Description
6
+
7
+ **Model type:** Neural Network with Matroid-based Feature Selection (MatroidNN)
8
+
9
+ **Version:** 1.0
10
+
11
+ **Framework:** PyTorch
12
+
13
+ **Last updated:** February 27, 2025
14
+
15
+ ### Overview
16
+
17
+ MatroidNN is a neural network architecture that incorporates matroid theory for feature selection. It addresses the challenge of feature redundancy by selecting a maximally independent set of features based on matroid theory principles before training the neural network.
18
+
19
+ ### Model Architecture
20
+
21
+ - **Feature Selection Component**: MatroidFeatureSelector using correlation-based dependency analysis
22
+ - **Neural Network**: 3-layer feedforward network with batch normalization and dropout
23
+ - **Input**: Varies based on the number of features selected by the matroid selector
24
+ - **Hidden Layers**: Configurable hidden layer sizes (default 64 → 32)
25
+ - **Output**: Multi-class classification (configurable number of classes)
26
+ - **Parameters**: ~5K-10K parameters (varies based on input/output dimensions)
27
+
28
+ ## Uses
29
+
30
+ ### Direct Use
31
+
32
+ MatroidNN is designed for classification tasks where feature redundancy is a potential issue. It's particularly useful for:
33
+
34
+ - High-dimensional datasets with correlated features
35
+ - Feature selection in biological/medical data
36
+ - Financial prediction with multicollinear variables
37
+ - Any classification task where feature independence is desired
38
+
39
+ ### Out-of-Scope Use
40
+
41
+ This model is not intended for:
42
+ - Regression tasks (without modification)
43
+ - Time series prediction (without temporal adaptations)
44
+ - Raw image or text classification (without appropriate feature extraction)
45
+
46
+ ## Training Data
47
+
48
+ The model was developed and tested using synthetic data with deliberate feature dependencies. For real-world applications, the model should be retrained on domain-specific data.
49
+
50
+ ### Training Dataset
51
+
52
+ - **Type**: Synthetic data with controlled dependencies
53
+ - **Size**: 1000 samples (default), configurable
54
+ - **Features**: 20 initial features (default), configurable
55
+ - **Classes**: 3 classes (default), configurable
56
+ - **Distribution**: Equal class distribution in the synthetic data
57
+
58
+ ## Performance
59
+
60
+ ### Metrics
61
+
62
+ On synthetic test data with 3 classes:
63
+ - **Accuracy**: 94.0%
64
+ - **Macro-average F1-score**: 0.93
65
+ - **Per-class metrics**:
66
+ - Class 0: Precision 0.96, Recall 1.00, F1 0.98
67
+ - Class 1: Precision 0.86, Recall 0.86, F1 0.86
68
+ - Class 2: Precision 0.97, Recall 0.93, F1 0.95
69
+
70
+ ### Factors
71
+
72
+ Performance may vary based on:
73
+ - Feature correlation structure in the dataset
74
+ - Number of initial features and their information content
75
+ - Class distribution balance
76
+ - Rank threshold parameter in the MatroidFeatureSelector
77
+
78
+ ## Limitations
79
+
80
+ - The matroid-based feature selection uses correlation as a proxy for independence, which may not capture all forms of dependency
81
+ - The current implementation assumes numerical features and may require adaptation for categorical features
82
+ - Feature selection is performed once before training and does not adapt during training
83
+ - The rank threshold parameter requires careful tuning based on the dataset
84
+
85
+ ## Ethical Considerations
86
+
87
+ - Feature selection might unintentionally exclude features that are important for fairness considerations
88
+ - The model inherits any biases present in the training data
89
+ - Results should be interpreted with caution in high-stakes applications, with human oversight
90
+
91
+ ## Technical Specifications
92
+
93
+ ### Hardware Requirements
94
+
95
+ - Training: CUDA-capable GPU recommended for larger datasets
96
+ - Inference: CPU sufficient for most applications
97
+
98
+ ### Software Requirements
99
+
100
+ - Python 3.8+
101
+ - PyTorch 1.8+
102
+ - NumPy 1.20+
103
+ - scikit-learn 0.24+
104
+
105
+ ### Training Hyperparameters
106
+
107
+ - **Batch size**: 32 (default)
108
+ - **Learning rate**: 0.001 (default)
109
+ - **Optimizer**: Adam
110
+ - **Loss function**: Cross-Entropy Loss
111
+ - **Epochs**: Early stopping based on validation loss (patience=10)
112
+ - **Feature selection rank threshold**: 0.7 (default, configurable)
113
+
114
+ ## How to Use
115
+
116
+ ```python
117
+ from matroid_nn import MatroidFeatureSelector, MatroidNN
118
+
119
+ # Initialize feature selector
120
+ selector = MatroidFeatureSelector(rank_threshold=0.7)
121
+
122
+ # Apply feature selection
123
+ X_train_selected = selector.fit_transform(X_train)
124
+ X_test_selected = selector.transform(X_test)
125
+
126
+ # Create and train model
127
+ model = MatroidNN(
128
+ input_size=X_train_selected.shape[1],
129
+ hidden_size=64,
130
+ output_size=num_classes
131
+ )