Upload README.md with huggingface_hub

README.md

@@ -12,12 +12,46 @@ This project implements an advanced fashion search system based on CLIP, with th
 
 ### Architecture
 
-The main model combines:
-- **16 dimensions** : Color embeddings (first 16 dimensions)
-- **64 dimensions** : Hierarchy embeddings (dimensions 16-80)
-- **512 dimensions** : Standard CLIP embeddings (following dimensions)
+The main model's embedding structure:
+- **Dimensions 0-15** (16 dims): Color embeddings aligned with specialized color model
+- **Dimensions 16-79** (64 dims): Hierarchy embeddings aligned with specialized hierarchy model
+- **Dimensions 80-511** (432 dims): Standard CLIP embeddings for general visual-semantic understanding
 
-Total : **512 dimensions** for each embedding
+**Total: 512 dimensions** per embedding (text or image)
+
+**Key Innovation**: The first 80 dimensions are explicitly trained to align with specialized models through direct MSE and cosine similarity losses, ensuring guaranteed attribute positioning (GAP) while maintaining full CLIP capabilities in the remaining dimensions.
+
+### Loss Functions
+
+**1. Enhanced Contrastive Loss** (`enhanced_contrastive_loss`):
+
+Combines multiple objectives:
+- **Original Triple Loss**: Text-image-attributes contrastive learning
+- **Color Alignment**: Forces dims 0-15 to match color model embeddings
+- **Hierarchy Alignment**: Forces dims 16-79 to match hierarchy model embeddings
+- **Reference Loss**: Optional regularization to stay close to base CLIP
+
+**2. Alignment Components**:
+```python
+# Color alignment (text & image)
+color_text_mse = F.mse_loss(main_color_dims, color_model_emb)
+color_text_cosine = 1 - F.cosine_similarity(main_color_dims, color_model_emb).mean()
+
+# Hierarchy alignment (text & image)  
+hierarchy_text_mse = F.mse_loss(main_hierarchy_dims, hierarchy_model_emb)
+hierarchy_text_cosine = 1 - F.cosine_similarity(main_hierarchy_dims, hierarchy_model_emb).mean()
+
+# Combined alignment
+alignment_loss = (color_alignment + hierarchy_alignment) / 2
+```
+
+**3. Final Loss**:
+```python
+total_loss = (1 - α) * contrastive_loss + α * alignment_loss + β * reference_loss
+```
+Where:
+- α (alignment_weight) = 0.2 : Balances contrastive and alignment objectives
+- β (reference_weight) = 0.1 : Keeps text space close to base CLIP
 
 ## 🚀 Installation
 
@@ -46,43 +80,107 @@ pip install -r requirements.txt
 
 ```
 .
-├── color_model.py              # Color model definition
-├── hierarchy_model.py           # Hierarchy model definition
-├── main_model.py               # Main CLIP model and training functions
+├── color_model.py              # Color model architecture and training
+├── hierarchy_model.py          # Hierarchy model architecture and training
+├── main_model.py               # Main GAP-CLIP model with enhanced loss functions
+├── train_main_model.py         # Training script with optimized hyperparameters
 ├── config.py                   # Configuration for paths and parameters
-├── tokenizer_vocab.json        # Tokenizer vocabulary
-├── example_usage.py            # Usage examples
+├── example_usage.py            # Usage examples and HuggingFace loading
+├── tokenizer_vocab.json        # Tokenizer vocabulary for color model
 ├── models/
-│   ├── color_model.pt          # Trained color model
-│   ├── hierarchy_model.pth     # Trained hierarchy model
-│   └── gap_clip.pth            # Main CLIP model
-├── evaluation/                 # Evaluation scripts
-│   ├── evaluate_color_embeddings.py
-│   ├── main_model_evaluation.py
-│   └── ...
-├── data/                       # Training data
-│   ├── download_data.py
+│   ├── color_model.pt          # Trained color model checkpoint
+│   ├── hierarchy_model.pth     # Trained hierarchy model checkpoint
+│   └── gap_clip.pth            # Main GAP-CLIP model checkpoint
+├── evaluation/                 # Comprehensive evaluation scripts
+│   ├── main_model_evaluation.py         # Main evaluation with 3 datasets
+│   ├── evaluate_color_embeddings.py     # Color embedding analysis
+│   ├── hierarchy_evaluation.py          # Hierarchy classification tests
+│   ├── fashion_search.py                # Interactive search demos
+│   ├── 0_shot_classification.py         # Zero-shot classification
+│   ├── heatmap_color_similarities.py    # Color similarity visualization
+│   ├── tsne_images.py                   # t-SNE embedding visualization
+│   └── basic_test_generalized.py        # Basic functionality tests
+├── data/
+│   ├── data_with_local_paths.csv        # Training dataset with annotations
+│   ├── fashion-mnist_test.csv           # Fashion-MNIST evaluation data
+│   ├── download_data.py                 # Dataset download utilities
+│   └── get_csv_from_chunks.py           # Dataset preprocessing
+├── optuna/                     # Hyperparameter optimization
+│   ├── optuna_optimisation.py           # Optuna optimization script
+│   ├── optuna_study.pkl                 # Saved optimization study
+│   ├── optuna_results.txt               # Best hyperparameters
+│   ├── optuna_optimization_history.png  # Optimization visualization
+│   ├── optuna_param_importances.png     # Parameter importance plot
+│   └── optuna_guide.md                  # Optuna usage guide
+├── upload_hf/                  # HuggingFace Hub upload utilities
+│   ├── upload_to_huggingface.py         # Upload script
+│   └── GUIDE_UPLOAD_HF.md               # Upload guide
 ├── requirements.txt            # Python dependencies
 └── README.md                   # This documentation
 ```
 
+### Key Files Description
+
+**Core Model Files**:
+- `color_model.py`: ResNet18-based color embedding model (16 dims)
+- `hierarchy_model.py`: ResNet18-based hierarchy classification model (64 dims)  
+- `main_model.py`: GAP-CLIP implementation with enhanced contrastive loss
+- `train_main_model.py`: Training with Optuna-optimized hyperparameters
+
+**Configuration**:
+- `config.py`: Central configuration for all paths, dimensions, and device settings
+- `tokenizer_vocab.json`: Vocabulary for color model's text encoder
+
+**Evaluation Suite**:
+- `main_model_evaluation.py`: Comprehensive evaluation across Fashion-MNIST, KAGL, and local datasets
+- Other evaluation scripts provide specialized analysis (color, hierarchy, search, etc.)
+
+**Training Data**:
+- `data_with_local_paths.csv`: Main training dataset with text, color, hierarchy, and image paths
+- `fashion-mnist_test.csv`: Evaluation dataset for zero-shot generalization testing
+
 ## 🔧 Configuration
 
 Main parameters are defined in `config.py`:
 
 ```python
-color_emb_dim = 16          # Color embedding dimension
-hierarchy_emb_dim = 64      # Hierarchy embedding dimension
+# Embedding dimensions
+color_emb_dim = 16          # Color embedding dimension (dims 0-15)
+hierarchy_emb_dim = 64      # Hierarchy embedding dimension (dims 16-79)
+
+# Device configuration
 device = torch.device("mps")  # Device (cuda, mps, cpu)
+
+# Column names for dataset
+text_column = 'text'                    # Description column
+color_column = 'color'                  # Color label column
+hierarchy_column = 'hierarchy'          # Hierarchy category column
+column_local_image_path = 'local_image_path'  # Image path column
 ```
 
 ### Model Paths
 
-Default paths are:
-- `models/color_model.pt` : Color model
-- `models/hierarchy_model.pth` : Hierarchy model
-- `models/gap_clip.pth` : Main CLIP model
-- `tokenizer_vocab.json` : Tokenizer vocabulary
+Default paths configured in `config.py`:
+- `models/color_model.pt` : Trained color model checkpoint
+- `models/hierarchy_model.pth` : Trained hierarchy model checkpoint
+- `models/gap_clip.pth` : Main GAP-CLIP model checkpoint
+- `tokenizer_vocab.json` : Tokenizer vocabulary for color model
+- `data_with_local_paths.csv` : Training/validation dataset
+
+### Dataset Format
+
+The training dataset CSV should contain:
+- `text`: Text description of the fashion item
+- `color`: Color label (e.g., "red", "blue", "black")
+- `hierarchy`: Category label (e.g., "dress", "shirt", "shoes")
+- `local_image_path`: Path to the image file
+
+Example:
+```csv
+text,color,hierarchy,local_image_path
+"red summer dress with floral pattern",red,dress,data/images/001.jpg
+"blue denim jeans casual style",blue,jeans,data/images/002.jpg
+```
 
 ## 📦 Usage
 
@@ -142,13 +240,32 @@ with torch.no_grad():
 
 ### 4. Using the Example Script
 
+The `example_usage.py` provides ready-to-use examples for loading and using GAP-CLIP:
+
 ```bash
+# Load from HuggingFace and search with text
+python example_usage.py \
+    --repo-id Leacb4/gap-clip \
+    --text "red summer dress"
+
+# Search with image
+python example_usage.py \
+    --repo-id Leacb4/gap-clip \
+    --image path/to/image.jpg
+
+# Both text and image
 python example_usage.py \
-    --repo-id your-username/your-model \
-    --text "blue jacket" \
+    --repo-id Leacb4/gap-clip \
+    --text "blue denim jeans" \
     --image path/to/image.jpg
 ```
 
+This script demonstrates:
+- Loading models from HuggingFace Hub
+- Extracting text and image embeddings
+- Accessing color and hierarchy subspaces
+- Measuring alignment quality with specialized models
+
 ## 🎯 Model Training
 
 ### Train the Color Model
@@ -179,27 +296,77 @@ train_hierarchy_model(model, train_loader, val_loader, num_epochs=20)
 
 ### Train the Main CLIP Model
 
-The main model trains with both specialized models:
+The main model trains with both specialized models using an enhanced contrastive loss.
 
+**Option 1: Train with optimized hyperparameters (recommended)**:
 ```bash
-python main_model.py
+python train_main_model.py
 ```
+This uses hyperparameters optimized with Optuna (Trial 29, validation loss ~0.1129).
 
-**Training Parameters** (in `main_model.py`):
-- `num_epochs = 20` : Number of epochs
-- `learning_rate = 1e-5` : Learning rate
-- `temperature = 0.07` : Temperature for contrastive loss
-- `alignment_weight = 0.5` : Weight for embedding alignment
+**Option 2: Train with default parameters**:
+```bash
+python main_model.py
+```
+This runs the main training loop with manually configured parameters.
+
+**Default Training Parameters** (in `main_model.py`):
+- `num_epochs = 20` : Number of training epochs
+- `learning_rate = 1.5e-5` : Learning rate with AdamW optimizer
+- `temperature = 0.09` : Temperature for softer contrastive learning
+- `alignment_weight = 0.2` : Weight for color/hierarchy alignment loss
+- `weight_decay = 5e-4` : L2 regularization to prevent overfitting
 - `batch_size = 32` : Batch size
-- `use_enhanced_loss = True` : Use enhanced loss with alignment
+- `subset_size = 20000` : Dataset size for better generalization
+- `reference_weight = 0.1` : Weight for base CLIP regularization
+
+**Enhanced Loss Function**:
+
+The training uses `enhanced_contrastive_loss` which combines:
 
-**Features**:
-- Triple contrastive loss (text-image-attributes)
-- Direct alignment between specialized models and main model
-- Early stopping with patience
-- Automatic learning rate reduction
-- Automatic best model saving
+1. **Triple Contrastive Loss** (weighted):
+   - Text-Image alignment (70%)
+   - Text-Attributes alignment (15%)
+   - Image-Attributes alignment (15%)
 
+2. **Direct Alignment Loss** (combines color & hierarchy):
+   - MSE loss between main model color dims (0-15) and color model embeddings
+   - MSE loss between main model hierarchy dims (16-79) and hierarchy model embeddings
+   - Cosine similarity losses for both color and hierarchy
+   - Applied to both text and image embeddings
+
+3. **Reference Model Loss** (optional):
+   - Keeps text embeddings close to base CLIP
+   - Improves cross-domain generalization
+
+**Training Features**:
+- Enhanced data augmentation (rotation, color jitter, blur, affine transforms)
+- Gradient clipping (max_norm=1.0) to prevent exploding gradients
+- ReduceLROnPlateau scheduler (patience=3, factor=0.5)
+- Early stopping (patience=7)
+- Automatic best model saving with checkpoints
+- Detailed metrics logging (alignment losses, cosine similarities)
+- Overfitting detection and warnings
+- Training curves visualization with 3 plots (losses, overfitting gap, comparison)
+
+### Hyperparameter Optimization
+
+The project includes Optuna-based hyperparameter optimization in `optuna/`:
+
+```bash
+cd optuna
+python optuna_optimisation.py
+```
+
+This optimizes:
+- Learning rate
+- Temperature for contrastive loss
+- Alignment weight
+- Weight decay
+
+Results are saved in `optuna_study.pkl` and visualizations in `optuna_optimization_history.png` and `optuna_param_importances.png`.
+
+The best hyperparameters from Optuna optimization are used in `train_main_model.py`.
 
 ## 📊 Models
 
@@ -217,22 +384,32 @@ python main_model.py
 - **Hierarchy classes** : shirt, dress, pant, shoe, bag, etc.
 - **Usage** : Classify and encode categorical hierarchy
 
-### Main CLIP Model
+### Main CLIP Model (GAP-CLIP)
 
 - **Architecture** : CLIP ViT-B/32 (LAION)
-- **Base** : `laion/CLIP-ViT-B-32-laion2B-s34B-b79K`
-- **Trained with** : Triple contrastive loss including color and hierarchy embeddings
-- **Dimensions** : 592 (512 CLIP + 16 color + 64 hierarchy)
+- **Base Model** : `laion/CLIP-ViT-B-32-laion2B-s34B-b79K`
+- **Training Approach** : Enhanced contrastive loss with direct attribute alignment
+- **Embedding Dimensions** : 512 total
+  - Color subspace: dims 0-15 (16 dims)
+  - Hierarchy subspace: dims 16-79 (64 dims)
+  - General CLIP: dims 80-511 (432 dims)
+- **Training Dataset** : 20,000 fashion items with color and hierarchy annotations
+- **Validation Split** : 80/20 train-validation split
+- **Optimizer** : AdamW with weight decay (5e-4)
+- **Best Checkpoint** : Automatically saved based on validation loss
 - **Features** :
-  - Text-image search
-  - Specialized embeddings (color + hierarchy)
-  - Alignment with specialized models
+  - Multi-modal text-image search
+  - Guaranteed attribute positioning (GAP) in specific dimensions
+  - Direct alignment with specialized color and hierarchy models
+  - Maintains general CLIP capabilities for cross-domain tasks
+  - Reduced overfitting through augmentation and regularization
 
 ## 🔍 Advanced Usage Examples
 
 ### Search with Combined Embeddings
 
 ```python
+import torch
 import torch.nn.functional as F
 
 # Text query
@@ -243,82 +420,290 @@ text_inputs = {k: v.to(device) for k, v in text_inputs.items()}
 # Main model embeddings
 with torch.no_grad():
     outputs = main_model(**text_inputs)
-    text_features = outputs.text_embeds
+    text_features = outputs.text_embeds  # Shape: [1, 512]
 
 # Extract specialized embeddings from main model
-main_color_emb = text_features[:, :16]  # First 16 dimensions
-main_hierarchy_emb = text_features[:, 16:80]  # Dimensions 16-80
-main_clip_emb = text_features[:, 80:]  # CLIP dimensions (80+)
+main_color_emb = text_features[:, :16]  # Color dimensions (0-15)
+main_hierarchy_emb = text_features[:, 16:80]  # Hierarchy dimensions (16-79)
+main_clip_emb = text_features[:, 80:]  # General CLIP dimensions (80-511)
 
 # Compare with specialized models
 color_emb = color_model.get_text_embeddings([text_query])
 hierarchy_emb = hierarchy_model.get_text_embeddings([text_query])
 
-# Cosine similarity
+# Measure alignment quality
 color_similarity = F.cosine_similarity(color_emb, main_color_emb, dim=1)
 hierarchy_similarity = F.cosine_similarity(hierarchy_emb, main_hierarchy_emb, dim=1)
+
+print(f"Color alignment: {color_similarity.item():.4f}")
+print(f"Hierarchy alignment: {hierarchy_similarity.item():.4f}")
+
+# For search, you can use different strategies:
+# 1. Use full embeddings for general search
+# 2. Use color subspace for color-specific search
+# 3. Use hierarchy subspace for category search
+# 4. Weighted combination of subspaces
 ```
 
 ### Search in an Image Database
 
 ```python
 import numpy as np
+import torch
+import torch.nn.functional as F
+from tqdm import tqdm
 
-# Load all images from database
+# Step 1: Pre-compute image embeddings (do this once)
 image_paths = [...]  # List of image paths
 image_features_list = []
 
-for img_path in image_paths:
+print("Computing image embeddings...")
+for img_path in tqdm(image_paths):
     image = Image.open(img_path).convert("RGB")
     image_inputs = processor(images=[image], return_tensors="pt")
     image_inputs = {k: v.to(device) for k, v in image_inputs.items()}
     
     with torch.no_grad():
         outputs = main_model(**image_inputs)
-        features = outputs.image_embeds
-        image_features_list.append(features.cpu().numpy())
+        features = outputs.image_embeds  # Shape: [1, 512]
+        image_features_list.append(features.cpu())
 
-# Convert to numpy array
-image_features = np.vstack(image_features_list)
+# Stack all features
+image_features = torch.cat(image_features_list, dim=0)  # Shape: [N, 512]
 
-# Search
+# Step 2: Search with text query
 query = "red dress"
-# ... get text_features ...
+text_inputs = processor(text=[query], padding=True, return_tensors="pt")
+text_inputs = {k: v.to(device) for k, v in text_inputs.items()}
+
+with torch.no_grad():
+    outputs = main_model(**text_inputs)
+    text_features = outputs.text_embeds  # Shape: [1, 512]
 
-# Calculate similarities
-similarities = F.cosine_similarity(
-    text_features, 
-    torch.from_numpy(image_features).to(device), 
-    dim=1
-)
+# Step 3: Calculate similarities
+# Normalize embeddings for cosine similarity
+text_features_norm = F.normalize(text_features, dim=-1)
+image_features_norm = F.normalize(image_features.to(device), dim=-1)
 
-# Sort by similarity
+# Compute cosine similarities
+similarities = (text_features_norm @ image_features_norm.T).squeeze(0)  # Shape: [N]
+
+# Step 4: Get top-k results
 top_k = 10
-top_indices = similarities.argsort(descending=True)[:top_k]
+top_scores, top_indices = similarities.topk(top_k, largest=True)
+
+# Display results
+print(f"\nTop {top_k} results for query: '{query}'")
+for i, (idx, score) in enumerate(zip(top_indices, top_scores)):
+    print(f"{i+1}. {image_paths[idx]} (similarity: {score.item():.4f})")
+
+# Optional: Filter by color or hierarchy
+# Extract color embeddings from query
+query_color_emb = text_features[:, :16]
+# Extract hierarchy embeddings from query  
+query_hierarchy_emb = text_features[:, 16:80]
+# Use these for more targeted search
 ```
 
 ## 📝 Evaluation
 
-Evaluation scripts are available in `evaluation/`:
+### Comprehensive Model Evaluation
+
+The main evaluation script `evaluation/main_model_evaluation.py` provides extensive testing across multiple datasets:
+
+```bash
+python evaluation/main_model_evaluation.py
+```
+
+**Evaluation Datasets**:
+1. **Fashion-MNIST** (~10,000 samples) - Grayscale fashion items
+2. **KAGL Marqo** (HuggingFace dataset) - Real fashion images with metadata
+3. **Local Validation Dataset** - Custom validation set with local images
+
+**Evaluation Metrics**:
+
+For each dataset, the evaluation measures:
+
+1. **Color Embeddings Performance** (dimensions 0-15):
+   - Nearest Neighbor (NN) Accuracy: Classification accuracy using nearest neighbor
+   - Centroid Accuracy: Classification using cluster centroids
+   - Separation Score: How well color embeddings separate different classes
+
+2. **Hierarchy Embeddings Performance** (dimensions 16-79):
+   - Nearest Neighbor (NN) Accuracy: Classification accuracy for fashion categories
+   - Centroid Accuracy: Cluster-based classification
+   - Separation Score: Class separation quality
+
+3. **Full Embeddings Performance** (all 512 dimensions):
+   - Evaluates complete embedding space
+   - Compares subspace (color/hierarchy) vs full embedding effectiveness
+
+**Baseline Comparison**:
+
+The evaluation includes comparison against `patrickjohncyh/fashion-clip`:
+- Direct performance comparison on the same datasets
+- Improvement metrics calculation
+- Statistical significance analysis
+
+**Key Evaluation Functions**:
+- `evaluate_fashion_mnist()` : Test on Fashion-MNIST dataset
+- `evaluate_kaggle_marqo()` : Test on real fashion images
+- `evaluate_local_validation()` : Test on local validation set
+- `evaluate_baseline_fashion_mnist()` : Baseline model on Fashion-MNIST
+- `evaluate_baseline_kaggle_marqo()` : Baseline model on KAGL
+- `evaluate_full_embeddings()` : Test complete 512D space
+- `analyze_baseline_vs_trained_performance()` : Comparative analysis
+- `compare_subspace_vs_full_embeddings()` : Subspace effectiveness
+
+**Visualization Outputs** (saved in analysis directory):
+- Confusion matrices for color and hierarchy classification
+- t-SNE projections of embeddings
+- Similarity heatmaps
+- Performance comparison charts
+
+**Other Evaluation Scripts**:
+- `evaluate_color_embeddings.py` : Focused color embeddings evaluation
+- `fashion_search.py` : Interactive fashion search tests
+- `hierarchy_evaluation.py` : Hierarchy classification analysis
+- `0_shot_classification.py` : Zero-shot classification tests
+
+
+## 📊 Performance & Results
+
+The evaluation framework (`main_model_evaluation.py`) tests the model across three datasets with comparison to a baseline fashion CLIP model.
+
+### Evaluation Metrics
+
+**Color Classification** (dimensions 0-15):
+- Nearest Neighbor Accuracy
+- Centroid-based Accuracy  
+- Separation Score (class separability)
+
+**Hierarchy Classification** (dimensions 16-79):
+- Nearest Neighbor Accuracy
+- Centroid-based Accuracy
+- Separation Score
+
+**Full Embedding Quality** (all 512 dims):
+- Tests whether the full space maintains performance
+- Compares subspace vs full embedding effectiveness
+
+### Datasets Used for Evaluation
 
-- `evaluate_color_embeddings.py` : Color embeddings evaluation
-- `main_model_evaluation.py` : Main model evaluation
-- `fashion_search.py` : Fashion search tests
+1. **Fashion-MNIST**: 10,000 grayscale fashion item images
+   - 10 categories (T-shirt, Trouser, Pullover, Dress, Coat, Sandal, Shirt, Sneaker, Bag, Ankle boot)
+   - Mapped to model's hierarchy classes
 
+2. **KAGL Marqo Dataset**: Real-world fashion images from HuggingFace
+   - Diverse fashion items with rich metadata
+   - Color and category annotations
+   - Realistic product images
+
+3. **Local Validation Set**: Custom validation dataset
+   - Fashion items with local image paths
+   - Annotated with colors and hierarchies
+   - Domain-specific evaluation
+
+### Comparative Analysis
+
+The evaluation includes:
+- **Baseline comparison**: GAP-CLIP vs `patrickjohncyh/fashion-clip`
+- **Subspace analysis**: Dedicated dimensions (0-79) vs full space (0-511)
+- **Cross-dataset generalization**: Performance consistency across datasets
+- **Alignment quality**: How well specialized dimensions match expert models
+
+All visualizations (confusion matrices, t-SNE plots, heatmaps) are automatically saved in the analysis directory.
 
 ## 📄 Citation
 
-If you use these models, please cite:
+If you use GAP-CLIP in your research, please cite:
 
 ```bibtex
-@misc{fashion-search-model,
-  title={GAP (Guaranteed Attribute Position) CLIP: A Multi-Loss Framework for Attribute-Aware Fashion Embeddings},
-  author={ },
+@misc{gap-clip-2024,
+  title={GAP-CLIP: Guaranteed Attribute Positioning in CLIP Embeddings for Fashion Search},
+  author={Sarfati, Lea Attia},
   year={2024},
-  howpublished={\url{https://huggingface.co/Leacb4/gap-clip}}
+  note={A multi-loss framework combining contrastive learning with direct attribute alignment},
+  howpublished={\url{https://huggingface.co/Leacb4/gap-clip}},
+  abstract={GAP-CLIP introduces a novel training approach that guarantees specific embedding 
+            dimensions encode color (dims 0-15) and hierarchy (dims 16-79) information through 
+            direct alignment with specialized models, while maintaining full CLIP capabilities 
+            in the remaining dimensions (80-511).}
 }
 ```
 
+### Key Contributions
+
+- **Guaranteed Attribute Positioning**: Specific dimensions reliably encode color and hierarchy
+- **Multi-Loss Training**: Combines contrastive learning with MSE and cosine alignment losses
+- **Specialized Model Alignment**: Direct supervision from expert color and hierarchy models
+- **Preserved Generalization**: Maintains base CLIP capabilities for cross-domain tasks
+- **Comprehensive Evaluation**: Tested across multiple datasets with baseline comparisons
+
+## ❓ FAQ & Troubleshooting
+
+### Q: What are the minimum hardware requirements?
+
+**A**: 
+- **GPU**: Recommended for training (CUDA or MPS). CPU training is very slow.
+- **RAM**: Minimum 16GB, recommended 32GB for training
+- **Storage**: ~5GB for models and datasets
+
+### Q: Why are my embeddings not aligned?
+
+**A**: Check that:
+1. You're using the correct dimension ranges (0-15 for color, 16-79 for hierarchy)
+2. The model was trained with alignment_weight > 0
+3. Color and hierarchy models were properly loaded during training
+
+### Q: How do I use only the color or hierarchy subspace for search?
+
+**A**:
+```python
+# Extract and use only color embeddings
+text_color_emb = text_features[:, :16]
+image_color_emb = image_features[:, :16]
+color_similarity = F.cosine_similarity(text_color_emb, image_color_emb)
+
+# Extract and use only hierarchy embeddings
+text_hierarchy_emb = text_features[:, 16:80]
+image_hierarchy_emb = image_features[:, 16:80]
+hierarchy_similarity = F.cosine_similarity(text_hierarchy_emb, image_hierarchy_emb)
+```
+
+### Q: Can I add more attributes beyond color and hierarchy?
+
+**A**: Yes! The architecture is extensible:
+1. Train a new specialized model for your attribute
+2. Reserve additional dimensions in the embedding space
+3. Add alignment losses for these dimensions in `enhanced_contrastive_loss`
+4. Update `config.py` with new dimension ranges
+
+### Q: How do I evaluate on my own dataset?
+
+**A**: 
+1. Format your dataset as CSV with columns: `text`, `color`, `hierarchy`, `local_image_path`
+2. Update `config.local_dataset_path` in `config.py`
+3. Run the evaluation: `python evaluation/main_model_evaluation.py`
+
+### Q: Training loss is decreasing but validation loss is increasing. What should I do?
+
+**A**: This indicates overfitting. Try:
+- Increase `weight_decay` (e.g., from 5e-4 to 1e-3)
+- Reduce `alignment_weight` (e.g., from 0.2 to 0.1)
+- Increase dataset size (`subset_size`)
+- Add more data augmentation in `CustomDataset`
+- Enable or increase early stopping patience
+
+### Q: Can I fine-tune GAP-CLIP on a specific domain?
+
+**A**: Yes! Load the checkpoint and continue training:
+```python
+checkpoint = torch.load('models/gap_clip.pth')
+model.load_state_dict(checkpoint['model_state_dict'])
+# Continue training with your domain-specific data
+```
+
 ## 🤝 Contributing
 
 Contributions are welcome! Feel free to open an issue or a pull request.