Model Interpretability
Explore how the model makes decisions through attention visualizations and SHAP analysis
Interpretability Analysis
Attention Visualization
Analyze a text to see how the model's attention mechanism focuses on different words and phrases.
The visualization will show:
- Attention patterns across all layers
- Heatmap of token relationships
- Interactive layer and head exploration
SHAP Explanation
SHAP (SHapley Additive exPlanations) provides detailed feature importance analysis.
Understanding SHAP values:
- Shows positive and negative contributions
- Highlights impactful words in red/blue
- Based on game theory principles
Token Importance
See which words contribute most to the model's decision.
This visualization shows:
- Relative importance of each token
- Attention weight distribution
- Key words influencing the prediction