charlesgchen/test / gradient_analysis_BarrierTrust_1.md
charlesgchen's picture
|
download
raw
3.48 kB
### Loss Values at Convergence
| Loss | Mean | Std |
|------|------|-----|
| nll | 0.007336 | 0.004056 |
| pairwise | 0.087583 | 0.034836 |
| trust | 0.000404 | 0.001692 |
| anchor | 0.006579 | 0.001478 |
| residual | 0.058852 | 0.014836 |
### Lambda-weighted Gradient Magnitudes
**Parameter group: `aleatoric`** (`unc_head.ale_head.weight`, `unc_head.ale_head.bias`)
| Loss | Mean \|\|grad\|\| | Std \|\|grad\|\| | CV |
|------|-------------------|------------------|------|
| nll | 0.000166 | 0.000114 | 69.0% |
| pairwise | 0.000827 | 0.000214 | 25.9% |
| trust | 0.000000 | 0.000000 | ∞ |
| anchor | 0.000180 | 0.000045 | 24.8% |
| residual | 0.000000 | 0.000000 | ∞ |
**Parameter group: `calibration`** (`unc_head.calibration_head.weight`, `unc_head.calibration_head.bias`)
| Loss | Mean \|\|grad\|\| | Std \|\|grad\|\| | CV |
|------|-------------------|------------------|------|
| nll | 0.000684 | 0.000391 | 57.1% |
| pairwise | 0.003024 | 0.000640 | 21.2% |
| trust | 0.000000 | 0.000000 | ∞ |
| anchor | 0.000692 | 0.000162 | 23.3% |
| residual | 0.000000 | 0.000000 | ∞ |
**Parameter group: `probe`** (`unc_head.alpha`, `unc_head.probe_conv.weight`, `unc_head.probe_to_class.weight`)
| Loss | Mean \|\|grad\|\| | Std \|\|grad\|\| | CV |
|------|-------------------|------------------|------|
| nll | 0.000228 | 0.000102 | 44.8% |
| pairwise | 0.004244 | 0.001395 | 32.9% |
| trust | 0.000052 | 0.000127 | 244.1% |
| anchor | 0.002462 | 0.000288 | 11.7% |
| residual | 0.006048 | 0.001423 | 23.5% |
**Parameter group: `ranking`** (`unc_head.err_a`, `unc_head.err_b`, `unc_head.err_c`)
| Loss | Mean \|\|grad\|\| | Std \|\|grad\|\| | CV |
|------|-------------------|------------------|------|
| nll | 0.000000 | 0.000000 | ∞ |
| pairwise | 0.000207 | 0.000117 | 56.6% |
| trust | 0.000000 | 0.000000 | ∞ |
| anchor | 0.000984 | 0.000111 | 11.3% |
| residual | 0.000000 | 0.000000 | ∞ |
### Gradient Cosine Similarities
**Parameter group: `aleatoric`** (`unc_head.ale_head.weight`, `unc_head.ale_head.bias`)
| Loss pair | Mean cos | Std cos | Negative % |
|-----------|----------|---------|------------|
| nll_vs_pairwise | +0.042 | 0.296 | 50% |
| nll_vs_anchor | -0.162 | 0.208 | 78% |
| pairwise_vs_anchor | +0.452 | 0.183 | 0% |
**Parameter group: `calibration`** (`unc_head.calibration_head.weight`, `unc_head.calibration_head.bias`)
| Loss pair | Mean cos | Std cos | Negative % |
|-----------|----------|---------|------------|
| nll_vs_pairwise | -0.172 | 0.041 | 100% |
| nll_vs_anchor | +0.149 | 0.071 | 0% |
| pairwise_vs_anchor | +0.926 | 0.028 | 0% |
**Parameter group: `probe`** (`unc_head.alpha`, `unc_head.probe_conv.weight`, `unc_head.probe_to_class.weight`)
| Loss pair | Mean cos | Std cos | Negative % |
|-----------|----------|---------|------------|
| nll_vs_pairwise | +0.749 | 0.313 | 8% |
| nll_vs_trust | -0.580 | 0.336 | 100% |
| nll_vs_anchor | +0.963 | 0.037 | 0% |
| nll_vs_residual | -0.928 | 0.039 | 100% |
| pairwise_vs_trust | -0.410 | 0.321 | 92% |
| pairwise_vs_anchor | +0.647 | 0.343 | 8% |
| pairwise_vs_residual | -0.613 | 0.285 | 92% |
| trust_vs_anchor | -0.573 | 0.336 | 100% |
| trust_vs_residual | +0.604 | 0.377 | 0% |
| anchor_vs_residual | -0.921 | 0.056 | 100% |
**Parameter group: `ranking`** (`unc_head.err_a`, `unc_head.err_b`, `unc_head.err_c`)
| Loss pair | Mean cos | Std cos | Negative % |
|-----------|----------|---------|------------|
| pairwise_vs_anchor | -0.213 | 0.132 | 92% |

Xet Storage Details

Size:
3.48 kB
·
Xet hash:
c2680aafc91ca3b2cd6aee9c49b018109981c8803d43d27371b8d6912a9f455e

Xet efficiently stores files, intelligently splitting them into unique chunks and accelerating uploads and downloads. More info.