==========================================================
   Clinical Decision Support Agent - Validation Suite
==========================================================

  Datasets:     MedQA 
  Cases/dataset: 1
  Drug check:    Yes
  Guidelines:    Yes
  Resume:        No
  Fetch only:    No

============================================================
  DATASET 1: MedQA (USMLE-style diagnostic accuracy)
============================================================
  Loading MedQA from cache: F:\kaggle\medgemma_impact_challenge\src\backend\validation\data\medqa_test.jsonl
  Loaded 1 MedQA cases

.\venv\Scripts\python.exe : 
At line:1 char:174
+ ... lyContinue; .\venv\Scripts\python.exe -m validation.run_validation -- ...
+                 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : NotSpecified: (:String) [], RemoteException
    + FullyQualifiedErrorId : NativeCommandError
 
Loading weights:   0%|          | 0/103 [00:00<?, ?it/s]
Loading weights:   1%|          | 1/103 [00:00<00:00, 19065.02it/s, 
Materializing param=embeddings.LayerNorm.bias]
Loading weights:   1%|          | 1/103 [00:00<00:00, 5504.34it/s, 
Materializing param=embeddings.LayerNorm.bias] 
Loading weights:   2%|ΓûÅ         | 2/103 [00:00<00:00, 3855.06it/s, 
Materializing param=embeddings.LayerNorm.weight]
Loading weights:   2%|ΓûÅ         | 2/103 [00:00<00:00, 3482.20it/s, 
Materializing param=embeddings.LayerNorm.weight]
Loading weights:   3%|ΓûÄ         | 3/103 [00:00<00:00, 4359.98it/s, 
Materializing param=embeddings.position_embeddings.weight]
Loading weights:   3%|ΓûÄ         | 3/103 [00:00<00:00, 4124.19it/s, 
Materializing param=embeddings.position_embeddings.weight]
Loading weights:   4%|Γûì         | 4/103 [00:00<00:00, 4960.74it/s, 
Materializing param=embeddings.token_type_embeddings.weight]
Loading weights:   4%|Γûì         | 4/103 [00:00<00:00, 4470.35it/s, 
Materializing param=embeddings.token_type_embeddings.weight]
Loading weights:   5%|Γûì         | 5/103 [00:00<00:00, 3788.21it/s, 
Materializing param=embeddings.word_embeddings.weight]      
Loading weights:   5%|Γûì         | 5/103 [00:00<00:00, 3614.53it/s, 
Materializing param=embeddings.word_embeddings.weight]
Loading weights:   6%|Γûî         | 6/103 [00:00<00:00, 3141.41it/s, 
Materializing param=encoder.layer.0.attention.output.LayerNorm.bias]
Loading weights:   6%|Γûî         | 6/103 [00:00<00:00, 3036.42it/s, 
Materializing param=encoder.layer.0.attention.output.LayerNorm.bias]
Loading weights:   7%|Γûï         | 7/103 [00:00<00:00, 3350.08it/s, 
Materializing param=encoder.layer.0.attention.output.LayerNorm.weight]
Loading weights:   7%|Γûï         | 7/103 [00:00<00:00, 3287.81it/s, 
Materializing param=encoder.layer.0.attention.output.LayerNorm.weight]
Loading weights:   8%|Γûè         | 8/103 [00:00<00:00, 3629.47it/s, 
Materializing param=encoder.layer.0.attention.output.dense.bias]      
Loading weights:   8%|Γûè         | 8/103 [00:00<00:00, 3572.66it/s, 
Materializing param=encoder.layer.0.attention.output.dense.bias]
Loading weights:   9%|Γûè         | 9/103 [00:00<00:00, 3874.84it/s, 
Materializing param=encoder.layer.0.attention.output.dense.weight]
Loading weights:   9%|Γûè         | 9/103 [00:00<00:00, 3819.18it/s, 
Materializing param=encoder.layer.0.attention.output.dense.weight]
Loading weights:  10%|Γûë         | 10/103 [00:00<00:00, 3813.00it/s, 
Materializing param=encoder.layer.0.attention.self.key.bias]     
Loading weights:  10%|Γûë         | 10/103 [00:00<00:00, 3603.04it/s, 
Materializing param=encoder.layer.0.attention.self.key.bias]
Loading weights:  11%|Γûê         | 11/103 [00:00<00:00, 3584.88it/s, 
Materializing param=encoder.layer.0.attention.self.key.weight]
Loading weights:  11%|Γûê         | 11/103 [00:00<00:00, 3500.56it/s, 
Materializing param=encoder.layer.0.attention.self.key.weight]
Loading weights:  12%|ΓûêΓûÅ        | 12/103 [00:00<00:00, 3682.44it/s, 
Materializing param=encoder.layer.0.attention.self.query.bias]
Loading weights:  12%|ΓûêΓûÅ        | 12/103 [00:00<00:00, 3581.30it/s, 
Materializing param=encoder.layer.0.attention.self.query.bias]
Loading weights:  13%|ΓûêΓûÄ        | 13/103 [00:00<00:00, 3355.86it/s, 
Materializing param=encoder.layer.0.attention.self.query.weight]
Loading weights:  13%|ΓûêΓûÄ        | 13/103 [00:00<00:00, 3265.42it/s, 
Materializing param=encoder.layer.0.attention.self.query.weight]
Loading weights:  14%|ΓûêΓûÄ        | 14/103 [00:00<00:00, 3102.62it/s, 
Materializing param=encoder.layer.0.attention.self.value.bias]  
Loading weights:  14%|ΓûêΓûÄ        | 14/103 [00:00<00:00, 3056.12it/s, 
Materializing param=encoder.layer.0.attention.self.value.bias]
Loading weights:  15%|ΓûêΓûì        | 15/103 [00:00<00:00, 2961.94it/s, 
Materializing param=encoder.layer.0.attention.self.value.weight]
Loading weights:  15%|ΓûêΓûì        | 15/103 [00:00<00:00, 2896.09it/s, 
Materializing param=encoder.layer.0.attention.self.value.weight]
Loading weights:  16%|ΓûêΓûî        | 16/103 [00:00<00:00, 2895.37it/s, 
Materializing param=encoder.layer.0.intermediate.dense.bias]    
Loading weights:  16%|ΓûêΓûî        | 16/103 [00:00<00:00, 2689.73it/s, 
Materializing param=encoder.layer.0.intermediate.dense.bias]
Loading weights:  17%|ΓûêΓûï        | 17/103 [00:00<00:00, 2702.62it/s, 
Materializing param=encoder.layer.0.intermediate.dense.weight]
Loading weights:  17%|ΓûêΓûï        | 17/103 [00:00<00:00, 2671.73it/s, 
Materializing param=encoder.layer.0.intermediate.dense.weight]
Loading weights:  17%|ΓûêΓûï        | 18/103 [00:00<00:00, 2678.17it/s, 
Materializing param=encoder.layer.0.output.LayerNorm.bias]    
Loading weights:  17%|ΓûêΓûï        | 18/103 [00:00<00:00, 2542.17it/s, 
Materializing param=encoder.layer.0.output.LayerNorm.bias]
Loading weights:  18%|ΓûêΓûè        | 19/103 [00:00<00:00, 2556.11it/s, 
Materializing param=encoder.layer.0.output.LayerNorm.weight]
Loading weights:  18%|ΓûêΓûè        | 19/103 [00:00<00:00, 2535.85it/s, 
Materializing param=encoder.layer.0.output.LayerNorm.weight]
Loading weights:  19%|ΓûêΓûë        | 20/103 [00:00<00:00, 2582.54it/s, 
Materializing param=encoder.layer.0.output.dense.bias]      
Loading weights:  19%|ΓûêΓûë        | 20/103 [00:00<00:00, 2498.10it/s, 
Materializing param=encoder.layer.0.output.dense.bias]
Loading weights:  20%|ΓûêΓûê        | 21/103 [00:00<00:00, 2512.99it/s, 
Materializing param=encoder.layer.0.output.dense.weight]
Loading weights:  20%|ΓûêΓûê        | 21/103 [00:00<00:00, 2433.29it/s, 
Materializing param=encoder.layer.0.output.dense.weight]
Loading weights:  21%|ΓûêΓûêΓûÅ       | 22/103 [00:00<00:00, 2454.44it/s, 
Materializing param=encoder.layer.1.attention.output.LayerNorm.bias]
Loading weights:  21%|ΓûêΓûêΓûÅ       | 22/103 [00:00<00:00, 2441.78it/s, 
Materializing param=encoder.layer.1.attention.output.LayerNorm.bias]
Loading weights:  22%|ΓûêΓûêΓûÅ       | 23/103 [00:00<00:00, 2473.82it/s, 
Materializing param=encoder.layer.1.attention.output.LayerNorm.weight]
Loading weights:  22%|ΓûêΓûêΓûÅ       | 23/103 [00:00<00:00, 2425.00it/s, 
Materializing param=encoder.layer.1.attention.output.LayerNorm.weight]
Loading weights:  23%|ΓûêΓûêΓûÄ       | 24/103 [00:00<00:00, 2464.22it/s, 
Materializing param=encoder.layer.1.attention.output.dense.bias]      
Loading weights:  23%|ΓûêΓûêΓûÄ       | 24/103 [00:00<00:00, 2448.69it/s, 
Materializing param=encoder.layer.1.attention.output.dense.bias]
Loading weights:  24%|ΓûêΓûêΓûì       | 25/103 [00:00<00:00, 2527.85it/s, 
Materializing param=encoder.layer.1.attention.output.dense.weight]
Loading weights:  24%|ΓûêΓûêΓûì       | 25/103 [00:00<00:00, 2518.56it/s, 
Materializing param=encoder.layer.1.attention.output.dense.weight]
Loading weights:  25%|ΓûêΓûêΓûî       | 26/103 [00:00<00:00, 2599.94it/s, 
Materializing param=encoder.layer.1.attention.self.key.bias]      
Loading weights:  25%|ΓûêΓûêΓûî       | 26/103 [00:00<00:00, 2591.23it/s, 
Materializing param=encoder.layer.1.attention.self.key.bias]
Loading weights:  26%|ΓûêΓûêΓûî       | 27/103 [00:00<00:00, 2594.06it/s, 
Materializing param=encoder.layer.1.attention.self.key.weight]
Loading weights:  26%|ΓûêΓûêΓûî       | 27/103 [00:00<00:00, 2573.43it/s, 
Materializing param=encoder.layer.1.attention.self.key.weight]
Loading weights:  27%|ΓûêΓûêΓûï       | 28/103 [00:00<00:00, 2616.42it/s, 
Materializing param=encoder.layer.1.attention.self.query.bias]
Loading weights:  27%|ΓûêΓûêΓûï       | 28/103 [00:00<00:00, 2605.10it/s, 
Materializing param=encoder.layer.1.attention.self.query.bias]
Loading weights:  28%|ΓûêΓûêΓûè       | 29/103 [00:00<00:00, 2679.12it/s, 
Materializing param=encoder.layer.1.attention.self.query.weight]
Loading weights:  28%|ΓûêΓûêΓûè       | 29/103 [00:00<00:00, 2670.48it/s, 
Materializing param=encoder.layer.1.attention.self.query.weight]
Loading weights:  29%|ΓûêΓûêΓûë       | 30/103 [00:00<00:00, 2726.64it/s, 
Materializing param=encoder.layer.1.attention.self.value.bias]  
Loading weights:  29%|ΓûêΓûêΓûë       | 30/103 [00:00<00:00, 2717.57it/s, 
Materializing param=encoder.layer.1.attention.self.value.bias]
Loading weights:  30%|ΓûêΓûêΓûê       | 31/103 [00:00<00:00, 2790.26it/s, 
Materializing param=encoder.layer.1.attention.self.value.weight]
Loading weights:  30%|ΓûêΓûêΓûê       | 31/103 [00:00<00:00, 2782.20it/s, 
Materializing param=encoder.layer.1.attention.self.value.weight]
Loading weights:  31%|ΓûêΓûêΓûê       | 32/103 [00:00<00:00, 2854.42it/s, 
Materializing param=encoder.layer.1.intermediate.dense.bias]    
Loading weights:  31%|ΓûêΓûêΓûê       | 32/103 [00:00<00:00, 2846.07it/s, 
Materializing param=encoder.layer.1.intermediate.dense.bias]
Loading weights:  32%|ΓûêΓûêΓûêΓûÅ      | 33/103 [00:00<00:00, 2881.66it/s, 
Materializing param=encoder.layer.1.intermediate.dense.weight]
Loading weights:  32%|ΓûêΓûêΓûêΓûÅ      | 33/103 [00:00<00:00, 2865.20it/s, 
Materializing param=encoder.layer.1.intermediate.dense.weight]
Loading weights:  33%|ΓûêΓûêΓûêΓûÄ      | 34/103 [00:00<00:00, 2928.02it/s, 
Materializing param=encoder.layer.1.output.LayerNorm.bias]    
Loading weights:  33%|ΓûêΓûêΓûêΓûÄ      | 34/103 [00:00<00:00, 2919.03it/s, 
Materializing param=encoder.layer.1.output.LayerNorm.bias]
Loading weights:  34%|ΓûêΓûêΓûêΓûì      | 35/103 [00:00<00:00, 2941.72it/s, 
Materializing param=encoder.layer.1.output.LayerNorm.weight]
Loading weights:  34%|ΓûêΓûêΓûêΓûì      | 35/103 [00:00<00:00, 2929.86it/s, 
Materializing param=encoder.layer.1.output.LayerNorm.weight]
Loading weights:  35%|ΓûêΓûêΓûêΓûì      | 36/103 [00:00<00:00, 2993.56it/s, 
Materializing param=encoder.layer.1.output.dense.bias]      
Loading weights:  35%|ΓûêΓûêΓûêΓûì      | 36/103 [00:00<00:00, 2985.51it/s, 
Materializing param=encoder.layer.1.output.dense.bias]
Loading weights:  36%|ΓûêΓûêΓûêΓûî      | 37/103 [00:00<00:00, 3010.29it/s, 
Materializing param=encoder.layer.1.output.dense.weight]
Loading weights:  36%|ΓûêΓûêΓûêΓûî      | 37/103 [00:00<00:00, 3001.44it/s, 
Materializing param=encoder.layer.1.output.dense.weight]
Loading weights:  37%|ΓûêΓûêΓûêΓûï      | 38/103 [00:00<00:00, 1948.86it/s, 
Materializing param=encoder.layer.2.attention.output.LayerNorm.bias]
Loading weights:  37%|ΓûêΓûêΓûêΓûï      | 38/103 [00:00<00:00, 1941.59it/s, 
Materializing param=encoder.layer.2.attention.output.LayerNorm.bias]
Loading weights:  38%|ΓûêΓûêΓûêΓûè      | 39/103 [00:00<00:00, 1983.39it/s, 
Materializing param=encoder.layer.2.attention.output.LayerNorm.weight]
Loading weights:  38%|ΓûêΓûêΓûêΓûè      | 39/103 [00:00<00:00, 1979.24it/s, 
Materializing param=encoder.layer.2.attention.output.LayerNorm.weight]
Loading weights:  39%|ΓûêΓûêΓûêΓûë      | 40/103 [00:00<00:00, 2022.62it/s, 
Materializing param=encoder.layer.2.attention.output.dense.bias]      
Loading weights:  39%|ΓûêΓûêΓûêΓûë      | 40/103 [00:00<00:00, 2018.97it/s, 
Materializing param=encoder.layer.2.attention.output.dense.bias]
Loading weights:  40%|ΓûêΓûêΓûêΓûë      | 41/103 [00:00<00:00, 2061.16it/s, 
Materializing param=encoder.layer.2.attention.output.dense.weight]
Loading weights:  40%|ΓûêΓûêΓûêΓûë      | 41/103 [00:00<00:00, 2057.73it/s, 
Materializing param=encoder.layer.2.attention.output.dense.weight]
Loading weights:  41%|ΓûêΓûêΓûêΓûê      | 42/103 [00:00<00:00, 2101.61it/s, 
Materializing param=encoder.layer.2.attention.self.key.bias]      
Loading weights:  41%|ΓûêΓûêΓûêΓûê      | 42/103 [00:00<00:00, 2098.30it/s, 
Materializing param=encoder.layer.2.attention.self.key.bias]
Loading weights:  42%|ΓûêΓûêΓûêΓûêΓûÅ     | 43/103 [00:00<00:00, 2141.78it/s, 
Materializing param=encoder.layer.2.attention.self.key.weight]
Loading weights:  42%|ΓûêΓûêΓûêΓûêΓûÅ     | 43/103 [00:00<00:00, 2138.48it/s, 
Materializing param=encoder.layer.2.attention.self.key.weight]
Loading weights:  43%|ΓûêΓûêΓûêΓûêΓûÄ     | 44/103 [00:00<00:00, 2182.05it/s, 
Materializing param=encoder.layer.2.attention.self.query.bias]
Loading weights:  43%|ΓûêΓûêΓûêΓûêΓûÄ     | 44/103 [00:00<00:00, 2178.81it/s, 
Materializing param=encoder.layer.2.attention.self.query.bias]
Loading weights:  44%|ΓûêΓûêΓûêΓûêΓûÄ     | 45/103 [00:00<00:00, 2222.13it/s, 
Materializing param=encoder.layer.2.attention.self.query.weight]
Loading weights:  44%|ΓûêΓûêΓûêΓûêΓûÄ     | 45/103 [00:00<00:00, 2218.66it/s, 
Materializing param=encoder.layer.2.attention.self.query.weight]
Loading weights:  45%|ΓûêΓûêΓûêΓûêΓûì     | 46/103 [00:00<00:00, 2261.66it/s, 
Materializing param=encoder.layer.2.attention.self.value.bias]  
Loading weights:  45%|ΓûêΓûêΓûêΓûêΓûì     | 46/103 [00:00<00:00, 2257.51it/s, 
Materializing param=encoder.layer.2.attention.self.value.bias]
Loading weights:  46%|ΓûêΓûêΓûêΓûêΓûî     | 47/103 [00:00<00:00, 2299.72it/s, 
Materializing param=encoder.layer.2.attention.self.value.weight]
Loading weights:  46%|ΓûêΓûêΓûêΓûêΓûî     | 47/103 [00:00<00:00, 2296.00it/s, 
Materializing param=encoder.layer.2.attention.self.value.weight]
Loading weights:  47%|ΓûêΓûêΓûêΓûêΓûï     | 48/103 [00:00<00:00, 2337.91it/s, 
Materializing param=encoder.layer.2.intermediate.dense.bias]    
Loading weights:  47%|ΓûêΓûêΓûêΓûêΓûï     | 48/103 [00:00<00:00, 2334.33it/s, 
Materializing param=encoder.layer.2.intermediate.dense.bias]
Loading weights:  48%|ΓûêΓûêΓûêΓûêΓûè     | 49/103 [00:00<00:00, 2376.32it/s, 
Materializing param=encoder.layer.2.intermediate.dense.weight]
Loading weights:  48%|ΓûêΓûêΓûêΓûêΓûè     | 49/103 [00:00<00:00, 2372.70it/s, 
Materializing param=encoder.layer.2.intermediate.dense.weight]
Loading weights:  49%|ΓûêΓûêΓûêΓûêΓûè     | 50/103 [00:00<00:00, 2407.75it/s, 
Materializing param=encoder.layer.2.output.LayerNorm.bias]    
Loading weights:  49%|ΓûêΓûêΓûêΓûêΓûè     | 50/103 [00:00<00:00, 2403.06it/s, 
Materializing param=encoder.layer.2.output.LayerNorm.bias]
Loading weights:  50%|ΓûêΓûêΓûêΓûêΓûë     | 51/103 [00:00<00:00, 2443.48it/s, 
Materializing param=encoder.layer.2.output.LayerNorm.weight]
Loading weights:  50%|ΓûêΓûêΓûêΓûêΓûë     | 51/103 [00:00<00:00, 2439.80it/s, 
Materializing param=encoder.layer.2.output.LayerNorm.weight]
Loading weights:  50%|ΓûêΓûêΓûêΓûêΓûê     | 52/103 [00:00<00:00, 2479.47it/s, 
Materializing param=encoder.layer.2.output.dense.bias]      
Loading weights:  50%|ΓûêΓûêΓûêΓûêΓûê     | 52/103 [00:00<00:00, 2475.44it/s, 
Materializing param=encoder.layer.2.output.dense.bias]
Loading weights:  51%|ΓûêΓûêΓûêΓûêΓûêΓûÅ    | 53/103 [00:00<00:00, 
2510.28it/s, Materializing param=encoder.layer.2.output.dense.weight]
Loading weights:  51%|ΓûêΓûêΓûêΓûêΓûêΓûÅ    | 53/103 [00:00<00:00, 
2499.39it/s, Materializing param=encoder.layer.2.output.dense.weight]
Loading weights:  52%|ΓûêΓûêΓûêΓûêΓûêΓûÅ    | 54/103 [00:00<00:00, 
2533.70it/s, Materializing 
param=encoder.layer.3.attention.output.LayerNorm.bias]
Loading weights:  52%|ΓûêΓûêΓûêΓûêΓûêΓûÅ    | 54/103 [00:00<00:00, 
2528.95it/s, Materializing 
param=encoder.layer.3.attention.output.LayerNorm.bias]
Loading weights:  53%|ΓûêΓûêΓûêΓûêΓûêΓûÄ    | 55/103 [00:00<00:00, 
2563.56it/s, Materializing 
param=encoder.layer.3.attention.output.LayerNorm.weight]
Loading weights:  53%|ΓûêΓûêΓûêΓûêΓûêΓûÄ    | 55/103 [00:00<00:00, 
2559.43it/s, Materializing 
param=encoder.layer.3.attention.output.LayerNorm.weight]
Loading weights:  54%|ΓûêΓûêΓûêΓûêΓûêΓûì    | 56/103 [00:00<00:00, 
2597.78it/s, Materializing param=encoder.layer.3.attention.output.dense.bias]  
    
Loading weights:  54%|ΓûêΓûêΓûêΓûêΓûêΓûì    | 56/103 [00:00<00:00, 
2593.65it/s, Materializing param=encoder.layer.3.attention.output.dense.bias]
Loading weights:  55%|ΓûêΓûêΓûêΓûêΓûêΓûî    | 57/103 [00:00<00:00, 
2631.65it/s, Materializing param=encoder.layer.3.attention.output.dense.weight]
Loading weights:  55%|ΓûêΓûêΓûêΓûêΓûêΓûî    | 57/103 [00:00<00:00, 
2627.61it/s, Materializing param=encoder.layer.3.attention.output.dense.weight]
Loading weights:  56%|ΓûêΓûêΓûêΓûêΓûêΓûï    | 58/103 [00:00<00:00, 
2664.92it/s, Materializing param=encoder.layer.3.attention.self.key.bias]      
Loading weights:  56%|ΓûêΓûêΓûêΓûêΓûêΓûï    | 58/103 [00:00<00:00, 
2659.88it/s, Materializing param=encoder.layer.3.attention.self.key.bias]
Loading weights:  57%|ΓûêΓûêΓûêΓûêΓûêΓûï    | 59/103 [00:00<00:00, 
2696.24it/s, Materializing param=encoder.layer.3.attention.self.key.weight]
Loading weights:  57%|ΓûêΓûêΓûêΓûêΓûêΓûï    | 59/103 [00:00<00:00, 
2691.38it/s, Materializing param=encoder.layer.3.attention.self.key.weight]
Loading weights:  58%|ΓûêΓûêΓûêΓûêΓûêΓûè    | 60/103 [00:00<00:00, 
2727.26it/s, Materializing param=encoder.layer.3.attention.self.query.bias]
Loading weights:  58%|ΓûêΓûêΓûêΓûêΓûêΓûè    | 60/103 [00:00<00:00, 
2722.51it/s, Materializing param=encoder.layer.3.attention.self.query.bias]
Loading weights:  59%|ΓûêΓûêΓûêΓûêΓûêΓûë    | 61/103 [00:00<00:00, 
2757.69it/s, Materializing param=encoder.layer.3.attention.self.query.weight]
Loading weights:  59%|ΓûêΓûêΓûêΓûêΓûêΓûë    | 61/103 [00:00<00:00, 
2751.31it/s, Materializing param=encoder.layer.3.attention.self.query.weight]
Loading weights:  60%|ΓûêΓûêΓûêΓûêΓûêΓûê    | 62/103 [00:00<00:00, 
2785.09it/s, Materializing param=encoder.layer.3.attention.self.value.bias]  
Loading weights:  60%|ΓûêΓûêΓûêΓûêΓûêΓûê    | 62/103 [00:00<00:00, 
2779.56it/s, Materializing param=encoder.layer.3.attention.self.value.bias]
Loading weights:  61%|ΓûêΓûêΓûêΓûêΓûêΓûê    | 63/103 [00:00<00:00, 
2813.77it/s, Materializing param=encoder.layer.3.attention.self.value.weight]
Loading weights:  61%|ΓûêΓûêΓûêΓûêΓûêΓûê    | 63/103 [00:00<00:00, 
2808.63it/s, Materializing param=encoder.layer.3.attention.self.value.weight]
Loading weights:  62%|ΓûêΓûêΓûêΓûêΓûêΓûêΓûÅ   | 64/103 [00:00<00:00, 
2844.89it/s, Materializing param=encoder.layer.3.intermediate.dense.bias]    
Loading weights:  62%|ΓûêΓûêΓûêΓûêΓûêΓûêΓûÅ   | 64/103 [00:00<00:00, 
2840.65it/s, Materializing param=encoder.layer.3.intermediate.dense.bias]
Loading weights:  63%|ΓûêΓûêΓûêΓûêΓûêΓûêΓûÄ   | 65/103 [00:00<00:00, 
2876.18it/s, Materializing param=encoder.layer.3.intermediate.dense.weight]
Loading weights:  63%|ΓûêΓûêΓûêΓûêΓûêΓûêΓûÄ   | 65/103 [00:00<00:00, 
2871.12it/s, Materializing param=encoder.layer.3.intermediate.dense.weight]
Loading weights:  64%|ΓûêΓûêΓûêΓûêΓûêΓûêΓûì   | 66/103 [00:00<00:00, 
2906.90it/s, Materializing param=encoder.layer.3.output.LayerNorm.bias]    
Loading weights:  64%|ΓûêΓûêΓûêΓûêΓûêΓûêΓûì   | 66/103 [00:00<00:00, 
2902.54it/s, Materializing param=encoder.layer.3.output.LayerNorm.bias]
Loading weights:  65%|ΓûêΓûêΓûêΓûêΓûêΓûêΓûî   | 67/103 [00:00<00:00, 
2937.92it/s, Materializing param=encoder.layer.3.output.LayerNorm.weight]
Loading weights:  65%|ΓûêΓûêΓûêΓûêΓûêΓûêΓûî   | 67/103 [00:00<00:00, 
2933.69it/s, Materializing param=encoder.layer.3.output.LayerNorm.weight]
Loading weights:  66%|ΓûêΓûêΓûêΓûêΓûêΓûêΓûî   | 68/103 [00:00<00:00, 
2969.26it/s, Materializing param=encoder.layer.3.output.dense.bias]      
Loading weights:  66%|ΓûêΓûêΓûêΓûêΓûêΓûêΓûî   | 68/103 [00:00<00:00, 
2965.07it/s, Materializing param=encoder.layer.3.output.dense.bias]
Loading weights:  67%|ΓûêΓûêΓûêΓûêΓûêΓûêΓûï   | 69/103 [00:00<00:00, 
3000.44it/s, Materializing param=encoder.layer.3.output.dense.weight]
Loading weights:  67%|ΓûêΓûêΓûêΓûêΓûêΓûêΓûï   | 69/103 [00:00<00:00, 
2995.71it/s, Materializing param=encoder.layer.3.output.dense.weight]
Loading weights:  68%|ΓûêΓûêΓûêΓûêΓûêΓûêΓûè   | 70/103 [00:00<00:00, 
3029.75it/s, Materializing 
param=encoder.layer.4.attention.output.LayerNorm.bias]
Loading weights:  68%|ΓûêΓûêΓûêΓûêΓûêΓûêΓûè   | 70/103 [00:00<00:00, 
3025.26it/s, Materializing 
param=encoder.layer.4.attention.output.LayerNorm.bias]
Loading weights:  69%|ΓûêΓûêΓûêΓûêΓûêΓûêΓûë   | 71/103 [00:00<00:00, 
3058.89it/s, Materializing 
param=encoder.layer.4.attention.output.LayerNorm.weight]
Loading weights:  69%|ΓûêΓûêΓûêΓûêΓûêΓûêΓûë   | 71/103 [00:00<00:00, 
3054.22it/s, Materializing 
param=encoder.layer.4.attention.output.LayerNorm.weight]
Loading weights:  70%|ΓûêΓûêΓûêΓûêΓûêΓûêΓûë   | 72/103 [00:00<00:00, 
3088.43it/s, Materializing param=encoder.layer.4.attention.output.dense.bias]  
    
Loading weights:  70%|ΓûêΓûêΓûêΓûêΓûêΓûêΓûë   | 72/103 [00:00<00:00, 
3083.29it/s, Materializing param=encoder.layer.4.attention.output.dense.bias]
Loading weights:  71%|ΓûêΓûêΓûêΓûêΓûêΓûêΓûê   | 73/103 [00:00<00:00, 
3114.48it/s, Materializing param=encoder.layer.4.attention.output.dense.weight]
Loading weights:  71%|ΓûêΓûêΓûêΓûêΓûêΓûêΓûê   | 73/103 [00:00<00:00, 
3109.48it/s, Materializing param=encoder.layer.4.attention.output.dense.weight]
Loading weights:  72%|ΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûÅ  | 74/103 [00:00<00:00, 
3143.01it/s, Materializing param=encoder.layer.4.attention.self.key.bias]      
Loading weights:  72%|ΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûÅ  | 74/103 [00:00<00:00, 
3138.78it/s, Materializing param=encoder.layer.4.attention.self.key.bias]
Loading weights:  73%|ΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûÄ  | 75/103 [00:00<00:00, 
3171.48it/s, Materializing param=encoder.layer.4.attention.self.key.weight]
Loading weights:  73%|ΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûÄ  | 75/103 [00:00<00:00, 
3166.28it/s, Materializing param=encoder.layer.4.attention.self.key.weight]
Loading weights:  74%|ΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûì  | 76/103 [00:00<00:00, 
3196.62it/s, Materializing param=encoder.layer.4.attention.self.query.bias]
Loading weights:  74%|ΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûì  | 76/103 [00:00<00:00, 
3190.96it/s, Materializing param=encoder.layer.4.attention.self.query.bias]
Loading weights:  75%|ΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûì  | 77/103 [00:00<00:00, 
3220.69it/s, Materializing param=encoder.layer.4.attention.self.query.weight]
Loading weights:  75%|ΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûì  | 77/103 [00:00<00:00, 
3216.07it/s, Materializing param=encoder.layer.4.attention.self.query.weight]
Loading weights:  76%|ΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûî  | 78/103 [00:00<00:00, 
3246.78it/s, Materializing param=encoder.layer.4.attention.self.value.bias]  
Loading weights:  76%|ΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûî  | 78/103 [00:00<00:00, 
3241.80it/s, Materializing param=encoder.layer.4.attention.self.value.bias]
Loading weights:  77%|ΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûï  | 79/103 [00:00<00:00, 
3271.33it/s, Materializing param=encoder.layer.4.attention.self.value.weight]
Loading weights:  77%|ΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûï  | 79/103 [00:00<00:00, 
3266.37it/s, Materializing param=encoder.layer.4.attention.self.value.weight]
Loading weights:  78%|ΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûè  | 80/103 [00:00<00:00, 
3296.53it/s, Materializing param=encoder.layer.4.intermediate.dense.bias]    
Loading weights:  78%|ΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûè  | 80/103 [00:00<00:00, 
3291.42it/s, Materializing param=encoder.layer.4.intermediate.dense.bias]
Loading weights:  79%|ΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûè  | 81/103 [00:00<00:00, 
3318.57it/s, Materializing param=encoder.layer.4.intermediate.dense.weight]
Loading weights:  79%|ΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûè  | 81/103 [00:00<00:00, 
3312.52it/s, Materializing param=encoder.layer.4.intermediate.dense.weight]
Loading weights:  80%|ΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûë  | 82/103 [00:00<00:00, 
3339.93it/s, Materializing param=encoder.layer.4.output.LayerNorm.bias]    
Loading weights:  80%|ΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûë  | 82/103 [00:00<00:00, 
3334.33it/s, Materializing param=encoder.layer.4.output.LayerNorm.bias]
Loading weights:  81%|ΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûê  | 83/103 [00:00<00:00, 
3363.48it/s, Materializing param=encoder.layer.4.output.LayerNorm.weight]
Loading weights:  81%|ΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûê  | 83/103 [00:00<00:00, 
3358.91it/s, Materializing param=encoder.layer.4.output.LayerNorm.weight]
Loading weights:  82%|ΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûÅ | 84/103 [00:00<00:00, 
3389.40it/s, Materializing param=encoder.layer.4.output.dense.bias]      
Loading weights:  82%|ΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûÅ | 84/103 [00:00<00:00, 
3384.62it/s, Materializing param=encoder.layer.4.output.dense.bias]
Loading weights:  83%|ΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûÄ | 85/103 [00:00<00:00, 
3415.69it/s, Materializing param=encoder.layer.4.output.dense.weight]
Loading weights:  83%|ΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûÄ | 85/103 [00:00<00:00, 
3410.72it/s, Materializing param=encoder.layer.4.output.dense.weight]
Loading weights:  83%|ΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûÄ | 86/103 [00:00<00:00, 
3440.61it/s, Materializing 
param=encoder.layer.5.attention.output.LayerNorm.bias]
Loading weights:  83%|ΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûÄ | 86/103 [00:00<00:00, 
3436.19it/s, Materializing 
param=encoder.layer.5.attention.output.LayerNorm.bias]
Loading weights:  84%|ΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûì | 87/103 [00:00<00:00, 
3467.59it/s, Materializing 
param=encoder.layer.5.attention.output.LayerNorm.weight]
Loading weights:  84%|ΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûì | 87/103 [00:00<00:00, 
3463.04it/s, Materializing 
param=encoder.layer.5.attention.output.LayerNorm.weight]
Loading weights:  85%|ΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûî | 88/103 [00:00<00:00, 
3493.53it/s, Materializing param=encoder.layer.5.attention.output.dense.bias]  
    
Loading weights:  85%|ΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûî | 88/103 [00:00<00:00, 
3489.07it/s, Materializing param=encoder.layer.5.attention.output.dense.bias]
Loading weights:  86%|ΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûï | 89/103 [00:00<00:00, 
3520.50it/s, Materializing param=encoder.layer.5.attention.output.dense.weight]
Loading weights:  86%|ΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûï | 89/103 [00:00<00:00, 
3515.86it/s, Materializing param=encoder.layer.5.attention.output.dense.weight]
Loading weights:  87%|ΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûï | 90/103 [00:00<00:00, 
3545.01it/s, Materializing param=encoder.layer.5.attention.self.key.bias]      
Loading weights:  87%|ΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûï | 90/103 [00:00<00:00, 
3540.39it/s, Materializing param=encoder.layer.5.attention.self.key.bias]
Loading weights:  88%|ΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûè | 91/103 [00:00<00:00, 
3569.62it/s, Materializing param=encoder.layer.5.attention.self.key.weight]
Loading weights:  88%|ΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûè | 91/103 [00:00<00:00, 
3564.89it/s, Materializing param=encoder.layer.5.attention.self.key.weight]
Loading weights:  89%|ΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûë | 92/103 [00:00<00:00, 
3595.66it/s, Materializing param=encoder.layer.5.attention.self.query.bias]
Loading weights:  89%|ΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûë | 92/103 [00:00<00:00, 
3591.18it/s, Materializing param=encoder.layer.5.attention.self.query.bias]
Loading weights:  90%|ΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûê | 93/103 [00:00<00:00, 
3620.11it/s, Materializing param=encoder.layer.5.attention.self.query.weight]
Loading weights:  90%|ΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûê | 93/103 [00:00<00:00, 
3615.31it/s, Materializing param=encoder.layer.5.attention.self.query.weight]
Loading weights:  91%|ΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûÅ| 94/103 [00:00<00:00, 
3644.19it/s, Materializing param=encoder.layer.5.attention.self.value.bias]  
Loading weights:  91%|ΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûÅ| 94/103 [00:00<00:00, 
3639.34it/s, Materializing param=encoder.layer.5.attention.self.value.bias]
Loading weights:  92%|ΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûÅ| 95/103 [00:00<00:00, 
3668.92it/s, Materializing param=encoder.layer.5.attention.self.value.weight]
Loading weights:  92%|ΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûÅ| 95/103 [00:00<00:00, 
3664.36it/s, Materializing param=encoder.layer.5.attention.self.value.weight]
Loading weights:  93%|ΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûÄ| 96/103 [00:00<00:00, 
3693.59it/s, Materializing param=encoder.layer.5.intermediate.dense.bias]    
Loading weights:  93%|ΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûÄ| 96/103 [00:00<00:00, 
3688.52it/s, Materializing param=encoder.layer.5.intermediate.dense.bias]
Loading weights:  94%|ΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûì| 97/103 [00:00<00:00, 
3718.18it/s, Materializing param=encoder.layer.5.intermediate.dense.weight]
Loading weights:  94%|ΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûì| 97/103 [00:00<00:00, 
3713.03it/s, Materializing param=encoder.layer.5.intermediate.dense.weight]
Loading weights:  95%|ΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûî| 98/103 [00:00<00:00, 
3742.63it/s, Materializing param=encoder.layer.5.output.LayerNorm.bias]    
Loading weights:  95%|ΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûî| 98/103 [00:00<00:00, 
3738.00it/s, Materializing param=encoder.layer.5.output.LayerNorm.bias]
Loading weights:  96%|ΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûî| 99/103 [00:00<00:00, 
3766.31it/s, Materializing param=encoder.layer.5.output.LayerNorm.weight]
Loading weights:  96%|ΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûî| 99/103 [00:00<00:00, 
3761.57it/s, Materializing param=encoder.layer.5.output.LayerNorm.weight]
Loading weights:  97%|ΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûï| 100/103 [00:00<00:00, 
3789.95it/s, Materializing param=encoder.layer.5.output.dense.bias]     
Loading weights:  97%|ΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûï| 100/103 [00:00<00:00, 
3785.13it/s, Materializing param=encoder.layer.5.output.dense.bias]
Loading weights:  98%|ΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûè| 101/103 [00:00<00:00, 
3813.96it/s, Materializing param=encoder.layer.5.output.dense.weight]
Loading weights:  98%|ΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûè| 101/103 [00:00<00:00, 
3809.23it/s, Materializing param=encoder.layer.5.output.dense.weight]
Loading weights:  99%|ΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûë| 102/103 [00:00<00:00, 
3837.97it/s, Materializing param=pooler.dense.bias]                  
Loading weights:  99%|ΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûë| 102/103 [00:00<00:00, 
3833.68it/s, Materializing param=pooler.dense.bias]
Loading weights: 100%|ΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûê| 103/103 [00:00<00:00, 
3862.33it/s, Materializing param=pooler.dense.weight]
Loading weights: 100%|ΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûê| 103/103 [00:00<00:00, 
3857.26it/s, Materializing param=pooler.dense.weight]
Loading weights: 100%|ΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûêΓûê| 103/103 [00:00<00:00, 
3842.34it/s, Materializing param=pooler.dense.weight]
BertModel LOAD REPORT from: sentence-transformers/all-MiniLM-L6-v2
Key                     | Status     |  | 
------------------------+------------+--+-
embeddings.position_ids | UNEXPECTED |  | 

Notes:
- UNEXPECTED	:can be ignored when loading from different 
task/architecture; not ok if you expect identical arch.
  [1/1] medqa_0000: Γ£ô top1=N top3=N diff=Y [differential] (281547ms)

============================================================
  Validation Results: MEDQA
============================================================
  Total cases:      1
  Successful:       1
  Failed:           0
  Duration:         281.5s

  Metrics:
    avg_pipeline_time_ms           281547ms
    differential_accuracy          100.0%
    mentioned_accuracy             100.0%
    parse_success                  100.0%
    top1_accuracy                  0.0%
    top3_accuracy                  0.0%
============================================================


======================================================================
  COMBINED VALIDATION REPORT
======================================================================

  Dataset          Cases  Success                Key Metric    Value
  --------------- ------ -------- ------------------------- --------
  medqa                1        1             top3_accuracy    0.0%

  ΓöÇΓöÇΓöÇΓöÇΓöÇΓöÇΓöÇΓöÇΓöÇΓöÇΓöÇΓöÇΓöÇΓöÇΓöÇΓöÇΓöÇΓöÇΓöÇΓöÇΓöÇΓöÇΓöÇΓöÇΓöÇΓöÇΓöÇΓöÇΓöÇΓöÇΓöÇΓöÇΓöÇΓöÇΓöÇΓöÇΓöÇΓöÇΓöÇΓöÇΓöÇΓöÇΓöÇΓöÇΓöÇΓöÇΓöÇΓöÇΓöÇΓöÇΓöÇΓöÇΓöÇΓöÇΓöÇΓöÇΓöÇΓöÇΓöÇΓöÇΓöÇΓöÇΓöÇΓöÇΓöÇΓöÇ

  MEDQA metrics:
    avg_pipeline_time_ms                281547ms
    differential_accuracy               100.0%
    mentioned_accuracy                  100.0%
    parse_success                       100.0%
    top1_accuracy                       0.0%
    top3_accuracy                       0.0%

  Total cases:     1
  Total success:   1
  Total duration:  281.6s (4.7min)
  Timestamp:       2026-02-15T06:15:42.932073+00:00
======================================================================

  Combined report saved to: F:\kaggle\medgemma_impact_challenge\src\backend\validation\results\combined_20260215_061542.json