| <!DOCTYPE html> |
| <html> |
| <head> |
| <meta charset="utf-8" /> |
| <meta name="viewport" content="width=device-width" /> |
| <title>MLTest Demo</title> |
| <link rel="stylesheet" href="style.css" /> |
| <link href="https://fonts.googleapis.com/css2?family=Source+Sans+Pro:ital,wght@0,200;0,300;0,400;0,600;0,700;0,900;1,200;1,300;1,400;1,600;1,700;1,900&display=swap" rel="stylesheet"> |
| <link href="https://fonts.googleapis.com/css2?family=IBM+Plex+Mono:wght@400;600;700&display=swap" rel="stylesheet"> |
| </head> |
| <body> |
| <div class="container"> |
| <div> |
| <h1>MLTest</h1> |
| <p> |
| This is a demo of MLTest on the dataset |
| <a href="https://huggingface.co/datasets/marmal88/skin_cancer"><code>marmal88/skin_cancer</code></a>. |
| Each image is labeled with one of three different kinds of cancers. |
| </p> |
| <p> |
| The model has been trained on five models: two variants of Swin Transformers, ViT, ResNet, and BEiT. The test results for each model can |
| be inspected in the dashboard below. |
| </p> |
| <p> |
| <b>Performance tests</b>: |
| in order to measure how well the model performs, we compute common performance metrics like accuracy, |
| precision, recall, F1 score, and more. |
| </p> |
| <p> |
| <b>Failure clusters</b>: |
| these clusters give meaningful insights when the model is failing and can be inspected in the "Failure Clusters" tab. |
| These failure clusters are automatically detected |
| for different combinations of metadata. |
| For example, the BEiT transformer has a significantly lower accuracy on images taken of cancers of the back with class label <code>0</code>. |
| </p> |
| <p> |
| <b>Robustness</b>: these tests help ML developers evaluate how well their model performs under different conditions. |
| These conditions could include different levels of brightness, compression, and many other types of interference. |
| </p><p> |
| The following robustness tests were enabled for this test case: |
| </p> |
| <ul> |
| <li>Brightness</li> |
| <li>CompressImage</li> |
| <li>Contrast</li> |
| <li>DarkSpots</li> |
| <li>GaussianBlur</li> |
| <li>GaussianNoise</li> |
| <li>Glare</li> |
| <li>GlassBlur</li> |
| <li>HorizontalFlip</li> |
| <li>MedianBlur</li> |
| <li>MotionBlur</li> |
| <li>OilSpots</li> |
| <li>Perspective</li> |
| <li>VerticalFlip</li> |
| </ul> |
|
|
|
|
| <p> |
| The full list of transforms supported by MLTest can be found in the <a target="_blank" href="https://docs.lakera.ai/configuration/robustness">documentation</a>. |
| </p> |
| <p> |
| <b>Fairness tests</b>: these tests measure how fair your model is. That means, whether its performance is dependent |
| on a protected attribute of a person. In this dataset, the age and gender of a subject may be considered |
| protected attributes. |
| </p> |
| <p> |
| We used two types of fairness tests on the age and gender of a person. |
| The <code>Equalized Odds</code> test checks that true positive and false positive rates are equal amongst protected attributes. |
| The <code>Predictive Equality</code> test checks that the false positive rates are equal amongst protected attributes. |
| </p> |
| <p> |
| More fairness tests supported by MLTest can be found in the <a target="_blank" href="https://docs.lakera.ai/configuration/fairness">documentation</a>. |
| </p> |
| </div> |
| |
| <iframe src="https://hf.lakera.ai/projects/skin_cancer_run2"></iframe> |
| </div> |
| </body> |
| </html> |
|
|
