📸 Vision Quickstart

Giskard is an open-source framework for testing all ML models, from LLMs to tabular models. Don’t hesitate to give the project a star on GitHub ⭐️ if you find it useful!

In this notebook, you’ll learn how to scan an image classification model in a few lines of code, thanks to Giskard’s open-source Python library.

Use-case:

• Skin cancer detection

• Hugging Face skin cancer classification model

• Hugging Face skin cancer dataset

Outline:

• Detect vulnerabilities automatically with Giskard’s scan

• Automatically generate a test report for your image classification model beyond accuracy-related metrics

Install dependencies

To run the scan on a vision model, you would need to install both the giskard-vision and the giskard library.

%pip install giskard giskard-vision

Import libraries

In this example, we load the demo wrapper for a Hugging Face skin cancer detection model and the demo dataloader for the Hugging Face skin cancer image classification dataset.

from giskard_vision.image_classification.models.wrappers import SkinCancerHFModel
from giskard_vision.image_classification.dataloaders.loaders import DataLoaderSkinCancer
from giskard_vision.core.scanner import scan

ds = DataLoaderSkinCancer()
model = SkinCancerHFModel()

Generate scan report

Giskard’s scan allows you to detect vulnerabilities in your model automatically. On landmark detection, these include performance biases, unrobustness and ethical issues.

results = scan(model, ds, raise_exceptions=True, num_images=5)

If you are running in a notebook, you can display the scan report directly in the notebook using display(...), otherwise you can export the report to an HTML file. Check the API Reference for more details on the export methods available on the ScanReport class.

display(results)

# Save it to file
results.to_html("scan_report.html")