Open In Colab View Notebook on GitHub

📸 Vision Quickstart

Giskard is an open-source framework for testing all ML models, from LLMs to tabular models. Don’t hesitate to give the project a star on GitHub ⭐️ if you find it useful!

In this notebook, you’ll learn how to create comprehensive test suites for your model in a few lines of code, thanks to Giskard’s open-source Python library.

Use-case:

Outline:

  • Detect vulnerabilities automatically with Giskard’s scan

Install dependencies

To run the scan on a vision model, you would need to install both the giskard-vision and the giskard library.

[ ]:
%pip install giskard giskard-vision

Two additional packages are required to run the scan, deepface that is used in EthnicityDetectorLandmark and sixdrepnet that is used in HeadPoseDetectorLandmark. We further install opencv_contrib_python to anticipate the known error mentionned here.

[ ]:
%pip install -U opencv_contrib_python deepface sixdrepnet

Needed imports

[2]:
from giskard_vision.landmark_detection.models.wrappers import OpenCVWrapper
from giskard_vision.landmark_detection.demo import get_300W

from giskard_vision.scanner import scan

Load model and dataset

In this example, we load the demo wrapper for an OpenCV facial landmark detection model and the demo dataloader for a sample from the 300-W faces dataset (see its official page here).

[ ]:
model = OpenCVWrapper()
dataloader = get_300W()

Detect vulnerabilities in the model

Giskard’s scan allows you to detect vulnerabilities in your model automatically. On landmark detection, these include performance biases, unrobustness and ethical issues.

[ ]:
results = scan(model, dataloader)

If you are running in a notebook, you can display the scan report directly in the notebook using display(...), otherwise you can export the report to an HTML file. Check the API Reference for more details on the export methods available on the ScanReport class.

[5]:
display(results)

# Save it to file
results.to_html("scan_report.html")