📸 Vision Quickstart¶
Giskard is an open-source framework for testing all ML models, from LLMs to tabular models. Don’t hesitate to give the project a star on GitHub ⭐️ if you find it useful!
In this notebook, you’ll learn how to create comprehensive test suites for your model in a few lines of code, thanks to Giskard’s open-source Python library.
Use-case:
Landmark detection
Model:
Face Detection Cascade Classifier: https://raw.githubusercontent.com/opencv/opencv/master/data/haarcascades/haarcascade_frontalface_alt2.xml
Facial Landmark Detection: https://github.com/kurnianggoro/GSOC2017/raw/master/data/lbfmodel.yaml
Dataset:
Outline:
Detect vulnerabilities automatically with Giskard’s scan
Install dependencies¶
To run the scan on a vision model, you would need to install both the giskard-vision and the giskard library.
[ ]:
%pip install giskard giskard-vision
Two additional packages are required to run the scan, deepface that is used in EthnicityDetectorLandmark and sixdrepnet that is used in HeadPoseDetectorLandmark. We further install opencv_contrib_python to anticipate the known error mentionned here.
[ ]:
%pip install -U opencv_contrib_python deepface sixdrepnet
Needed imports¶
[2]:
from giskard_vision.landmark_detection.models.wrappers import OpenCVWrapper
from giskard_vision.landmark_detection.demo import get_300W
from giskard_vision.scanner import scan
Load model and dataset¶
In this example, we load the demo wrapper for an OpenCV facial landmark detection model and the demo dataloader for a sample from the 300-W faces dataset (see its official page here).
[ ]:
model = OpenCVWrapper()
dataloader = get_300W()
Detect vulnerabilities in the model¶
Giskard’s scan allows you to detect vulnerabilities in your model automatically. On landmark detection, these include performance biases, unrobustness and ethical issues.
[ ]:
results = scan(model, dataloader)
If you are running in a notebook, you can display the scan report directly in the notebook using display(...), otherwise you can export the report to an HTML file. Check the API Reference for more details on the export methods available on the ScanReport class.
[5]:
display(results)
# Save it to file
results.to_html("scan_report.html")