Contribute to Giskard¶

How to add your custom ML tests to Giskard’s open-source project

Push your tests in the Giskard repo¶

Clone the Giskard repository
Create a GitHub branch with the base as main, starting with test-contribution/name-of-your-branch
Write your test inside one of the classes (MetamorphicTests, HeuristicTests, PerformanceTests or DriftTests) inside this repo. If your test does not fit these classes, you can also create a custom class in a new file.
We recommend writing unit tests for your test functions: this is the way you can execute and debug your test! Unit tests should be placed in this directory.

Hint

Fixtures

A unit test is executed with a test model and test data provided as fixtures.

For example, in test_precision function, we use german_credit_data and german_credit_model as fixtures. If necessary, you can create your own fixtures; check this directory to know how.

Create a Pull Request

Hint

Example

Let us guide you with an example where you want to create a heuristic test function:

Since you are writing a heuristic test, you will select heuristic_tests.py from the list of files under https://github.com/Giskard-AI/giskard/tree/main/giskard/testing
In heuristic_tests.py, write your code under class HeuristicTests(AbstractTestCollection) after the existing codes
To write the unit test we select test_heuristic.py under the list of files https://github.com/Giskard-AI/giskard/tree/main/tests
Follow the pattern of parameterizing the inputs which helps you to test the function using different fixtures. You can read more about parameterizing unit tests here https://docs.pytest.org/en/7.1.x/example/parametrize.html
If you want to create your own dataset, You can create your own fixture python file under https://github.com/Giskard-AI/giskard/tree/main/tests
You can use https://github.com/Giskard-AI/giskard/tree/main/tests/fixtures/german_credit_scoring.py for your reference. Make sure you return the model, data(with target) and test_data(without target) in the expected format.