Contribute to Giskard#
How to add your custom ML tests to Giskard’s open-source project
Push your tests in the Giskard repo#
Clone the Giskard repository
Create a GitHub branch with the base as main, starting with test-contribution/name-of-your-branch
From the root of the cloned repo run ./gradlew generateProto. This will generate the module`generated` that you will need to create your tests.
Write your test inside one of the classes (MetamorphicTests, HeuristicTests, PerformanceTests or DriftTests) inside this repo. If your test does not fit these classes, you can also create a custom class in a new file.
We recommend writing unit tests for your test functions: this is the way you can execute and debug your test! Unit tests should be placed in this directory.
A unit test is executed with a test model and test data provided as fixtures.
Create a Pull Request
Let us guide you with an example where you want to create a heuristic test function:
Since you are writing a heuristic test, you will select heuristic_tests.py from the list of files under https://github.com/Giskard-AI/giskard/tree/main/giskard-ml-worker/ml_worker/testing
In heuristic_tests.py, write your code under class HeuristicTests(AbstractTestCollection) after the existing codes
To write the unit test we select test_heuristic.py under the list of files https://github.com/Giskard-AI/giskard/tree/main/giskard-ml-worker/test
Follow the pattern of parameterizing the inputs which helps you to test the function using different fixtures. You can read more about parameterizing unit tests here https://docs.pytest.org/en/7.1.x/example/parametrize.html
If you want to create your own dataset, You can create your own fixture python file under https://github.com/Giskard-AI/giskard/tree/main/giskard-ml-worker/test/fixtures
You can use https://github.com/Giskard-AI/giskard/blob/main/giskard-ml-worker/test/fixtures/german_credit_scoring.py for your reference. Make sure you return the model, data(with target) and test_data(without target) in the expected format.