Stability tests¶
- giskard.testing.test_smoothness(model: SuiteInput | BaseModel | None = None, dataset: SuiteInput | Dataset | None = None, feature_names: SuiteInput | List[str] | None = None, feature_values: SuiteInput | dict | None = None, random_state: SuiteInput | int | None = 123456, num_samples: SuiteInput | int | None = 100, num_grid: SuiteInput | int | None = 50, classif_index_label: SuiteInput | int | None = 0, threshold: SuiteInput | float | None = 2, ord: SuiteInput | int | None = 2, ref_function: SuiteInput | ndarray | None = None) GiskardTestMethod[source]¶
- Test if the model is smooth with respect to given columns.
The smoothness score is computed as follows $$S(y) =
rac{1}{N} sum_{i=1}^N (y_{i-1} - 2y_i + y_{i+1})^2$$
Then a ratio $$S(y) / S(y_{ ext{ref}})$$, where $$y_{ ext{ref}}$$ is given through ref_function, is compared to the threshold in log value. If ref_functino is None, $$S(y_{ ext{ref}}) = 1$$
- modelBaseModel
Model used to compute the test
- datasetDataset
Actual dataset used to compute the test
- feature_namesList[str]
Column names to test (at the same time)
- feature_valuesOptional[dict]
Values to test for each column, the values should always have the same length
- random_stateint
Random state for sampling
- num_samplesint
Number of samples from which generate the grid and test
- num_gridint
Number of points in the grid of values for the colum
- classif_index_labelint
If classification, which index to consider for the test
- thresholdfloat
Threshold over which the test is failed
- ordint
Order of the norm used to evaluate smoothness
- ref_functionnp.ndarray
Function values to compare to
- TestResult
The test result.
- giskard.testing.test_monotonicity(model: SuiteInput | BaseModel | None = None, dataset: SuiteInput | Dataset | None = None, feature_names: SuiteInput | List[str] | None = None, feature_values: SuiteInput | dict | None = None, increasing: SuiteInput | bool | None = True, random_state: SuiteInput | int | None = 123456, num_samples: SuiteInput | int | None = 100, num_grid: SuiteInput | int | None = 50, classif_index_label: SuiteInput | int | None = 0) GiskardTestMethod[source]¶
Test if the model is monotonic for a given column name by selecting random samples from the dataset and augmenting them with different values for the specified column
- Parameters:
model (BaseModel) – Model used to compute the test
dataset (Dataset) – Actual dataset used to compute the test
feature_names (List[str]) – Column names to test (at the same time)
feature_values (Optional[dict]) – Values to test for each column, the values should always have the same length
increasing (bool) – Whether to test for increasing or decreasing monotonicity
random_state (int) – Random state for sampling
num_samples (int) – Number of samples from which generate the grid and test
num_grid (int) – Number of points in the grid of values for the colum
classif_index_label (int) – If classification, which index to consider for the test
- Returns:
The test result.
- Return type: