Stability tests¶
- giskard.testing.test_smoothness(model: SuiteInput | BaseModel | None = None, dataset: SuiteInput | Dataset | None = None, feature_names: SuiteInput | List[str] | None = None, feature_values: SuiteInput | dict | None = None, random_state: SuiteInput | int | None = 123456, num_samples: SuiteInput | int | None = 100, num_grid: SuiteInput | int | None = 50, classif_index_label: SuiteInput | int | None = 0, threshold: SuiteInput | float | None = 2, ord: SuiteInput | int | None = 2, ref_function: SuiteInput | ndarray | None = None) GiskardTestMethod[source]¶
Test if the model is smooth with respect to given columns. The smoothness score is computed as follows S(y) = 1/N * sum((y_{i-1} - 2y_i + y_{i+1})^2) for i=1 to N
Then a ratio S(y) / S(y_ref), where y_ref is given through ref_function, is compared to the threshold in log value. If ref_function is None, S(y_ref) = 1
- Parameters:
model (BaseModel) – Model used to compute the test
dataset (Dataset) – Actual dataset used to compute the test
feature_names (List[str]) – Column names to test (at the same time)
feature_values (Optional[dict]) – Values to test for each column, the values should always have the same length
random_state (int) – Random state for sampling
num_samples (int) – Number of samples from which generate the grid and test
num_grid (int) – Number of points in the grid of values for the colum
classif_index_label (int) – If classification, which index to consider for the test
threshold (float) – Threshold over which the test is failed
ord (int) – Order of the norm used to evaluate smoothness
ref_function (np.ndarray) – Function values to compare to
- Returns:
The test result.
- Return type:
- giskard.testing.test_monotonicity(model: SuiteInput | BaseModel | None = None, dataset: SuiteInput | Dataset | None = None, feature_names: SuiteInput | List[str] | None = None, feature_values: SuiteInput | dict | None = None, increasing: SuiteInput | bool | None = True, random_state: SuiteInput | int | None = 123456, num_samples: SuiteInput | int | None = 100, num_grid: SuiteInput | int | None = 50, classif_index_label: SuiteInput | int | None = 0) GiskardTestMethod[source]¶
Test if the model is monotonic for a given column name by selecting random samples from the dataset and augmenting them with different values for the specified column
- Parameters:
model (BaseModel) – Model used to compute the test
dataset (Dataset) – Actual dataset used to compute the test
feature_names (List[str]) – Column names to test (at the same time)
feature_values (Optional[dict]) – Values to test for each column, the values should always have the same length
increasing (bool) – Whether to test for increasing or decreasing monotonicity
random_state (int) – Random state for sampling
num_samples (int) – Number of samples from which generate the grid and test
num_grid (int) – Number of points in the grid of values for the colum
classif_index_label (int) – If classification, which index to consider for the test
- Returns:
The test result.
- Return type: