Breast cancer detection [XGBoost]

Giskard is an open-source framework for testing all ML models, from LLMs to tabular models. Don’t hesitate to give the project a star on GitHub ⭐️ if you find it useful!

In this notebook, you’ll learn how to create comprehensive test suites for your model in a few lines of code, thanks to Giskard’s open-source Python library.

Use-case:

• Binary classification. Whether the patient has the breast cancer.

• Model: XGBClassifier

• Dataset

Outline:

• Detect vulnerabilities automatically with Giskard’s scan

• Automatically generate & curate a comprehensive test suite to test your model beyond accuracy-related metrics

Install dependencies

Make sure to install the giskard

%pip install giskard --upgrade

Import libraries

import pandas as pd
from sklearn.datasets import load_breast_cancer
from sklearn.metrics import classification_report
from sklearn.model_selection import train_test_split
from xgboost import XGBClassifier

from giskard import Dataset, Model, scan, testing

Define constants

# Constants.
TARGET_COLUMN_NAME = "target"

RANDOM_SEED = 42

Dataset preparation

Load data

data = load_breast_cancer(as_frame=True)
features = data['data']
target = data[TARGET_COLUMN_NAME]

Train-test split

# Train/test split.
X_train, X_test, y_train, y_test = train_test_split(features.loc[:, features.columns != TARGET_COLUMN_NAME], target,
                                                    random_state=RANDOM_SEED)

Wrap dataset with Giskard

To prepare for the vulnerability scan, make sure to wrap your dataset using Giskard’s Dataset class. More details here.

raw_data = pd.concat([X_test, y_test], axis=1)
giskard_dataset = Dataset(
    df=raw_data,
    # A pandas.DataFrame that contains the raw data (before all the pre-processing steps) and the actual ground truth variable (target).
    target="target",  # Ground truth variable.
    name="breast_cancer",  # Optional.
)

Model building

Build estimator

xgb = XGBClassifier()
xgb.fit(X_train, y_train)

# Evaluation.
y_test_pred = xgb.predict(X_test)
print(classification_report(y_test, y_test_pred))

Wrap model with Giskard

To prepare for the vulnerability scan, make sure to wrap your model using Giskard’s Model class. You can choose to either wrap the prediction function (preferred option) or the model object. More details here.

giskard_model = Model(
    model=xgb,
    # A prediction function that encapsulates all the data pre-processing steps and that could be executed with the dataset used by the scan.
    model_type="classification",  # Either regression, classification or text_generation.
    name="breast_cancer_xgboost",  # Optional.
    classification_labels=[0, 1],  # Their order MUST be identical to the prediction_function's output order.
    feature_names=X_test.columns,  # Default: all columns of your dataset.
)

# Validate the model wrapping.
y_test_pred_wrapped = giskard_model.predict(giskard_dataset).raw_prediction
print(classification_report(y_test, y_test_pred_wrapped))

Detect vulnerabilities in your model

Scan your model for vulnerabilities with Giskard

Giskard’s scan allows you to detect vulnerabilities in your model automatically. These include performance biases, unrobustness, data leakage, stochasticity, underconfidence, ethical issues, and more. For detailed information about the scan feature, please refer to our scan documentation.

results = scan(giskard_model, giskard_dataset)

display(results)  # in your notebook

Generate comprehensive test suites automatically for your model

Generate test suites from the scan

The objects produced by the scan can be used as fixtures to generate a test suite that integrate all detected vulnerabilities. Test suites allow you to evaluate and validate your model’s performance, ensuring that it behaves as expected on a set of predefined test cases, and to identify any regressions or issues that might arise during development or updates.

test_suite = results.generate_test_suite("My first test suite")
test_suite.run()

2024-05-29 11:38:26,362 pid:50687 MainThread giskard.datasets.base INFO     Casting dataframe columns from {'mean radius': 'float64', 'mean texture': 'float64', 'mean perimeter': 'float64', 'mean area': 'float64', 'mean smoothness': 'float64', 'mean compactness': 'float64', 'mean concavity': 'float64', 'mean concave points': 'float64', 'mean symmetry': 'float64', 'mean fractal dimension': 'float64', 'radius error': 'float64', 'texture error': 'float64', 'perimeter error': 'float64', 'area error': 'float64', 'smoothness error': 'float64', 'compactness error': 'float64', 'concavity error': 'float64', 'concave points error': 'float64', 'symmetry error': 'float64', 'fractal dimension error': 'float64', 'worst radius': 'float64', 'worst texture': 'float64', 'worst perimeter': 'float64', 'worst area': 'float64', 'worst smoothness': 'float64', 'worst compactness': 'float64', 'worst concavity': 'float64', 'worst concave points': 'float64', 'worst symmetry': 'float64', 'worst fractal dimension': 'float64'} to {'mean radius': 'float64', 'mean texture': 'float64', 'mean perimeter': 'float64', 'mean area': 'float64', 'mean smoothness': 'float64', 'mean compactness': 'float64', 'mean concavity': 'float64', 'mean concave points': 'float64', 'mean symmetry': 'float64', 'mean fractal dimension': 'float64', 'radius error': 'float64', 'texture error': 'float64', 'perimeter error': 'float64', 'area error': 'float64', 'smoothness error': 'float64', 'compactness error': 'float64', 'concavity error': 'float64', 'concave points error': 'float64', 'symmetry error': 'float64', 'fractal dimension error': 'float64', 'worst radius': 'float64', 'worst texture': 'float64', 'worst perimeter': 'float64', 'worst area': 'float64', 'worst smoothness': 'float64', 'worst compactness': 'float64', 'worst concavity': 'float64', 'worst concave points': 'float64', 'worst symmetry': 'float64', 'worst fractal dimension': 'float64'}
2024-05-29 11:38:26,366 pid:50687 MainThread giskard.utils.logging_utils INFO     Predicted dataset with shape (143, 31) executed in 0:00:00.007008
Executed 'Nominal association (Theil's U) on data slice “`worst concave points` >= 0.144”' with arguments {'model': <giskard.models.sklearn.SKLearnModel object at 0x17f45eec0>, 'dataset': <giskard.datasets.base.Dataset object at 0x17f2b1db0>, 'slicing_function': <giskard.slicing.slice.QueryBasedSliceFunction object at 0x3322aeaa0>, 'threshold': 0.5}:
               Test failed
               Metric: 0.77
                - [INFO] metric = 0.7745014140794255, threshold = 0.5

2024-05-29 11:38:26,374 pid:50687 MainThread giskard.datasets.base INFO     Casting dataframe columns from {'mean radius': 'float64', 'mean texture': 'float64', 'mean perimeter': 'float64', 'mean area': 'float64', 'mean smoothness': 'float64', 'mean compactness': 'float64', 'mean concavity': 'float64', 'mean concave points': 'float64', 'mean symmetry': 'float64', 'mean fractal dimension': 'float64', 'radius error': 'float64', 'texture error': 'float64', 'perimeter error': 'float64', 'area error': 'float64', 'smoothness error': 'float64', 'compactness error': 'float64', 'concavity error': 'float64', 'concave points error': 'float64', 'symmetry error': 'float64', 'fractal dimension error': 'float64', 'worst radius': 'float64', 'worst texture': 'float64', 'worst perimeter': 'float64', 'worst area': 'float64', 'worst smoothness': 'float64', 'worst compactness': 'float64', 'worst concavity': 'float64', 'worst concave points': 'float64', 'worst symmetry': 'float64', 'worst fractal dimension': 'float64'} to {'mean radius': 'float64', 'mean texture': 'float64', 'mean perimeter': 'float64', 'mean area': 'float64', 'mean smoothness': 'float64', 'mean compactness': 'float64', 'mean concavity': 'float64', 'mean concave points': 'float64', 'mean symmetry': 'float64', 'mean fractal dimension': 'float64', 'radius error': 'float64', 'texture error': 'float64', 'perimeter error': 'float64', 'area error': 'float64', 'smoothness error': 'float64', 'compactness error': 'float64', 'concavity error': 'float64', 'concave points error': 'float64', 'symmetry error': 'float64', 'fractal dimension error': 'float64', 'worst radius': 'float64', 'worst texture': 'float64', 'worst perimeter': 'float64', 'worst area': 'float64', 'worst smoothness': 'float64', 'worst compactness': 'float64', 'worst concavity': 'float64', 'worst concave points': 'float64', 'worst symmetry': 'float64', 'worst fractal dimension': 'float64'}
2024-05-29 11:38:26,376 pid:50687 MainThread giskard.utils.logging_utils INFO     Predicted dataset with shape (143, 31) executed in 0:00:00.005423
Executed 'Nominal association (Theil's U) on data slice “`worst radius` >= 17.420”' with arguments {'model': <giskard.models.sklearn.SKLearnModel object at 0x17f45eec0>, 'dataset': <giskard.datasets.base.Dataset object at 0x17f2b1db0>, 'slicing_function': <giskard.slicing.slice.QueryBasedSliceFunction object at 0x3322add50>, 'threshold': 0.5}:
               Test failed
               Metric: 0.76
                - [INFO] metric = 0.7639160128151123, threshold = 0.5

2024-05-29 11:38:26,385 pid:50687 MainThread giskard.datasets.base INFO     Casting dataframe columns from {'mean radius': 'float64', 'mean texture': 'float64', 'mean perimeter': 'float64', 'mean area': 'float64', 'mean smoothness': 'float64', 'mean compactness': 'float64', 'mean concavity': 'float64', 'mean concave points': 'float64', 'mean symmetry': 'float64', 'mean fractal dimension': 'float64', 'radius error': 'float64', 'texture error': 'float64', 'perimeter error': 'float64', 'area error': 'float64', 'smoothness error': 'float64', 'compactness error': 'float64', 'concavity error': 'float64', 'concave points error': 'float64', 'symmetry error': 'float64', 'fractal dimension error': 'float64', 'worst radius': 'float64', 'worst texture': 'float64', 'worst perimeter': 'float64', 'worst area': 'float64', 'worst smoothness': 'float64', 'worst compactness': 'float64', 'worst concavity': 'float64', 'worst concave points': 'float64', 'worst symmetry': 'float64', 'worst fractal dimension': 'float64'} to {'mean radius': 'float64', 'mean texture': 'float64', 'mean perimeter': 'float64', 'mean area': 'float64', 'mean smoothness': 'float64', 'mean compactness': 'float64', 'mean concavity': 'float64', 'mean concave points': 'float64', 'mean symmetry': 'float64', 'mean fractal dimension': 'float64', 'radius error': 'float64', 'texture error': 'float64', 'perimeter error': 'float64', 'area error': 'float64', 'smoothness error': 'float64', 'compactness error': 'float64', 'concavity error': 'float64', 'concave points error': 'float64', 'symmetry error': 'float64', 'fractal dimension error': 'float64', 'worst radius': 'float64', 'worst texture': 'float64', 'worst perimeter': 'float64', 'worst area': 'float64', 'worst smoothness': 'float64', 'worst compactness': 'float64', 'worst concavity': 'float64', 'worst concave points': 'float64', 'worst symmetry': 'float64', 'worst fractal dimension': 'float64'}
2024-05-29 11:38:26,388 pid:50687 MainThread giskard.utils.logging_utils INFO     Predicted dataset with shape (143, 31) executed in 0:00:00.005598
Executed 'Nominal association (Theil's U) on data slice “`worst perimeter` >= 110.950”' with arguments {'model': <giskard.models.sklearn.SKLearnModel object at 0x17f45eec0>, 'dataset': <giskard.datasets.base.Dataset object at 0x17f2b1db0>, 'slicing_function': <giskard.slicing.slice.QueryBasedSliceFunction object at 0x3322ae5c0>, 'threshold': 0.5}:
               Test failed
               Metric: 0.74
                - [INFO] metric = 0.7427987555404146, threshold = 0.5

2024-05-29 11:38:26,394 pid:50687 MainThread giskard.datasets.base INFO     Casting dataframe columns from {'mean radius': 'float64', 'mean texture': 'float64', 'mean perimeter': 'float64', 'mean area': 'float64', 'mean smoothness': 'float64', 'mean compactness': 'float64', 'mean concavity': 'float64', 'mean concave points': 'float64', 'mean symmetry': 'float64', 'mean fractal dimension': 'float64', 'radius error': 'float64', 'texture error': 'float64', 'perimeter error': 'float64', 'area error': 'float64', 'smoothness error': 'float64', 'compactness error': 'float64', 'concavity error': 'float64', 'concave points error': 'float64', 'symmetry error': 'float64', 'fractal dimension error': 'float64', 'worst radius': 'float64', 'worst texture': 'float64', 'worst perimeter': 'float64', 'worst area': 'float64', 'worst smoothness': 'float64', 'worst compactness': 'float64', 'worst concavity': 'float64', 'worst concave points': 'float64', 'worst symmetry': 'float64', 'worst fractal dimension': 'float64'} to {'mean radius': 'float64', 'mean texture': 'float64', 'mean perimeter': 'float64', 'mean area': 'float64', 'mean smoothness': 'float64', 'mean compactness': 'float64', 'mean concavity': 'float64', 'mean concave points': 'float64', 'mean symmetry': 'float64', 'mean fractal dimension': 'float64', 'radius error': 'float64', 'texture error': 'float64', 'perimeter error': 'float64', 'area error': 'float64', 'smoothness error': 'float64', 'compactness error': 'float64', 'concavity error': 'float64', 'concave points error': 'float64', 'symmetry error': 'float64', 'fractal dimension error': 'float64', 'worst radius': 'float64', 'worst texture': 'float64', 'worst perimeter': 'float64', 'worst area': 'float64', 'worst smoothness': 'float64', 'worst compactness': 'float64', 'worst concavity': 'float64', 'worst concave points': 'float64', 'worst symmetry': 'float64', 'worst fractal dimension': 'float64'}
2024-05-29 11:38:26,399 pid:50687 MainThread giskard.utils.logging_utils INFO     Predicted dataset with shape (143, 31) executed in 0:00:00.006882
Executed 'Nominal association (Theil's U) on data slice “`worst area` >= 885.950”' with arguments {'model': <giskard.models.sklearn.SKLearnModel object at 0x17f45eec0>, 'dataset': <giskard.datasets.base.Dataset object at 0x17f2b1db0>, 'slicing_function': <giskard.slicing.slice.QueryBasedSliceFunction object at 0x3322acaf0>, 'threshold': 0.5}:
               Test failed
               Metric: 0.71
                - [INFO] metric = 0.7131111241996619, threshold = 0.5

2024-05-29 11:38:26,406 pid:50687 MainThread giskard.datasets.base INFO     Casting dataframe columns from {'mean radius': 'float64', 'mean texture': 'float64', 'mean perimeter': 'float64', 'mean area': 'float64', 'mean smoothness': 'float64', 'mean compactness': 'float64', 'mean concavity': 'float64', 'mean concave points': 'float64', 'mean symmetry': 'float64', 'mean fractal dimension': 'float64', 'radius error': 'float64', 'texture error': 'float64', 'perimeter error': 'float64', 'area error': 'float64', 'smoothness error': 'float64', 'compactness error': 'float64', 'concavity error': 'float64', 'concave points error': 'float64', 'symmetry error': 'float64', 'fractal dimension error': 'float64', 'worst radius': 'float64', 'worst texture': 'float64', 'worst perimeter': 'float64', 'worst area': 'float64', 'worst smoothness': 'float64', 'worst compactness': 'float64', 'worst concavity': 'float64', 'worst concave points': 'float64', 'worst symmetry': 'float64', 'worst fractal dimension': 'float64'} to {'mean radius': 'float64', 'mean texture': 'float64', 'mean perimeter': 'float64', 'mean area': 'float64', 'mean smoothness': 'float64', 'mean compactness': 'float64', 'mean concavity': 'float64', 'mean concave points': 'float64', 'mean symmetry': 'float64', 'mean fractal dimension': 'float64', 'radius error': 'float64', 'texture error': 'float64', 'perimeter error': 'float64', 'area error': 'float64', 'smoothness error': 'float64', 'compactness error': 'float64', 'concavity error': 'float64', 'concave points error': 'float64', 'symmetry error': 'float64', 'fractal dimension error': 'float64', 'worst radius': 'float64', 'worst texture': 'float64', 'worst perimeter': 'float64', 'worst area': 'float64', 'worst smoothness': 'float64', 'worst compactness': 'float64', 'worst concavity': 'float64', 'worst concave points': 'float64', 'worst symmetry': 'float64', 'worst fractal dimension': 'float64'}
2024-05-29 11:38:26,409 pid:50687 MainThread giskard.utils.logging_utils INFO     Predicted dataset with shape (143, 31) executed in 0:00:00.005363
Executed 'Nominal association (Theil's U) on data slice “`mean concave points` >= 0.053”' with arguments {'model': <giskard.models.sklearn.SKLearnModel object at 0x17f45eec0>, 'dataset': <giskard.datasets.base.Dataset object at 0x17f2b1db0>, 'slicing_function': <giskard.slicing.slice.QueryBasedSliceFunction object at 0x3322f0730>, 'threshold': 0.5}:
               Test failed
               Metric: 0.64
                - [INFO] metric = 0.6377395426892722, threshold = 0.5

2024-05-29 11:38:26,419 pid:50687 MainThread giskard.datasets.base INFO     Casting dataframe columns from {'mean radius': 'float64', 'mean texture': 'float64', 'mean perimeter': 'float64', 'mean area': 'float64', 'mean smoothness': 'float64', 'mean compactness': 'float64', 'mean concavity': 'float64', 'mean concave points': 'float64', 'mean symmetry': 'float64', 'mean fractal dimension': 'float64', 'radius error': 'float64', 'texture error': 'float64', 'perimeter error': 'float64', 'area error': 'float64', 'smoothness error': 'float64', 'compactness error': 'float64', 'concavity error': 'float64', 'concave points error': 'float64', 'symmetry error': 'float64', 'fractal dimension error': 'float64', 'worst radius': 'float64', 'worst texture': 'float64', 'worst perimeter': 'float64', 'worst area': 'float64', 'worst smoothness': 'float64', 'worst compactness': 'float64', 'worst concavity': 'float64', 'worst concave points': 'float64', 'worst symmetry': 'float64', 'worst fractal dimension': 'float64'} to {'mean radius': 'float64', 'mean texture': 'float64', 'mean perimeter': 'float64', 'mean area': 'float64', 'mean smoothness': 'float64', 'mean compactness': 'float64', 'mean concavity': 'float64', 'mean concave points': 'float64', 'mean symmetry': 'float64', 'mean fractal dimension': 'float64', 'radius error': 'float64', 'texture error': 'float64', 'perimeter error': 'float64', 'area error': 'float64', 'smoothness error': 'float64', 'compactness error': 'float64', 'concavity error': 'float64', 'concave points error': 'float64', 'symmetry error': 'float64', 'fractal dimension error': 'float64', 'worst radius': 'float64', 'worst texture': 'float64', 'worst perimeter': 'float64', 'worst area': 'float64', 'worst smoothness': 'float64', 'worst compactness': 'float64', 'worst concavity': 'float64', 'worst concave points': 'float64', 'worst symmetry': 'float64', 'worst fractal dimension': 'float64'}
2024-05-29 11:38:26,424 pid:50687 MainThread giskard.utils.logging_utils INFO     Predicted dataset with shape (143, 31) executed in 0:00:00.009371
Executed 'Nominal association (Theil's U) on data slice “`mean perimeter` >= 96.380”' with arguments {'model': <giskard.models.sklearn.SKLearnModel object at 0x17f45eec0>, 'dataset': <giskard.datasets.base.Dataset object at 0x17f2b1db0>, 'slicing_function': <giskard.slicing.slice.QueryBasedSliceFunction object at 0x3322f3fa0>, 'threshold': 0.5}:
               Test failed
               Metric: 0.62
                - [INFO] metric = 0.6227293846650519, threshold = 0.5

2024-05-29 11:38:26,433 pid:50687 MainThread giskard.datasets.base INFO     Casting dataframe columns from {'mean radius': 'float64', 'mean texture': 'float64', 'mean perimeter': 'float64', 'mean area': 'float64', 'mean smoothness': 'float64', 'mean compactness': 'float64', 'mean concavity': 'float64', 'mean concave points': 'float64', 'mean symmetry': 'float64', 'mean fractal dimension': 'float64', 'radius error': 'float64', 'texture error': 'float64', 'perimeter error': 'float64', 'area error': 'float64', 'smoothness error': 'float64', 'compactness error': 'float64', 'concavity error': 'float64', 'concave points error': 'float64', 'symmetry error': 'float64', 'fractal dimension error': 'float64', 'worst radius': 'float64', 'worst texture': 'float64', 'worst perimeter': 'float64', 'worst area': 'float64', 'worst smoothness': 'float64', 'worst compactness': 'float64', 'worst concavity': 'float64', 'worst concave points': 'float64', 'worst symmetry': 'float64', 'worst fractal dimension': 'float64'} to {'mean radius': 'float64', 'mean texture': 'float64', 'mean perimeter': 'float64', 'mean area': 'float64', 'mean smoothness': 'float64', 'mean compactness': 'float64', 'mean concavity': 'float64', 'mean concave points': 'float64', 'mean symmetry': 'float64', 'mean fractal dimension': 'float64', 'radius error': 'float64', 'texture error': 'float64', 'perimeter error': 'float64', 'area error': 'float64', 'smoothness error': 'float64', 'compactness error': 'float64', 'concavity error': 'float64', 'concave points error': 'float64', 'symmetry error': 'float64', 'fractal dimension error': 'float64', 'worst radius': 'float64', 'worst texture': 'float64', 'worst perimeter': 'float64', 'worst area': 'float64', 'worst smoothness': 'float64', 'worst compactness': 'float64', 'worst concavity': 'float64', 'worst concave points': 'float64', 'worst symmetry': 'float64', 'worst fractal dimension': 'float64'}
2024-05-29 11:38:26,439 pid:50687 MainThread giskard.utils.logging_utils INFO     Predicted dataset with shape (143, 31) executed in 0:00:00.009926
Executed 'Nominal association (Theil's U) on data slice “`mean area` >= 697.300”' with arguments {'model': <giskard.models.sklearn.SKLearnModel object at 0x17f45eec0>, 'dataset': <giskard.datasets.base.Dataset object at 0x17f2b1db0>, 'slicing_function': <giskard.slicing.slice.QueryBasedSliceFunction object at 0x3322f0df0>, 'threshold': 0.5}:
               Test failed
               Metric: 0.61
                - [INFO] metric = 0.6087180717056299, threshold = 0.5

2024-05-29 11:38:26,449 pid:50687 MainThread giskard.datasets.base INFO     Casting dataframe columns from {'mean radius': 'float64', 'mean texture': 'float64', 'mean perimeter': 'float64', 'mean area': 'float64', 'mean smoothness': 'float64', 'mean compactness': 'float64', 'mean concavity': 'float64', 'mean concave points': 'float64', 'mean symmetry': 'float64', 'mean fractal dimension': 'float64', 'radius error': 'float64', 'texture error': 'float64', 'perimeter error': 'float64', 'area error': 'float64', 'smoothness error': 'float64', 'compactness error': 'float64', 'concavity error': 'float64', 'concave points error': 'float64', 'symmetry error': 'float64', 'fractal dimension error': 'float64', 'worst radius': 'float64', 'worst texture': 'float64', 'worst perimeter': 'float64', 'worst area': 'float64', 'worst smoothness': 'float64', 'worst compactness': 'float64', 'worst concavity': 'float64', 'worst concave points': 'float64', 'worst symmetry': 'float64', 'worst fractal dimension': 'float64'} to {'mean radius': 'float64', 'mean texture': 'float64', 'mean perimeter': 'float64', 'mean area': 'float64', 'mean smoothness': 'float64', 'mean compactness': 'float64', 'mean concavity': 'float64', 'mean concave points': 'float64', 'mean symmetry': 'float64', 'mean fractal dimension': 'float64', 'radius error': 'float64', 'texture error': 'float64', 'perimeter error': 'float64', 'area error': 'float64', 'smoothness error': 'float64', 'compactness error': 'float64', 'concavity error': 'float64', 'concave points error': 'float64', 'symmetry error': 'float64', 'fractal dimension error': 'float64', 'worst radius': 'float64', 'worst texture': 'float64', 'worst perimeter': 'float64', 'worst area': 'float64', 'worst smoothness': 'float64', 'worst compactness': 'float64', 'worst concavity': 'float64', 'worst concave points': 'float64', 'worst symmetry': 'float64', 'worst fractal dimension': 'float64'}
2024-05-29 11:38:26,455 pid:50687 MainThread giskard.utils.logging_utils INFO     Predicted dataset with shape (143, 31) executed in 0:00:00.010707
Executed 'Nominal association (Theil's U) on data slice “`mean radius` >= 15.060”' with arguments {'model': <giskard.models.sklearn.SKLearnModel object at 0x17f45eec0>, 'dataset': <giskard.datasets.base.Dataset object at 0x17f2b1db0>, 'slicing_function': <giskard.slicing.slice.QueryBasedSliceFunction object at 0x33230d4e0>, 'threshold': 0.5}:
               Test failed
               Metric: 0.61
                - [INFO] metric = 0.6087180717056299, threshold = 0.5

2024-05-29 11:38:26,463 pid:50687 MainThread giskard.datasets.base INFO     Casting dataframe columns from {'mean radius': 'float64', 'mean texture': 'float64', 'mean perimeter': 'float64', 'mean area': 'float64', 'mean smoothness': 'float64', 'mean compactness': 'float64', 'mean concavity': 'float64', 'mean concave points': 'float64', 'mean symmetry': 'float64', 'mean fractal dimension': 'float64', 'radius error': 'float64', 'texture error': 'float64', 'perimeter error': 'float64', 'area error': 'float64', 'smoothness error': 'float64', 'compactness error': 'float64', 'concavity error': 'float64', 'concave points error': 'float64', 'symmetry error': 'float64', 'fractal dimension error': 'float64', 'worst radius': 'float64', 'worst texture': 'float64', 'worst perimeter': 'float64', 'worst area': 'float64', 'worst smoothness': 'float64', 'worst compactness': 'float64', 'worst concavity': 'float64', 'worst concave points': 'float64', 'worst symmetry': 'float64', 'worst fractal dimension': 'float64'} to {'mean radius': 'float64', 'mean texture': 'float64', 'mean perimeter': 'float64', 'mean area': 'float64', 'mean smoothness': 'float64', 'mean compactness': 'float64', 'mean concavity': 'float64', 'mean concave points': 'float64', 'mean symmetry': 'float64', 'mean fractal dimension': 'float64', 'radius error': 'float64', 'texture error': 'float64', 'perimeter error': 'float64', 'area error': 'float64', 'smoothness error': 'float64', 'compactness error': 'float64', 'concavity error': 'float64', 'concave points error': 'float64', 'symmetry error': 'float64', 'fractal dimension error': 'float64', 'worst radius': 'float64', 'worst texture': 'float64', 'worst perimeter': 'float64', 'worst area': 'float64', 'worst smoothness': 'float64', 'worst compactness': 'float64', 'worst concavity': 'float64', 'worst concave points': 'float64', 'worst symmetry': 'float64', 'worst fractal dimension': 'float64'}
2024-05-29 11:38:26,467 pid:50687 MainThread giskard.utils.logging_utils INFO     Predicted dataset with shape (143, 31) executed in 0:00:00.007197
Executed 'Nominal association (Theil's U) on data slice “`area error` >= 39.690”' with arguments {'model': <giskard.models.sklearn.SKLearnModel object at 0x17f45eec0>, 'dataset': <giskard.datasets.base.Dataset object at 0x17f2b1db0>, 'slicing_function': <giskard.slicing.slice.QueryBasedSliceFunction object at 0x3322afee0>, 'threshold': 0.5}:
               Test failed
               Metric: 0.52
                - [INFO] metric = 0.5166070702259753, threshold = 0.5

2024-05-29 11:38:26,475 pid:50687 MainThread giskard.datasets.base INFO     Casting dataframe columns from {'mean radius': 'float64', 'mean texture': 'float64', 'mean perimeter': 'float64', 'mean area': 'float64', 'mean smoothness': 'float64', 'mean compactness': 'float64', 'mean concavity': 'float64', 'mean concave points': 'float64', 'mean symmetry': 'float64', 'mean fractal dimension': 'float64', 'radius error': 'float64', 'texture error': 'float64', 'perimeter error': 'float64', 'area error': 'float64', 'smoothness error': 'float64', 'compactness error': 'float64', 'concavity error': 'float64', 'concave points error': 'float64', 'symmetry error': 'float64', 'fractal dimension error': 'float64', 'worst radius': 'float64', 'worst texture': 'float64', 'worst perimeter': 'float64', 'worst area': 'float64', 'worst smoothness': 'float64', 'worst compactness': 'float64', 'worst concavity': 'float64', 'worst concave points': 'float64', 'worst symmetry': 'float64', 'worst fractal dimension': 'float64'} to {'mean radius': 'float64', 'mean texture': 'float64', 'mean perimeter': 'float64', 'mean area': 'float64', 'mean smoothness': 'float64', 'mean compactness': 'float64', 'mean concavity': 'float64', 'mean concave points': 'float64', 'mean symmetry': 'float64', 'mean fractal dimension': 'float64', 'radius error': 'float64', 'texture error': 'float64', 'perimeter error': 'float64', 'area error': 'float64', 'smoothness error': 'float64', 'compactness error': 'float64', 'concavity error': 'float64', 'concave points error': 'float64', 'symmetry error': 'float64', 'fractal dimension error': 'float64', 'worst radius': 'float64', 'worst texture': 'float64', 'worst perimeter': 'float64', 'worst area': 'float64', 'worst smoothness': 'float64', 'worst compactness': 'float64', 'worst concavity': 'float64', 'worst concave points': 'float64', 'worst symmetry': 'float64', 'worst fractal dimension': 'float64'}
2024-05-29 11:38:26,481 pid:50687 MainThread giskard.utils.logging_utils INFO     Predicted dataset with shape (30, 31) executed in 0:00:00.009332
Executed 'Accuracy on data slice “`worst radius` >= 14.765 AND `worst radius` < 17.625”' with arguments {'model': <giskard.models.sklearn.SKLearnModel object at 0x17f45eec0>, 'dataset': <giskard.datasets.base.Dataset object at 0x17f2b1db0>, 'slicing_function': <giskard.slicing.slice.QueryBasedSliceFunction object at 0x3322acee0>, 'threshold': 0.9101398601398601}:
               Test failed
               Metric: 0.8


2024-05-29 11:38:26,493 pid:50687 MainThread giskard.datasets.base INFO     Casting dataframe columns from {'mean radius': 'float64', 'mean texture': 'float64', 'mean perimeter': 'float64', 'mean area': 'float64', 'mean smoothness': 'float64', 'mean compactness': 'float64', 'mean concavity': 'float64', 'mean concave points': 'float64', 'mean symmetry': 'float64', 'mean fractal dimension': 'float64', 'radius error': 'float64', 'texture error': 'float64', 'perimeter error': 'float64', 'area error': 'float64', 'smoothness error': 'float64', 'compactness error': 'float64', 'concavity error': 'float64', 'concave points error': 'float64', 'symmetry error': 'float64', 'fractal dimension error': 'float64', 'worst radius': 'float64', 'worst texture': 'float64', 'worst perimeter': 'float64', 'worst area': 'float64', 'worst smoothness': 'float64', 'worst compactness': 'float64', 'worst concavity': 'float64', 'worst concave points': 'float64', 'worst symmetry': 'float64', 'worst fractal dimension': 'float64'} to {'mean radius': 'float64', 'mean texture': 'float64', 'mean perimeter': 'float64', 'mean area': 'float64', 'mean smoothness': 'float64', 'mean compactness': 'float64', 'mean concavity': 'float64', 'mean concave points': 'float64', 'mean symmetry': 'float64', 'mean fractal dimension': 'float64', 'radius error': 'float64', 'texture error': 'float64', 'perimeter error': 'float64', 'area error': 'float64', 'smoothness error': 'float64', 'compactness error': 'float64', 'concavity error': 'float64', 'concave points error': 'float64', 'symmetry error': 'float64', 'fractal dimension error': 'float64', 'worst radius': 'float64', 'worst texture': 'float64', 'worst perimeter': 'float64', 'worst area': 'float64', 'worst smoothness': 'float64', 'worst compactness': 'float64', 'worst concavity': 'float64', 'worst concave points': 'float64', 'worst symmetry': 'float64', 'worst fractal dimension': 'float64'}
2024-05-29 11:38:26,497 pid:50687 MainThread giskard.utils.logging_utils INFO     Predicted dataset with shape (30, 31) executed in 0:00:00.008997
Executed 'Accuracy on data slice “`worst perimeter` >= 96.625 AND `worst perimeter` < 122.350”' with arguments {'model': <giskard.models.sklearn.SKLearnModel object at 0x17f45eec0>, 'dataset': <giskard.datasets.base.Dataset object at 0x17f2b1db0>, 'slicing_function': <giskard.slicing.slice.QueryBasedSliceFunction object at 0x33229a260>, 'threshold': 0.9101398601398601}:
               Test failed
               Metric: 0.8


2024-05-29 11:38:26,507 pid:50687 MainThread giskard.datasets.base INFO     Casting dataframe columns from {'mean radius': 'float64', 'mean texture': 'float64', 'mean perimeter': 'float64', 'mean area': 'float64', 'mean smoothness': 'float64', 'mean compactness': 'float64', 'mean concavity': 'float64', 'mean concave points': 'float64', 'mean symmetry': 'float64', 'mean fractal dimension': 'float64', 'radius error': 'float64', 'texture error': 'float64', 'perimeter error': 'float64', 'area error': 'float64', 'smoothness error': 'float64', 'compactness error': 'float64', 'concavity error': 'float64', 'concave points error': 'float64', 'symmetry error': 'float64', 'fractal dimension error': 'float64', 'worst radius': 'float64', 'worst texture': 'float64', 'worst perimeter': 'float64', 'worst area': 'float64', 'worst smoothness': 'float64', 'worst compactness': 'float64', 'worst concavity': 'float64', 'worst concave points': 'float64', 'worst symmetry': 'float64', 'worst fractal dimension': 'float64'} to {'mean radius': 'float64', 'mean texture': 'float64', 'mean perimeter': 'float64', 'mean area': 'float64', 'mean smoothness': 'float64', 'mean compactness': 'float64', 'mean concavity': 'float64', 'mean concave points': 'float64', 'mean symmetry': 'float64', 'mean fractal dimension': 'float64', 'radius error': 'float64', 'texture error': 'float64', 'perimeter error': 'float64', 'area error': 'float64', 'smoothness error': 'float64', 'compactness error': 'float64', 'concavity error': 'float64', 'concave points error': 'float64', 'symmetry error': 'float64', 'fractal dimension error': 'float64', 'worst radius': 'float64', 'worst texture': 'float64', 'worst perimeter': 'float64', 'worst area': 'float64', 'worst smoothness': 'float64', 'worst compactness': 'float64', 'worst concavity': 'float64', 'worst concave points': 'float64', 'worst symmetry': 'float64', 'worst fractal dimension': 'float64'}
2024-05-29 11:38:26,511 pid:50687 MainThread giskard.utils.logging_utils INFO     Predicted dataset with shape (30, 31) executed in 0:00:00.008990
Executed 'Accuracy on data slice “`mean perimeter` >= 86.140 AND `mean perimeter` < 98.085”' with arguments {'model': <giskard.models.sklearn.SKLearnModel object at 0x17f45eec0>, 'dataset': <giskard.datasets.base.Dataset object at 0x17f2b1db0>, 'slicing_function': <giskard.slicing.slice.QueryBasedSliceFunction object at 0x3322f0f10>, 'threshold': 0.9101398601398601}:
               Test failed
               Metric: 0.8


2024-05-29 11:38:26,523 pid:50687 MainThread giskard.datasets.base INFO     Casting dataframe columns from {'mean radius': 'float64', 'mean texture': 'float64', 'mean perimeter': 'float64', 'mean area': 'float64', 'mean smoothness': 'float64', 'mean compactness': 'float64', 'mean concavity': 'float64', 'mean concave points': 'float64', 'mean symmetry': 'float64', 'mean fractal dimension': 'float64', 'radius error': 'float64', 'texture error': 'float64', 'perimeter error': 'float64', 'area error': 'float64', 'smoothness error': 'float64', 'compactness error': 'float64', 'concavity error': 'float64', 'concave points error': 'float64', 'symmetry error': 'float64', 'fractal dimension error': 'float64', 'worst radius': 'float64', 'worst texture': 'float64', 'worst perimeter': 'float64', 'worst area': 'float64', 'worst smoothness': 'float64', 'worst compactness': 'float64', 'worst concavity': 'float64', 'worst concave points': 'float64', 'worst symmetry': 'float64', 'worst fractal dimension': 'float64'} to {'mean radius': 'float64', 'mean texture': 'float64', 'mean perimeter': 'float64', 'mean area': 'float64', 'mean smoothness': 'float64', 'mean compactness': 'float64', 'mean concavity': 'float64', 'mean concave points': 'float64', 'mean symmetry': 'float64', 'mean fractal dimension': 'float64', 'radius error': 'float64', 'texture error': 'float64', 'perimeter error': 'float64', 'area error': 'float64', 'smoothness error': 'float64', 'compactness error': 'float64', 'concavity error': 'float64', 'concave points error': 'float64', 'symmetry error': 'float64', 'fractal dimension error': 'float64', 'worst radius': 'float64', 'worst texture': 'float64', 'worst perimeter': 'float64', 'worst area': 'float64', 'worst smoothness': 'float64', 'worst compactness': 'float64', 'worst concavity': 'float64', 'worst concave points': 'float64', 'worst symmetry': 'float64', 'worst fractal dimension': 'float64'}
2024-05-29 11:38:26,527 pid:50687 MainThread giskard.utils.logging_utils INFO     Predicted dataset with shape (30, 31) executed in 0:00:00.008275
Executed 'Accuracy on data slice “`mean area` >= 518.300 AND `mean area` < 664.350”' with arguments {'model': <giskard.models.sklearn.SKLearnModel object at 0x17f45eec0>, 'dataset': <giskard.datasets.base.Dataset object at 0x17f2b1db0>, 'slicing_function': <giskard.slicing.slice.QueryBasedSliceFunction object at 0x3322f31c0>, 'threshold': 0.9101398601398601}:
               Test failed
               Metric: 0.8


2024-05-29 11:38:26,538 pid:50687 MainThread giskard.datasets.base INFO     Casting dataframe columns from {'mean radius': 'float64', 'mean texture': 'float64', 'mean perimeter': 'float64', 'mean area': 'float64', 'mean smoothness': 'float64', 'mean compactness': 'float64', 'mean concavity': 'float64', 'mean concave points': 'float64', 'mean symmetry': 'float64', 'mean fractal dimension': 'float64', 'radius error': 'float64', 'texture error': 'float64', 'perimeter error': 'float64', 'area error': 'float64', 'smoothness error': 'float64', 'compactness error': 'float64', 'concavity error': 'float64', 'concave points error': 'float64', 'symmetry error': 'float64', 'fractal dimension error': 'float64', 'worst radius': 'float64', 'worst texture': 'float64', 'worst perimeter': 'float64', 'worst area': 'float64', 'worst smoothness': 'float64', 'worst compactness': 'float64', 'worst concavity': 'float64', 'worst concave points': 'float64', 'worst symmetry': 'float64', 'worst fractal dimension': 'float64'} to {'mean radius': 'float64', 'mean texture': 'float64', 'mean perimeter': 'float64', 'mean area': 'float64', 'mean smoothness': 'float64', 'mean compactness': 'float64', 'mean concavity': 'float64', 'mean concave points': 'float64', 'mean symmetry': 'float64', 'mean fractal dimension': 'float64', 'radius error': 'float64', 'texture error': 'float64', 'perimeter error': 'float64', 'area error': 'float64', 'smoothness error': 'float64', 'compactness error': 'float64', 'concavity error': 'float64', 'concave points error': 'float64', 'symmetry error': 'float64', 'fractal dimension error': 'float64', 'worst radius': 'float64', 'worst texture': 'float64', 'worst perimeter': 'float64', 'worst area': 'float64', 'worst smoothness': 'float64', 'worst compactness': 'float64', 'worst concavity': 'float64', 'worst concave points': 'float64', 'worst symmetry': 'float64', 'worst fractal dimension': 'float64'}
2024-05-29 11:38:26,541 pid:50687 MainThread giskard.utils.logging_utils INFO     Predicted dataset with shape (30, 31) executed in 0:00:00.010876
Executed 'Accuracy on data slice “`mean radius` >= 13.310 AND `mean radius` < 15.005”' with arguments {'model': <giskard.models.sklearn.SKLearnModel object at 0x17f45eec0>, 'dataset': <giskard.datasets.base.Dataset object at 0x17f2b1db0>, 'slicing_function': <giskard.slicing.slice.QueryBasedSliceFunction object at 0x332343400>, 'threshold': 0.9101398601398601}:
               Test failed
               Metric: 0.8


2024-05-29 11:38:26,562 pid:50687 MainThread giskard.datasets.base INFO     Casting dataframe columns from {'mean radius': 'float64', 'mean texture': 'float64', 'mean perimeter': 'float64', 'mean area': 'float64', 'mean smoothness': 'float64', 'mean compactness': 'float64', 'mean concavity': 'float64', 'mean concave points': 'float64', 'mean symmetry': 'float64', 'mean fractal dimension': 'float64', 'radius error': 'float64', 'texture error': 'float64', 'perimeter error': 'float64', 'area error': 'float64', 'smoothness error': 'float64', 'compactness error': 'float64', 'concavity error': 'float64', 'concave points error': 'float64', 'symmetry error': 'float64', 'fractal dimension error': 'float64', 'worst radius': 'float64', 'worst texture': 'float64', 'worst perimeter': 'float64', 'worst area': 'float64', 'worst smoothness': 'float64', 'worst compactness': 'float64', 'worst concavity': 'float64', 'worst concave points': 'float64', 'worst symmetry': 'float64', 'worst fractal dimension': 'float64'} to {'mean radius': 'float64', 'mean texture': 'float64', 'mean perimeter': 'float64', 'mean area': 'float64', 'mean smoothness': 'float64', 'mean compactness': 'float64', 'mean concavity': 'float64', 'mean concave points': 'float64', 'mean symmetry': 'float64', 'mean fractal dimension': 'float64', 'radius error': 'float64', 'texture error': 'float64', 'perimeter error': 'float64', 'area error': 'float64', 'smoothness error': 'float64', 'compactness error': 'float64', 'concavity error': 'float64', 'concave points error': 'float64', 'symmetry error': 'float64', 'fractal dimension error': 'float64', 'worst radius': 'float64', 'worst texture': 'float64', 'worst perimeter': 'float64', 'worst area': 'float64', 'worst smoothness': 'float64', 'worst compactness': 'float64', 'worst concavity': 'float64', 'worst concave points': 'float64', 'worst symmetry': 'float64', 'worst fractal dimension': 'float64'}
2024-05-29 11:38:26,577 pid:50687 MainThread giskard.utils.logging_utils INFO     Predicted dataset with shape (30, 31) executed in 0:00:00.020399
Executed 'Accuracy on data slice “`mean concave points` >= 0.048 AND `mean concave points` < 0.079”' with arguments {'model': <giskard.models.sklearn.SKLearnModel object at 0x17f45eec0>, 'dataset': <giskard.datasets.base.Dataset object at 0x17f2b1db0>, 'slicing_function': <giskard.slicing.slice.QueryBasedSliceFunction object at 0x33230c3d0>, 'threshold': 0.9101398601398601}:
               Test failed
               Metric: 0.8


2024-05-29 11:38:26,592 pid:50687 MainThread giskard.datasets.base INFO     Casting dataframe columns from {'mean radius': 'float64', 'mean texture': 'float64', 'mean perimeter': 'float64', 'mean area': 'float64', 'mean smoothness': 'float64', 'mean compactness': 'float64', 'mean concavity': 'float64', 'mean concave points': 'float64', 'mean symmetry': 'float64', 'mean fractal dimension': 'float64', 'radius error': 'float64', 'texture error': 'float64', 'perimeter error': 'float64', 'area error': 'float64', 'smoothness error': 'float64', 'compactness error': 'float64', 'concavity error': 'float64', 'concave points error': 'float64', 'symmetry error': 'float64', 'fractal dimension error': 'float64', 'worst radius': 'float64', 'worst texture': 'float64', 'worst perimeter': 'float64', 'worst area': 'float64', 'worst smoothness': 'float64', 'worst compactness': 'float64', 'worst concavity': 'float64', 'worst concave points': 'float64', 'worst symmetry': 'float64', 'worst fractal dimension': 'float64'} to {'mean radius': 'float64', 'mean texture': 'float64', 'mean perimeter': 'float64', 'mean area': 'float64', 'mean smoothness': 'float64', 'mean compactness': 'float64', 'mean concavity': 'float64', 'mean concave points': 'float64', 'mean symmetry': 'float64', 'mean fractal dimension': 'float64', 'radius error': 'float64', 'texture error': 'float64', 'perimeter error': 'float64', 'area error': 'float64', 'smoothness error': 'float64', 'compactness error': 'float64', 'concavity error': 'float64', 'concave points error': 'float64', 'symmetry error': 'float64', 'fractal dimension error': 'float64', 'worst radius': 'float64', 'worst texture': 'float64', 'worst perimeter': 'float64', 'worst area': 'float64', 'worst smoothness': 'float64', 'worst compactness': 'float64', 'worst concavity': 'float64', 'worst concave points': 'float64', 'worst symmetry': 'float64', 'worst fractal dimension': 'float64'}
2024-05-29 11:38:26,601 pid:50687 MainThread giskard.utils.logging_utils INFO     Predicted dataset with shape (30, 31) executed in 0:00:00.013636
Executed 'Accuracy on data slice “`mean concavity` >= 0.078 AND `mean concavity` < 0.140”' with arguments {'model': <giskard.models.sklearn.SKLearnModel object at 0x17f45eec0>, 'dataset': <giskard.datasets.base.Dataset object at 0x17f2b1db0>, 'slicing_function': <giskard.slicing.slice.QueryBasedSliceFunction object at 0x3322bd510>, 'threshold': 0.9101398601398601}:
               Test failed
               Metric: 0.8


2024-05-29 11:38:26,616 pid:50687 MainThread giskard.datasets.base INFO     Casting dataframe columns from {'mean radius': 'float64', 'mean texture': 'float64', 'mean perimeter': 'float64', 'mean area': 'float64', 'mean smoothness': 'float64', 'mean compactness': 'float64', 'mean concavity': 'float64', 'mean concave points': 'float64', 'mean symmetry': 'float64', 'mean fractal dimension': 'float64', 'radius error': 'float64', 'texture error': 'float64', 'perimeter error': 'float64', 'area error': 'float64', 'smoothness error': 'float64', 'compactness error': 'float64', 'concavity error': 'float64', 'concave points error': 'float64', 'symmetry error': 'float64', 'fractal dimension error': 'float64', 'worst radius': 'float64', 'worst texture': 'float64', 'worst perimeter': 'float64', 'worst area': 'float64', 'worst smoothness': 'float64', 'worst compactness': 'float64', 'worst concavity': 'float64', 'worst concave points': 'float64', 'worst symmetry': 'float64', 'worst fractal dimension': 'float64'} to {'mean radius': 'float64', 'mean texture': 'float64', 'mean perimeter': 'float64', 'mean area': 'float64', 'mean smoothness': 'float64', 'mean compactness': 'float64', 'mean concavity': 'float64', 'mean concave points': 'float64', 'mean symmetry': 'float64', 'mean fractal dimension': 'float64', 'radius error': 'float64', 'texture error': 'float64', 'perimeter error': 'float64', 'area error': 'float64', 'smoothness error': 'float64', 'compactness error': 'float64', 'concavity error': 'float64', 'concave points error': 'float64', 'symmetry error': 'float64', 'fractal dimension error': 'float64', 'worst radius': 'float64', 'worst texture': 'float64', 'worst perimeter': 'float64', 'worst area': 'float64', 'worst smoothness': 'float64', 'worst compactness': 'float64', 'worst concavity': 'float64', 'worst concave points': 'float64', 'worst symmetry': 'float64', 'worst fractal dimension': 'float64'}
2024-05-29 11:38:26,625 pid:50687 MainThread giskard.utils.logging_utils INFO     Predicted dataset with shape (33, 31) executed in 0:00:00.013964
Executed 'Accuracy on data slice “`worst area` >= 610.200 AND `worst area` < 828.500”' with arguments {'model': <giskard.models.sklearn.SKLearnModel object at 0x17f45eec0>, 'dataset': <giskard.datasets.base.Dataset object at 0x17f2b1db0>, 'slicing_function': <giskard.slicing.slice.QueryBasedSliceFunction object at 0x332243d00>, 'threshold': 0.9101398601398601}:
               Test failed
               Metric: 0.82


2024-05-29 11:38:26,637 pid:50687 MainThread giskard.datasets.base INFO     Casting dataframe columns from {'mean radius': 'float64', 'mean texture': 'float64', 'mean perimeter': 'float64', 'mean area': 'float64', 'mean smoothness': 'float64', 'mean compactness': 'float64', 'mean concavity': 'float64', 'mean concave points': 'float64', 'mean symmetry': 'float64', 'mean fractal dimension': 'float64', 'radius error': 'float64', 'texture error': 'float64', 'perimeter error': 'float64', 'area error': 'float64', 'smoothness error': 'float64', 'compactness error': 'float64', 'concavity error': 'float64', 'concave points error': 'float64', 'symmetry error': 'float64', 'fractal dimension error': 'float64', 'worst radius': 'float64', 'worst texture': 'float64', 'worst perimeter': 'float64', 'worst area': 'float64', 'worst smoothness': 'float64', 'worst compactness': 'float64', 'worst concavity': 'float64', 'worst concave points': 'float64', 'worst symmetry': 'float64', 'worst fractal dimension': 'float64'} to {'mean radius': 'float64', 'mean texture': 'float64', 'mean perimeter': 'float64', 'mean area': 'float64', 'mean smoothness': 'float64', 'mean compactness': 'float64', 'mean concavity': 'float64', 'mean concave points': 'float64', 'mean symmetry': 'float64', 'mean fractal dimension': 'float64', 'radius error': 'float64', 'texture error': 'float64', 'perimeter error': 'float64', 'area error': 'float64', 'smoothness error': 'float64', 'compactness error': 'float64', 'concavity error': 'float64', 'concave points error': 'float64', 'symmetry error': 'float64', 'fractal dimension error': 'float64', 'worst radius': 'float64', 'worst texture': 'float64', 'worst perimeter': 'float64', 'worst area': 'float64', 'worst smoothness': 'float64', 'worst compactness': 'float64', 'worst concavity': 'float64', 'worst concave points': 'float64', 'worst symmetry': 'float64', 'worst fractal dimension': 'float64'}
2024-05-29 11:38:26,649 pid:50687 MainThread giskard.utils.logging_utils INFO     Predicted dataset with shape (30, 31) executed in 0:00:00.019259
Executed 'Accuracy on data slice “`fractal dimension error` < 0.003 AND `fractal dimension error` >= 0.002”' with arguments {'model': <giskard.models.sklearn.SKLearnModel object at 0x17f45eec0>, 'dataset': <giskard.datasets.base.Dataset object at 0x17f2b1db0>, 'slicing_function': <giskard.slicing.slice.QueryBasedSliceFunction object at 0x332240a90>, 'threshold': 0.9101398601398601}:
               Test failed
               Metric: 0.83


2024-05-29 11:38:26,662 pid:50687 MainThread giskard.datasets.base INFO     Casting dataframe columns from {'mean radius': 'float64', 'mean texture': 'float64', 'mean perimeter': 'float64', 'mean area': 'float64', 'mean smoothness': 'float64', 'mean compactness': 'float64', 'mean concavity': 'float64', 'mean concave points': 'float64', 'mean symmetry': 'float64', 'mean fractal dimension': 'float64', 'radius error': 'float64', 'texture error': 'float64', 'perimeter error': 'float64', 'area error': 'float64', 'smoothness error': 'float64', 'compactness error': 'float64', 'concavity error': 'float64', 'concave points error': 'float64', 'symmetry error': 'float64', 'fractal dimension error': 'float64', 'worst radius': 'float64', 'worst texture': 'float64', 'worst perimeter': 'float64', 'worst area': 'float64', 'worst smoothness': 'float64', 'worst compactness': 'float64', 'worst concavity': 'float64', 'worst concave points': 'float64', 'worst symmetry': 'float64', 'worst fractal dimension': 'float64'} to {'mean radius': 'float64', 'mean texture': 'float64', 'mean perimeter': 'float64', 'mean area': 'float64', 'mean smoothness': 'float64', 'mean compactness': 'float64', 'mean concavity': 'float64', 'mean concave points': 'float64', 'mean symmetry': 'float64', 'mean fractal dimension': 'float64', 'radius error': 'float64', 'texture error': 'float64', 'perimeter error': 'float64', 'area error': 'float64', 'smoothness error': 'float64', 'compactness error': 'float64', 'concavity error': 'float64', 'concave points error': 'float64', 'symmetry error': 'float64', 'fractal dimension error': 'float64', 'worst radius': 'float64', 'worst texture': 'float64', 'worst perimeter': 'float64', 'worst area': 'float64', 'worst smoothness': 'float64', 'worst compactness': 'float64', 'worst concavity': 'float64', 'worst concave points': 'float64', 'worst symmetry': 'float64', 'worst fractal dimension': 'float64'}
2024-05-29 11:38:26,667 pid:50687 MainThread giskard.utils.logging_utils INFO     Predicted dataset with shape (30, 31) executed in 0:00:00.010570
Executed 'Accuracy on data slice “`worst concavity` >= 0.277 AND `worst concavity` < 0.415”' with arguments {'model': <giskard.models.sklearn.SKLearnModel object at 0x17f45eec0>, 'dataset': <giskard.datasets.base.Dataset object at 0x17f2b1db0>, 'slicing_function': <giskard.slicing.slice.QueryBasedSliceFunction object at 0x3322ac820>, 'threshold': 0.9101398601398601}:
               Test failed
               Metric: 0.83


2024-05-29 11:38:26,667 pid:50687 Thread-54 (_track) urllib3.connectionpool WARNING  Connection pool is full, discarding connection: api.mixpanel.com. Connection pool size: 10
2024-05-29 11:38:26,668 pid:50687 Thread-47 (_track) urllib3.connectionpool WARNING  Connection pool is full, discarding connection: api.mixpanel.com. Connection pool size: 10
2024-05-29 11:38:26,674 pid:50687 Thread-48 (_track) urllib3.connectionpool WARNING  Connection pool is full, discarding connection: api.mixpanel.com. Connection pool size: 10
2024-05-29 11:38:26,675 pid:50687 MainThread giskard.datasets.base INFO     Casting dataframe columns from {'mean radius': 'float64', 'mean texture': 'float64', 'mean perimeter': 'float64', 'mean area': 'float64', 'mean smoothness': 'float64', 'mean compactness': 'float64', 'mean concavity': 'float64', 'mean concave points': 'float64', 'mean symmetry': 'float64', 'mean fractal dimension': 'float64', 'radius error': 'float64', 'texture error': 'float64', 'perimeter error': 'float64', 'area error': 'float64', 'smoothness error': 'float64', 'compactness error': 'float64', 'concavity error': 'float64', 'concave points error': 'float64', 'symmetry error': 'float64', 'fractal dimension error': 'float64', 'worst radius': 'float64', 'worst texture': 'float64', 'worst perimeter': 'float64', 'worst area': 'float64', 'worst smoothness': 'float64', 'worst compactness': 'float64', 'worst concavity': 'float64', 'worst concave points': 'float64', 'worst symmetry': 'float64', 'worst fractal dimension': 'float64'} to {'mean radius': 'float64', 'mean texture': 'float64', 'mean perimeter': 'float64', 'mean area': 'float64', 'mean smoothness': 'float64', 'mean compactness': 'float64', 'mean concavity': 'float64', 'mean concave points': 'float64', 'mean symmetry': 'float64', 'mean fractal dimension': 'float64', 'radius error': 'float64', 'texture error': 'float64', 'perimeter error': 'float64', 'area error': 'float64', 'smoothness error': 'float64', 'compactness error': 'float64', 'concavity error': 'float64', 'concave points error': 'float64', 'symmetry error': 'float64', 'fractal dimension error': 'float64', 'worst radius': 'float64', 'worst texture': 'float64', 'worst perimeter': 'float64', 'worst area': 'float64', 'worst smoothness': 'float64', 'worst compactness': 'float64', 'worst concavity': 'float64', 'worst concave points': 'float64', 'worst symmetry': 'float64', 'worst fractal dimension': 'float64'}
2024-05-29 11:38:26,681 pid:50687 Thread-50 (_track) urllib3.connectionpool WARNING  Connection pool is full, discarding connection: api.mixpanel.com. Connection pool size: 10
2024-05-29 11:38:26,684 pid:50687 MainThread giskard.utils.logging_utils INFO     Predicted dataset with shape (32, 31) executed in 0:00:00.013371
Executed 'Accuracy on data slice “`compactness error` >= 0.020 AND `compactness error` < 0.030”' with arguments {'model': <giskard.models.sklearn.SKLearnModel object at 0x17f45eec0>, 'dataset': <giskard.datasets.base.Dataset object at 0x17f2b1db0>, 'slicing_function': <giskard.slicing.slice.QueryBasedSliceFunction object at 0x3322bfdc0>, 'threshold': 0.9101398601398601}:
               Test failed
               Metric: 0.84


2024-05-29 11:38:26,693 pid:50687 MainThread giskard.datasets.base INFO     Casting dataframe columns from {'mean radius': 'float64', 'mean texture': 'float64', 'mean perimeter': 'float64', 'mean area': 'float64', 'mean smoothness': 'float64', 'mean compactness': 'float64', 'mean concavity': 'float64', 'mean concave points': 'float64', 'mean symmetry': 'float64', 'mean fractal dimension': 'float64', 'radius error': 'float64', 'texture error': 'float64', 'perimeter error': 'float64', 'area error': 'float64', 'smoothness error': 'float64', 'compactness error': 'float64', 'concavity error': 'float64', 'concave points error': 'float64', 'symmetry error': 'float64', 'fractal dimension error': 'float64', 'worst radius': 'float64', 'worst texture': 'float64', 'worst perimeter': 'float64', 'worst area': 'float64', 'worst smoothness': 'float64', 'worst compactness': 'float64', 'worst concavity': 'float64', 'worst concave points': 'float64', 'worst symmetry': 'float64', 'worst fractal dimension': 'float64'} to {'mean radius': 'float64', 'mean texture': 'float64', 'mean perimeter': 'float64', 'mean area': 'float64', 'mean smoothness': 'float64', 'mean compactness': 'float64', 'mean concavity': 'float64', 'mean concave points': 'float64', 'mean symmetry': 'float64', 'mean fractal dimension': 'float64', 'radius error': 'float64', 'texture error': 'float64', 'perimeter error': 'float64', 'area error': 'float64', 'smoothness error': 'float64', 'compactness error': 'float64', 'concavity error': 'float64', 'concave points error': 'float64', 'symmetry error': 'float64', 'fractal dimension error': 'float64', 'worst radius': 'float64', 'worst texture': 'float64', 'worst perimeter': 'float64', 'worst area': 'float64', 'worst smoothness': 'float64', 'worst compactness': 'float64', 'worst concavity': 'float64', 'worst concave points': 'float64', 'worst symmetry': 'float64', 'worst fractal dimension': 'float64'}
2024-05-29 11:38:26,697 pid:50687 MainThread giskard.utils.logging_utils INFO     Predicted dataset with shape (32, 31) executed in 0:00:00.007443
Executed 'Accuracy on data slice “`mean compactness` >= 0.099 AND `mean compactness` < 0.128”' with arguments {'model': <giskard.models.sklearn.SKLearnModel object at 0x17f45eec0>, 'dataset': <giskard.datasets.base.Dataset object at 0x17f2b1db0>, 'slicing_function': <giskard.slicing.slice.QueryBasedSliceFunction object at 0x3322bc5b0>, 'threshold': 0.9101398601398601}:
               Test failed
               Metric: 0.84


2024-05-29 11:38:26,707 pid:50687 MainThread giskard.datasets.base INFO     Casting dataframe columns from {'mean radius': 'float64', 'mean texture': 'float64', 'mean perimeter': 'float64', 'mean area': 'float64', 'mean smoothness': 'float64', 'mean compactness': 'float64', 'mean concavity': 'float64', 'mean concave points': 'float64', 'mean symmetry': 'float64', 'mean fractal dimension': 'float64', 'radius error': 'float64', 'texture error': 'float64', 'perimeter error': 'float64', 'area error': 'float64', 'smoothness error': 'float64', 'compactness error': 'float64', 'concavity error': 'float64', 'concave points error': 'float64', 'symmetry error': 'float64', 'fractal dimension error': 'float64', 'worst radius': 'float64', 'worst texture': 'float64', 'worst perimeter': 'float64', 'worst area': 'float64', 'worst smoothness': 'float64', 'worst compactness': 'float64', 'worst concavity': 'float64', 'worst concave points': 'float64', 'worst symmetry': 'float64', 'worst fractal dimension': 'float64'} to {'mean radius': 'float64', 'mean texture': 'float64', 'mean perimeter': 'float64', 'mean area': 'float64', 'mean smoothness': 'float64', 'mean compactness': 'float64', 'mean concavity': 'float64', 'mean concave points': 'float64', 'mean symmetry': 'float64', 'mean fractal dimension': 'float64', 'radius error': 'float64', 'texture error': 'float64', 'perimeter error': 'float64', 'area error': 'float64', 'smoothness error': 'float64', 'compactness error': 'float64', 'concavity error': 'float64', 'concave points error': 'float64', 'symmetry error': 'float64', 'fractal dimension error': 'float64', 'worst radius': 'float64', 'worst texture': 'float64', 'worst perimeter': 'float64', 'worst area': 'float64', 'worst smoothness': 'float64', 'worst compactness': 'float64', 'worst concavity': 'float64', 'worst concave points': 'float64', 'worst symmetry': 'float64', 'worst fractal dimension': 'float64'}
2024-05-29 11:38:26,710 pid:50687 MainThread giskard.utils.logging_utils INFO     Predicted dataset with shape (51, 31) executed in 0:00:00.007322
Executed 'Recall on data slice “`concave points error` >= 0.009 AND `concave points error` < 0.015”' with arguments {'model': <giskard.models.sklearn.SKLearnModel object at 0x17f45eec0>, 'dataset': <giskard.datasets.base.Dataset object at 0x17f2b1db0>, 'slicing_function': <giskard.slicing.slice.QueryBasedSliceFunction object at 0x33230de40>, 'threshold': 0.9179775280898876}:
               Test failed
               Metric: 0.88


2024-05-29 11:38:26,720 pid:50687 MainThread giskard.datasets.base INFO     Casting dataframe columns from {'mean radius': 'float64', 'mean texture': 'float64', 'mean perimeter': 'float64', 'mean area': 'float64', 'mean smoothness': 'float64', 'mean compactness': 'float64', 'mean concavity': 'float64', 'mean concave points': 'float64', 'mean symmetry': 'float64', 'mean fractal dimension': 'float64', 'radius error': 'float64', 'texture error': 'float64', 'perimeter error': 'float64', 'area error': 'float64', 'smoothness error': 'float64', 'compactness error': 'float64', 'concavity error': 'float64', 'concave points error': 'float64', 'symmetry error': 'float64', 'fractal dimension error': 'float64', 'worst radius': 'float64', 'worst texture': 'float64', 'worst perimeter': 'float64', 'worst area': 'float64', 'worst smoothness': 'float64', 'worst compactness': 'float64', 'worst concavity': 'float64', 'worst concave points': 'float64', 'worst symmetry': 'float64', 'worst fractal dimension': 'float64'} to {'mean radius': 'float64', 'mean texture': 'float64', 'mean perimeter': 'float64', 'mean area': 'float64', 'mean smoothness': 'float64', 'mean compactness': 'float64', 'mean concavity': 'float64', 'mean concave points': 'float64', 'mean symmetry': 'float64', 'mean fractal dimension': 'float64', 'radius error': 'float64', 'texture error': 'float64', 'perimeter error': 'float64', 'area error': 'float64', 'smoothness error': 'float64', 'compactness error': 'float64', 'concavity error': 'float64', 'concave points error': 'float64', 'symmetry error': 'float64', 'fractal dimension error': 'float64', 'worst radius': 'float64', 'worst texture': 'float64', 'worst perimeter': 'float64', 'worst area': 'float64', 'worst smoothness': 'float64', 'worst compactness': 'float64', 'worst concavity': 'float64', 'worst concave points': 'float64', 'worst symmetry': 'float64', 'worst fractal dimension': 'float64'}
2024-05-29 11:38:26,723 pid:50687 MainThread giskard.utils.logging_utils INFO     Predicted dataset with shape (35, 31) executed in 0:00:00.008618
Executed 'Accuracy on data slice “`perimeter error` < 2.151 AND `perimeter error` >= 1.505”' with arguments {'model': <giskard.models.sklearn.SKLearnModel object at 0x17f45eec0>, 'dataset': <giskard.datasets.base.Dataset object at 0x17f2b1db0>, 'slicing_function': <giskard.slicing.slice.QueryBasedSliceFunction object at 0x3322f3310>, 'threshold': 0.9101398601398601}:
               Test failed
               Metric: 0.89


2024-05-29 11:38:26,732 pid:50687 MainThread giskard.datasets.base INFO     Casting dataframe columns from {'mean radius': 'float64', 'mean texture': 'float64', 'mean perimeter': 'float64', 'mean area': 'float64', 'mean smoothness': 'float64', 'mean compactness': 'float64', 'mean concavity': 'float64', 'mean concave points': 'float64', 'mean symmetry': 'float64', 'mean fractal dimension': 'float64', 'radius error': 'float64', 'texture error': 'float64', 'perimeter error': 'float64', 'area error': 'float64', 'smoothness error': 'float64', 'compactness error': 'float64', 'concavity error': 'float64', 'concave points error': 'float64', 'symmetry error': 'float64', 'fractal dimension error': 'float64', 'worst radius': 'float64', 'worst texture': 'float64', 'worst perimeter': 'float64', 'worst area': 'float64', 'worst smoothness': 'float64', 'worst compactness': 'float64', 'worst concavity': 'float64', 'worst concave points': 'float64', 'worst symmetry': 'float64', 'worst fractal dimension': 'float64'} to {'mean radius': 'float64', 'mean texture': 'float64', 'mean perimeter': 'float64', 'mean area': 'float64', 'mean smoothness': 'float64', 'mean compactness': 'float64', 'mean concavity': 'float64', 'mean concave points': 'float64', 'mean symmetry': 'float64', 'mean fractal dimension': 'float64', 'radius error': 'float64', 'texture error': 'float64', 'perimeter error': 'float64', 'area error': 'float64', 'smoothness error': 'float64', 'compactness error': 'float64', 'concavity error': 'float64', 'concave points error': 'float64', 'symmetry error': 'float64', 'fractal dimension error': 'float64', 'worst radius': 'float64', 'worst texture': 'float64', 'worst perimeter': 'float64', 'worst area': 'float64', 'worst smoothness': 'float64', 'worst compactness': 'float64', 'worst concavity': 'float64', 'worst concave points': 'float64', 'worst symmetry': 'float64', 'worst fractal dimension': 'float64'}
2024-05-29 11:38:26,736 pid:50687 MainThread giskard.utils.logging_utils INFO     Predicted dataset with shape (45, 31) executed in 0:00:00.009279
Executed 'Accuracy on data slice “`area error` < 40.745 AND `area error` >= 19.215”' with arguments {'model': <giskard.models.sklearn.SKLearnModel object at 0x17f45eec0>, 'dataset': <giskard.datasets.base.Dataset object at 0x17f2b1db0>, 'slicing_function': <giskard.slicing.slice.QueryBasedSliceFunction object at 0x33230fc40>, 'threshold': 0.9101398601398601}:
               Test failed
               Metric: 0.89


2024-05-29 11:38:26,737 pid:50687 MainThread giskard.core.suite INFO     Executed test suite 'My first test suite'
2024-05-29 11:38:26,738 pid:50687 MainThread giskard.core.suite INFO     result: failed
2024-05-29 11:38:26,738 pid:50687 MainThread giskard.core.suite INFO     Nominal association (Theil's U) on data slice “`worst concave points` >= 0.144” ({'model': <giskard.models.sklearn.SKLearnModel object at 0x17f45eec0>, 'dataset': <giskard.datasets.base.Dataset object at 0x17f2b1db0>, 'slicing_function': <giskard.slicing.slice.QueryBasedSliceFunction object at 0x3322aeaa0>, 'threshold': 0.5}): {failed, metric=0.7745014140794255}
2024-05-29 11:38:26,738 pid:50687 MainThread giskard.core.suite INFO     Nominal association (Theil's U) on data slice “`worst radius` >= 17.420” ({'model': <giskard.models.sklearn.SKLearnModel object at 0x17f45eec0>, 'dataset': <giskard.datasets.base.Dataset object at 0x17f2b1db0>, 'slicing_function': <giskard.slicing.slice.QueryBasedSliceFunction object at 0x3322add50>, 'threshold': 0.5}): {failed, metric=0.7639160128151123}
2024-05-29 11:38:26,739 pid:50687 MainThread giskard.core.suite INFO     Nominal association (Theil's U) on data slice “`worst perimeter` >= 110.950” ({'model': <giskard.models.sklearn.SKLearnModel object at 0x17f45eec0>, 'dataset': <giskard.datasets.base.Dataset object at 0x17f2b1db0>, 'slicing_function': <giskard.slicing.slice.QueryBasedSliceFunction object at 0x3322ae5c0>, 'threshold': 0.5}): {failed, metric=0.7427987555404146}
2024-05-29 11:38:26,740 pid:50687 MainThread giskard.core.suite INFO     Nominal association (Theil's U) on data slice “`worst area` >= 885.950” ({'model': <giskard.models.sklearn.SKLearnModel object at 0x17f45eec0>, 'dataset': <giskard.datasets.base.Dataset object at 0x17f2b1db0>, 'slicing_function': <giskard.slicing.slice.QueryBasedSliceFunction object at 0x3322acaf0>, 'threshold': 0.5}): {failed, metric=0.7131111241996619}
2024-05-29 11:38:26,740 pid:50687 MainThread giskard.core.suite INFO     Nominal association (Theil's U) on data slice “`mean concave points` >= 0.053” ({'model': <giskard.models.sklearn.SKLearnModel object at 0x17f45eec0>, 'dataset': <giskard.datasets.base.Dataset object at 0x17f2b1db0>, 'slicing_function': <giskard.slicing.slice.QueryBasedSliceFunction object at 0x3322f0730>, 'threshold': 0.5}): {failed, metric=0.6377395426892722}
2024-05-29 11:38:26,741 pid:50687 MainThread giskard.core.suite INFO     Nominal association (Theil's U) on data slice “`mean perimeter` >= 96.380” ({'model': <giskard.models.sklearn.SKLearnModel object at 0x17f45eec0>, 'dataset': <giskard.datasets.base.Dataset object at 0x17f2b1db0>, 'slicing_function': <giskard.slicing.slice.QueryBasedSliceFunction object at 0x3322f3fa0>, 'threshold': 0.5}): {failed, metric=0.6227293846650519}
2024-05-29 11:38:26,741 pid:50687 MainThread giskard.core.suite INFO     Nominal association (Theil's U) on data slice “`mean area` >= 697.300” ({'model': <giskard.models.sklearn.SKLearnModel object at 0x17f45eec0>, 'dataset': <giskard.datasets.base.Dataset object at 0x17f2b1db0>, 'slicing_function': <giskard.slicing.slice.QueryBasedSliceFunction object at 0x3322f0df0>, 'threshold': 0.5}): {failed, metric=0.6087180717056299}
2024-05-29 11:38:26,743 pid:50687 MainThread giskard.core.suite INFO     Nominal association (Theil's U) on data slice “`mean radius` >= 15.060” ({'model': <giskard.models.sklearn.SKLearnModel object at 0x17f45eec0>, 'dataset': <giskard.datasets.base.Dataset object at 0x17f2b1db0>, 'slicing_function': <giskard.slicing.slice.QueryBasedSliceFunction object at 0x33230d4e0>, 'threshold': 0.5}): {failed, metric=0.6087180717056299}
2024-05-29 11:38:26,744 pid:50687 MainThread giskard.core.suite INFO     Nominal association (Theil's U) on data slice “`area error` >= 39.690” ({'model': <giskard.models.sklearn.SKLearnModel object at 0x17f45eec0>, 'dataset': <giskard.datasets.base.Dataset object at 0x17f2b1db0>, 'slicing_function': <giskard.slicing.slice.QueryBasedSliceFunction object at 0x3322afee0>, 'threshold': 0.5}): {failed, metric=0.5166070702259753}
2024-05-29 11:38:26,745 pid:50687 MainThread giskard.core.suite INFO     Accuracy on data slice “`worst radius` >= 14.765 AND `worst radius` < 17.625” ({'model': <giskard.models.sklearn.SKLearnModel object at 0x17f45eec0>, 'dataset': <giskard.datasets.base.Dataset object at 0x17f2b1db0>, 'slicing_function': <giskard.slicing.slice.QueryBasedSliceFunction object at 0x3322acee0>, 'threshold': 0.9101398601398601}): {failed, metric=0.8}
2024-05-29 11:38:26,746 pid:50687 MainThread giskard.core.suite INFO     Accuracy on data slice “`worst perimeter` >= 96.625 AND `worst perimeter` < 122.350” ({'model': <giskard.models.sklearn.SKLearnModel object at 0x17f45eec0>, 'dataset': <giskard.datasets.base.Dataset object at 0x17f2b1db0>, 'slicing_function': <giskard.slicing.slice.QueryBasedSliceFunction object at 0x33229a260>, 'threshold': 0.9101398601398601}): {failed, metric=0.8}
2024-05-29 11:38:26,747 pid:50687 MainThread giskard.core.suite INFO     Accuracy on data slice “`mean perimeter` >= 86.140 AND `mean perimeter` < 98.085” ({'model': <giskard.models.sklearn.SKLearnModel object at 0x17f45eec0>, 'dataset': <giskard.datasets.base.Dataset object at 0x17f2b1db0>, 'slicing_function': <giskard.slicing.slice.QueryBasedSliceFunction object at 0x3322f0f10>, 'threshold': 0.9101398601398601}): {failed, metric=0.8}
2024-05-29 11:38:26,748 pid:50687 MainThread giskard.core.suite INFO     Accuracy on data slice “`mean area` >= 518.300 AND `mean area` < 664.350” ({'model': <giskard.models.sklearn.SKLearnModel object at 0x17f45eec0>, 'dataset': <giskard.datasets.base.Dataset object at 0x17f2b1db0>, 'slicing_function': <giskard.slicing.slice.QueryBasedSliceFunction object at 0x3322f31c0>, 'threshold': 0.9101398601398601}): {failed, metric=0.8}
2024-05-29 11:38:26,748 pid:50687 MainThread giskard.core.suite INFO     Accuracy on data slice “`mean radius` >= 13.310 AND `mean radius` < 15.005” ({'model': <giskard.models.sklearn.SKLearnModel object at 0x17f45eec0>, 'dataset': <giskard.datasets.base.Dataset object at 0x17f2b1db0>, 'slicing_function': <giskard.slicing.slice.QueryBasedSliceFunction object at 0x332343400>, 'threshold': 0.9101398601398601}): {failed, metric=0.8}
2024-05-29 11:38:26,749 pid:50687 MainThread giskard.core.suite INFO     Accuracy on data slice “`mean concave points` >= 0.048 AND `mean concave points` < 0.079” ({'model': <giskard.models.sklearn.SKLearnModel object at 0x17f45eec0>, 'dataset': <giskard.datasets.base.Dataset object at 0x17f2b1db0>, 'slicing_function': <giskard.slicing.slice.QueryBasedSliceFunction object at 0x33230c3d0>, 'threshold': 0.9101398601398601}): {failed, metric=0.8}
2024-05-29 11:38:26,749 pid:50687 MainThread giskard.core.suite INFO     Accuracy on data slice “`mean concavity` >= 0.078 AND `mean concavity` < 0.140” ({'model': <giskard.models.sklearn.SKLearnModel object at 0x17f45eec0>, 'dataset': <giskard.datasets.base.Dataset object at 0x17f2b1db0>, 'slicing_function': <giskard.slicing.slice.QueryBasedSliceFunction object at 0x3322bd510>, 'threshold': 0.9101398601398601}): {failed, metric=0.8}
2024-05-29 11:38:26,752 pid:50687 MainThread giskard.core.suite INFO     Accuracy on data slice “`worst area` >= 610.200 AND `worst area` < 828.500” ({'model': <giskard.models.sklearn.SKLearnModel object at 0x17f45eec0>, 'dataset': <giskard.datasets.base.Dataset object at 0x17f2b1db0>, 'slicing_function': <giskard.slicing.slice.QueryBasedSliceFunction object at 0x332243d00>, 'threshold': 0.9101398601398601}): {failed, metric=0.8181818181818182}
2024-05-29 11:38:26,752 pid:50687 MainThread giskard.core.suite INFO     Accuracy on data slice “`fractal dimension error` < 0.003 AND `fractal dimension error` >= 0.002” ({'model': <giskard.models.sklearn.SKLearnModel object at 0x17f45eec0>, 'dataset': <giskard.datasets.base.Dataset object at 0x17f2b1db0>, 'slicing_function': <giskard.slicing.slice.QueryBasedSliceFunction object at 0x332240a90>, 'threshold': 0.9101398601398601}): {failed, metric=0.8333333333333334}
2024-05-29 11:38:26,752 pid:50687 MainThread giskard.core.suite INFO     Accuracy on data slice “`worst concavity` >= 0.277 AND `worst concavity` < 0.415” ({'model': <giskard.models.sklearn.SKLearnModel object at 0x17f45eec0>, 'dataset': <giskard.datasets.base.Dataset object at 0x17f2b1db0>, 'slicing_function': <giskard.slicing.slice.QueryBasedSliceFunction object at 0x3322ac820>, 'threshold': 0.9101398601398601}): {failed, metric=0.8333333333333334}
2024-05-29 11:38:26,753 pid:50687 MainThread giskard.core.suite INFO     Accuracy on data slice “`compactness error` >= 0.020 AND `compactness error` < 0.030” ({'model': <giskard.models.sklearn.SKLearnModel object at 0x17f45eec0>, 'dataset': <giskard.datasets.base.Dataset object at 0x17f2b1db0>, 'slicing_function': <giskard.slicing.slice.QueryBasedSliceFunction object at 0x3322bfdc0>, 'threshold': 0.9101398601398601}): {failed, metric=0.84375}
2024-05-29 11:38:26,753 pid:50687 MainThread giskard.core.suite INFO     Accuracy on data slice “`mean compactness` >= 0.099 AND `mean compactness` < 0.128” ({'model': <giskard.models.sklearn.SKLearnModel object at 0x17f45eec0>, 'dataset': <giskard.datasets.base.Dataset object at 0x17f2b1db0>, 'slicing_function': <giskard.slicing.slice.QueryBasedSliceFunction object at 0x3322bc5b0>, 'threshold': 0.9101398601398601}): {failed, metric=0.84375}
2024-05-29 11:38:26,753 pid:50687 MainThread giskard.core.suite INFO     Recall on data slice “`concave points error` >= 0.009 AND `concave points error` < 0.015” ({'model': <giskard.models.sklearn.SKLearnModel object at 0x17f45eec0>, 'dataset': <giskard.datasets.base.Dataset object at 0x17f2b1db0>, 'slicing_function': <giskard.slicing.slice.QueryBasedSliceFunction object at 0x33230de40>, 'threshold': 0.9179775280898876}): {failed, metric=0.875}
2024-05-29 11:38:26,754 pid:50687 MainThread giskard.core.suite INFO     Accuracy on data slice “`perimeter error` < 2.151 AND `perimeter error` >= 1.505” ({'model': <giskard.models.sklearn.SKLearnModel object at 0x17f45eec0>, 'dataset': <giskard.datasets.base.Dataset object at 0x17f2b1db0>, 'slicing_function': <giskard.slicing.slice.QueryBasedSliceFunction object at 0x3322f3310>, 'threshold': 0.9101398601398601}): {failed, metric=0.8857142857142857}
2024-05-29 11:38:26,754 pid:50687 MainThread giskard.core.suite INFO     Accuracy on data slice “`area error` < 40.745 AND `area error` >= 19.215” ({'model': <giskard.models.sklearn.SKLearnModel object at 0x17f45eec0>, 'dataset': <giskard.datasets.base.Dataset object at 0x17f2b1db0>, 'slicing_function': <giskard.slicing.slice.QueryBasedSliceFunction object at 0x33230fc40>, 'threshold': 0.9101398601398601}): {failed, metric=0.8888888888888888}

Test suite failed.

Test Nominal association (Theil's U) on data slice “`worst concave points` >= 0.144”

Measured Metric = 0.7745 Failed

model breast_cancer_xgboost

dataset breast_cancer

slicing_function `worst concave points` >= 0.144

threshold 0.5

Test Nominal association (Theil's U) on data slice “`worst radius` >= 17.420”

Measured Metric = 0.76392 Failed

model breast_cancer_xgboost

dataset breast_cancer

slicing_function `worst radius` >= 17.420

threshold 0.5

Test Nominal association (Theil's U) on data slice “`worst perimeter` >= 110.950”

Measured Metric = 0.7428 Failed

model breast_cancer_xgboost

dataset breast_cancer

slicing_function `worst perimeter` >= 110.950

threshold 0.5

Test Nominal association (Theil's U) on data slice “`worst area` >= 885.950”

Measured Metric = 0.71311 Failed

model breast_cancer_xgboost

dataset breast_cancer

slicing_function `worst area` >= 885.950

threshold 0.5

Test Nominal association (Theil's U) on data slice “`mean concave points` >= 0.053”

Measured Metric = 0.63774 Failed

model breast_cancer_xgboost

dataset breast_cancer

slicing_function `mean concave points` >= 0.053

threshold 0.5

Test Nominal association (Theil's U) on data slice “`mean perimeter` >= 96.380”

Measured Metric = 0.62273 Failed

model breast_cancer_xgboost

dataset breast_cancer

slicing_function `mean perimeter` >= 96.380

threshold 0.5

Test Nominal association (Theil's U) on data slice “`mean area` >= 697.300”

Measured Metric = 0.60872 Failed

model breast_cancer_xgboost

dataset breast_cancer

slicing_function `mean area` >= 697.300

threshold 0.5

Test Nominal association (Theil's U) on data slice “`mean radius` >= 15.060”

Measured Metric = 0.60872 Failed

model breast_cancer_xgboost

dataset breast_cancer

slicing_function `mean radius` >= 15.060

threshold 0.5

Test Nominal association (Theil's U) on data slice “`area error` >= 39.690”

Measured Metric = 0.51661 Failed

model breast_cancer_xgboost

dataset breast_cancer

slicing_function `area error` >= 39.690

threshold 0.5

Test Accuracy on data slice “`worst radius` >= 14.765 AND `worst radius` < 17.625”

Measured Metric = 0.8 Failed

model breast_cancer_xgboost

dataset breast_cancer

slicing_function `worst radius` >= 14.765 AND `worst radius` < 17.625

threshold 0.9101398601398601

Test Accuracy on data slice “`worst perimeter` >= 96.625 AND `worst perimeter` < 122.350”

Measured Metric = 0.8 Failed

model breast_cancer_xgboost

dataset breast_cancer

slicing_function `worst perimeter` >= 96.625 AND `worst perimeter` < 122.350

threshold 0.9101398601398601

Test Accuracy on data slice “`mean perimeter` >= 86.140 AND `mean perimeter` < 98.085”

Measured Metric = 0.8 Failed

model breast_cancer_xgboost

dataset breast_cancer

slicing_function `mean perimeter` >= 86.140 AND `mean perimeter` < 98.085

threshold 0.9101398601398601

Test Accuracy on data slice “`mean area` >= 518.300 AND `mean area` < 664.350”

Measured Metric = 0.8 Failed

model breast_cancer_xgboost

dataset breast_cancer

slicing_function `mean area` >= 518.300 AND `mean area` < 664.350

threshold 0.9101398601398601

Test Accuracy on data slice “`mean radius` >= 13.310 AND `mean radius` < 15.005”

Measured Metric = 0.8 Failed

model breast_cancer_xgboost

dataset breast_cancer

slicing_function `mean radius` >= 13.310 AND `mean radius` < 15.005

threshold 0.9101398601398601

Test Accuracy on data slice “`mean concave points` >= 0.048 AND `mean concave points` < 0.079”

Measured Metric = 0.8 Failed

model breast_cancer_xgboost

dataset breast_cancer

slicing_function `mean concave points` >= 0.048 AND `mean concave points` < 0.079

threshold 0.9101398601398601

Test Accuracy on data slice “`mean concavity` >= 0.078 AND `mean concavity` < 0.140”

Measured Metric = 0.8 Failed

model breast_cancer_xgboost

dataset breast_cancer

slicing_function `mean concavity` >= 0.078 AND `mean concavity` < 0.140

threshold 0.9101398601398601

Test Accuracy on data slice “`worst area` >= 610.200 AND `worst area` < 828.500”

Measured Metric = 0.81818 Failed

model breast_cancer_xgboost

dataset breast_cancer

slicing_function `worst area` >= 610.200 AND `worst area` < 828.500

threshold 0.9101398601398601

Test Accuracy on data slice “`fractal dimension error` < 0.003 AND `fractal dimension error` >= 0.002”

Measured Metric = 0.83333 Failed

model breast_cancer_xgboost

dataset breast_cancer

slicing_function `fractal dimension error` < 0.003 AND `fractal dimension error` >= 0.002

threshold 0.9101398601398601

Test Accuracy on data slice “`worst concavity` >= 0.277 AND `worst concavity` < 0.415”

Measured Metric = 0.83333 Failed

model breast_cancer_xgboost

dataset breast_cancer

slicing_function `worst concavity` >= 0.277 AND `worst concavity` < 0.415

threshold 0.9101398601398601

Test Accuracy on data slice “`compactness error` >= 0.020 AND `compactness error` < 0.030”

Measured Metric = 0.84375 Failed

model breast_cancer_xgboost

dataset breast_cancer

slicing_function `compactness error` >= 0.020 AND `compactness error` < 0.030

threshold 0.9101398601398601

Test Accuracy on data slice “`mean compactness` >= 0.099 AND `mean compactness` < 0.128”

Measured Metric = 0.84375 Failed

model breast_cancer_xgboost

dataset breast_cancer

slicing_function `mean compactness` >= 0.099 AND `mean compactness` < 0.128

threshold 0.9101398601398601

Test Recall on data slice “`concave points error` >= 0.009 AND `concave points error` < 0.015”

Measured Metric = 0.875 Failed

model breast_cancer_xgboost

dataset breast_cancer

slicing_function `concave points error` >= 0.009 AND `concave points error` < 0.015

threshold 0.9179775280898876

Test Accuracy on data slice “`perimeter error` < 2.151 AND `perimeter error` >= 1.505”

Measured Metric = 0.88571 Failed

model breast_cancer_xgboost

dataset breast_cancer

slicing_function `perimeter error` < 2.151 AND `perimeter error` >= 1.505

threshold 0.9101398601398601

Test Accuracy on data slice “`area error` < 40.745 AND `area error` >= 19.215”

Measured Metric = 0.88889 Failed

model breast_cancer_xgboost

dataset breast_cancer

slicing_function `area error` < 40.745 AND `area error` >= 19.215

threshold 0.9101398601398601

2024-05-29 11:38:26,789 pid:50687 Thread-59 (_track) urllib3.connectionpool WARNING  Connection pool is full, discarding connection: api.mixpanel.com. Connection pool size: 10
2024-05-29 11:38:26,796 pid:50687 Thread-58 (_track) urllib3.connectionpool WARNING  Connection pool is full, discarding connection: api.mixpanel.com. Connection pool size: 10
2024-05-29 11:38:26,822 pid:50687 Thread-60 (_track) urllib3.connectionpool WARNING  Connection pool is full, discarding connection: api.mixpanel.com. Connection pool size: 10
2024-05-29 11:38:26,848 pid:50687 Thread-61 (_track) urllib3.connectionpool WARNING  Connection pool is full, discarding connection: api.mixpanel.com. Connection pool size: 10
2024-05-29 11:38:26,853 pid:50687 Thread-63 (_track) urllib3.connectionpool WARNING  Connection pool is full, discarding connection: api.mixpanel.com. Connection pool size: 10
2024-05-29 11:38:26,873 pid:50687 Thread-62 (_track) urllib3.connectionpool WARNING  Connection pool is full, discarding connection: api.mixpanel.com. Connection pool size: 10
2024-05-29 11:38:26,899 pid:50687 Thread-64 (_track) urllib3.connectionpool WARNING  Connection pool is full, discarding connection: api.mixpanel.com. Connection pool size: 10

Customize your suite by loading objects from the Giskard catalog

Customize your suite by loading objects from the Giskard catalog

The Giskard open source catalog will enable to load:

• Tests such as metamorphic, performance, prediction & data drift, statistical tests, etc

• Slicing functions such as detectors of toxicity, hate, emotion, etc

• Transformation functions such as generators of typos, paraphrase, style tune, etc

To create custom tests, refer to this page.

For demo purposes, we will load a simple unit test (test_f1) that checks if the test F1 score is above the given threshold. For more examples of tests and functions, refer to the Giskard catalog.

test_suite.add_test(testing.test_f1(model=giskard_model, dataset=giskard_dataset, threshold=0.7)).run()