Client reference

The HubClient is the main entry point for interacting with the Giskard Hub through the Python SDK.

class giskard_hub.client.HubClient(hub_url: str | None = None, api_key: str | None = None, auto_add_api_suffix: bool | None = True, **kwargs)[source]

Bases: SyncClient

Client class to handle interaction with the hub.

chat_test_cases

Resource to interact with chat test cases (conversations).

Type:: ChatTestCasesResource

checks

Resource to interact with checks.

Type:: ChecksResource

datasets

Resource to interact with datasets.

Type:: DatasetsResource

evaluations

Resource to interact with evaluations.

Type:: EvaluationsResource

knowledge_bases

Resource to interact with knowledge bases.

Type:: KnowledgeBasesResource

models

Resource to interact with models.

Type:: ModelsResource

projects

Resource to interact with projects.

Type:: ProjectsResource

scheduled_evaluations

Resource to interact with scheduled evaluations.

Type:: ScheduledEvaluationsResource

scans

Resource to interact with scans and probes.

Type:: ScansResource

evaluate(*, dataset: str | Dataset, model: Model | str | Callable[[List[ChatMessage]], ModelOutput | str], name: str = NOT_GIVEN, tags: List[str] = NOT_GIVEN)[source]

Method to run an evaluation, either locally or remotely.

Parameters:

dataset (str | Dataset) – ID of the dataset that will be used for the evaluation, or the dataset entity itself. List of tags to filter the chat test cases that will be evaluated.
model (str | Model | Callable[[List[ChatMessage]], ModelOutput | str]) – ID of the model to evaluate, or a model entity, or a local model function. A local model function is a function that takes a list of messages and returns a ModelOutput or a string.
name (str, optional) – The name of the evaluation run. If not provided, a random name will be automatically generated.
tags (List[str], optional) – List of tags to filter the chat test cases that will be evaluated.

Returns:

The evaluation run entity.

Return type:

EvaluationRun

class giskard_hub.client.SyncClient(*, http_client: Client | None = None)[source]

close()[source]

delete(path: str, **kwargs)[source]

get(path: str, **kwargs)[source]

patch(path: str, **kwargs)[source]

post(path: str, **kwargs)[source]

put(path: str, **kwargs)[source]