Client reference

The HubClient is the main entry point for interacting with the Giskard Hub through the Python SDK.

class giskard_hub.client.HubClient(hub_url: str | None = None, api_key: str | None = None, auto_add_api_suffix: bool | None = True, **kwargs)[source]

Bases: SyncClient

Client class to handle interaction with the hub.

chat_test_cases

Resource to interact with chat test cases (conversations).

Type:

ChatTestCasesResource

checks

Resource to interact with checks.

Type:

ChecksResource

datasets

Resource to interact with datasets.

Type:

DatasetsResource

evaluations

Resource to interact with evaluations.

Type:

EvaluationsResource

knowledge_bases

Resource to interact with knowledge bases.

Type:

KnowledgeBasesResource

models

Resource to interact with models.

Type:

ModelsResource

projects

Resource to interact with projects.

Type:

ProjectsResource

scheduled_evaluations

Resource to interact with scheduled evaluations.

Type:

ScheduledEvaluationsResource

evaluate(*, dataset: str | Dataset, model: Model | str | Callable[[List[ChatMessage]], ModelOutput | str], name: str = NOT_GIVEN, tags: List[str] = NOT_GIVEN)[source]

Method to run an evaluation, either locally or remotely.

Parameters:
  • dataset (str | Dataset) – ID of the dataset that will be used for the evaluation, or the dataset entity itself. List of tags to filter the chat test cases that will be evaluated.

  • model (str | Model | Callable[[List[ChatMessage]], ModelOutput | str]) – ID of the model to evaluate, or a model entity, or a local model function. A local model function is a function that takes a list of messages and returns a ModelOutput or a string.

  • name (str, optional) – The name of the evaluation run. If not provided, a random name will be automatically generated.

  • tags (List[str], optional) – List of tags to filter the chat test cases that will be evaluated.

Returns:

The evaluation run entity.

Return type:

EvaluationRun

class giskard_hub.client.SyncClient(*, http_client: Client | None = None)[source]
close()[source]
delete(path: str, **kwargs)[source]
get(path: str, **kwargs)[source]
patch(path: str, **kwargs)[source]
post(path: str, **kwargs)[source]
put(path: str, **kwargs)[source]