Client reference
The HubClient is the main entry point for interacting with the Giskard Hub through the Python SDK.
- class giskard_hub.client.HubClient(hub_url: str | None = None, api_key: str | None = None, auto_add_api_suffix: bool | None = True, **kwargs)[source]
Bases:
SyncClient
Client class to handle interaction with the hub.
- chat_test_cases
Resource to interact with chat test cases (conversations).
- Type:
- checks
Resource to interact with checks.
- Type:
- datasets
Resource to interact with datasets.
- Type:
- evaluations
Resource to interact with evaluations.
- Type:
- knowledge_bases
Resource to interact with knowledge bases.
- Type:
- models
Resource to interact with models.
- Type:
- projects
Resource to interact with projects.
- Type:
- scheduled_evaluations
Resource to interact with scheduled evaluations.
- evaluate(*, dataset: str | Dataset, model: Model | str | Callable[[List[ChatMessage]], ModelOutput | str], name: str = NOT_GIVEN, tags: List[str] = NOT_GIVEN)[source]
Method to run an evaluation, either locally or remotely.
- Parameters:
dataset (str | Dataset) – ID of the dataset that will be used for the evaluation, or the dataset entity itself. List of tags to filter the chat test cases that will be evaluated.
model (str | Model | Callable[[List[ChatMessage]], ModelOutput | str]) – ID of the model to evaluate, or a model entity, or a local model function. A local model function is a function that takes a list of messages and returns a ModelOutput or a string.
name (str, optional) – The name of the evaluation run. If not provided, a random name will be automatically generated.
tags (List[str], optional) – List of tags to filter the chat test cases that will be evaluated.
- Returns:
The evaluation run entity.
- Return type:
EvaluationRun