Migration Guide

Hub v3 pairs with SDK 3.2.0. This guide covers what changes when you move from Hub v2 (SDK 3.1.x) to Hub v3. Most SDK renames are backwards compatible and only emit a DeprecationWarning. Some check identifier renames are breaking, so read that section first.

Upgrade the SDK

pip install --upgrade "giskard-hub>=3.2.0"

Verify the installed version:

python -c "import giskard_hub; print(giskard_hub.__version__)"

Breaking: check identifiers renamed

The Hub renamed the built-in check identifiers listed below. Requests that pass one of these old identifiers get a 422 error from the Hub, usually with a “Did you mean the ’…’ check?” tip. This applies everywhere an identifier appears: checks inside scenarios, hub.evaluations.run_single(), custom check params, and uploaded dataset files.

Old identifier	New identifier
`correctness`	`hub_correctness`
`metadata`	`hub_metadata`
`string_match`	`string_matching`

# Hub v2 (SDK 3.1)
hub.test_cases.create(
    dataset_id=dataset_id,
    messages=[{"role": "user", "content": "What is your refund policy?"}],
    checks=[{"identifier": "correctness", "params": {"reference": "30 days."}}],
)

# Hub v3 (SDK 3.2)
hub.scenarios.create(
    dataset_id=dataset_id,
    interactions=[
        {
            "input": {
                "messages": [{"role": "user", "content": "What is your refund policy?"}]
            },
            "checks": [
                {"identifier": "hub_correctness", "params": {"reference": "30 days."}}
            ],
        }
    ],
)

Conformity and Groundedness remain compatible

You can continue using the conformity and groundedness identifiers from Hub v2. They are aliases for the Hub checks:

Hub v2 identifier	Canonical identifier
`conformity`	`hub_conformity`
`groundedness`	`hub_groundedness`

Prefer hub_conformity and hub_groundedness in new code. The short names still work, but the Hub stores the canonical hub_ identifiers. When you fetch scenarios or evaluation results, these checks come back as hub_conformity and hub_groundedness, even if you created them with the aliases.

Your existing check configurations still work with SDK 3.2.0. For new code, use the canonical identifiers:

checks = [
    {"identifier": "hub_conformity", "params": {"rules": ["Use formal language."]}},
    {
        "identifier": "hub_groundedness",
        "params": {"context": "Our return window is 30 days."},
    },
]

Conformity: for new configurations, use rule: str. Your existing rules: list[str] is still accepted and converted to a single string with one bullet line per rule. The UI shows these instructions in one Rule field.
Groundedness: your existing fixed context remains supported. You can also pass a list of strings or use context_key to read context from the trace. A supplied context takes precedence over context_key.
Target path: use target_key for both checks. Your existing text_key is still accepted. The default target for scenario checks remains the response content (trace.last.outputs.response.content).

Use one parameter name for each setting: rule or rules, and target_key or text_key. See Built-in checks for all parameters and defaults.

Check params renamed

Whether you pass raw dicts or the typed params classes:

CorrectnessParams is removed. Use HubCorrectnessParams (reference).
MetadataParams is removed. Use HubMetadataParams (json_path_rules).
StringMatchParams is removed. Use StringMatchingParams.
ConformityParams and HubConformityParams accept rule and the legacy rules list. Both accept target_key and the legacy text_key. Existing fields remain supported without deprecation warnings. GroundednessParams and HubGroundednessParams also accept these target fields, context as a string or list, and context_key.
semantic_similarity keeps its identifier, but its reference param is renamed to reference_text. Scripts passing {"reference": ...} to this check get a 422.
Typed params classes cover built-in checks and their compatibility aliases (e.g. HubGroundednessParams, GroundednessParams, SemanticSimilarityParams, LLMJudgeParams).

Validation moved server-side

The SDK no longer validates check identifiers or params locally, the Hub does. Errors that were previously raised locally as ValueError now surface as UnprocessableEntityError (HTTP 422). Wrong or unknown check params, which were previously accepted and silently dropped, are now rejected at save time and at run time. Update any except ValueError handling around check creation accordingly.

Custom check identifiers require a `custom_` prefix

Custom check identifiers must now start with custom_ (e.g. custom_tone_professional). hub.checks.create() and hub.checks.update() reject any other identifier.

The Hub upgrade renames your existing custom checks automatically: tone_professional becomes custom_tone_professional. Stored scenarios keep working, since their check references are updated by the same migration. Scripts are not: any code that references a custom check by its old identifier (in scenario checks, hub.evaluations.run_single(), or uploaded dataset files) must switch to the prefixed name.

# Hub v2 (SDK 3.1)
check = hub.checks.create(
    project_id=project_id,
    identifier="tone_professional",
    name="Professional tone",
    params={"type": "conformity", "rules": ["Use formal language."]},
)
checks = [{"identifier": "tone_professional"}]

# Hub v3 (SDK 3.2)
check = hub.checks.create(
    project_id=project_id,
    identifier="custom_tone_professional",
    name="Professional tone",
    params={"type": "hub_conformity", "rule": "Use formal language."},
)
checks = [{"identifier": "custom_tone_professional"}]

Deprecated: chat-shaped arguments become structured input/output

Hub v3 supports agents with arbitrary input and output schemas, so the SDK moved from chat-only arguments to structured input / output dicts. The old chat-shaped arguments still work with a DeprecationWarning and are translated for you.

Scenario creation: `messages` / `demo_output` / `checks` become `interactions`

The flat scenario shape maps into a single interaction. messages becomes input["messages"], demo_output becomes output (a plain string is wrapped as an assistant response, a dict’s metadata key is split out), and checks attach to the interaction:

# Hub v2 (SDK 3.1) — deprecated, still works
hub.test_cases.create(
    dataset_id=dataset_id,
    messages=[{"role": "user", "content": "What is your refund policy?"}],
    demo_output={
        "role": "assistant",
        "content": "We offer a 30-day return policy.",
        "metadata": {"category": "returns"},
    },
    checks=[{"identifier": "hub_correctness", "params": {"reference": "30 days."}}],
)

# Hub v3 (SDK 3.2)
hub.scenarios.create(
    dataset_id=dataset_id,
    interactions=[
        {
            "input": {
                "messages": [{"role": "user", "content": "What is your refund policy?"}]
            },
            "output": {
                "response": {
                    "role": "assistant",
                    "content": "We offer a 30-day return policy.",
                },
                "metadata": {"category": "returns"},
            },
            "checks": [
                {"identifier": "hub_correctness", "params": {"reference": "30 days."}}
            ],
        }
    ],
)

You cannot mix interactions= with the legacy arguments in one call. The same applies to scenarios.update().

`datasets.upload()` records

Records in the legacy {messages, demo_output, checks} shape are still translated on upload, with a DeprecationWarning. Use the new {"interactions": [{"position", "input", "output", "checks"}]} shape, and update the renamed identifiers listed above. The conformity and groundedness aliases remain valid.

`agents.generate_completion()`: `messages` becomes `input`

# Hub v2 (SDK 3.1) — deprecated, still works
output = hub.agents.generate_completion(agent_id, messages=[{"role": "user", "content": "Hi"}])
print(output.response.content)

# Hub v3 (SDK 3.2)
output = hub.agents.generate_completion(
    agent_id, input={"messages": [{"role": "user", "content": "Hi"}]}
)
print(output.output["response"]["content"])

The GenerateCompletionOutput.response and .message accessors are deprecated. Read the structured output dict directly.

`evaluations.run_single()`: `messages` becomes `input_data`

# Hub v2 (SDK 3.1) — deprecated, still works
hub.evaluations.run_single(project_id=project_id, messages=[...], agent_output=..., checks=[...])

# Hub v3 (SDK 3.2)
hub.evaluations.run_single(project_id=project_id, input_data={"messages": [...]}, agent_output=..., checks=[...])

Deprecated flattened accessors

These model properties still work but emit a DeprecationWarning. They flatten structured data back into chat messages, which loses information for non-chat agents:

Deprecated accessor	Read instead
`Scenario.messages`	`scenario.interactions[i].input`
`PlaygroundChat.messages`	`chat.exchanges` (`input` / `output`)
`ScanProbeAttempt.messages`	`attempt.input` / `attempt.output`

Deprecated: test cases renamed to scenarios

The Hub renamed test cases to scenarios. The old SDK surface still works and maps to the new endpoints, but every call emits a DeprecationWarning. Update at your own pace:

Deprecated (still works)	Use instead
`hub.test_cases.*`	`hub.scenarios.*`
`hub.test_cases.comments`	`hub.scenarios.comments`
`hub.datasets.list_test_cases()`	`hub.datasets.list_scenarios()`
`hub.datasets.search_test_cases()`	`hub.datasets.search_scenarios()`
`hub.evaluations.results.rerun_test_case()`	`hub.evaluations.results.rerun_scenario()`
`test_case_ids=` (bulk operations)	`scenario_ids=`
`include=["test_case"]` (results)	`include=["scenario"]`
`set_test_case_draft=`	`set_scenario_draft=`
`dataset_test_case_id=` (`hub.tasks.create`)	`dataset_scenario_id=`
`set_test_case_status=` (`hub.tasks.update`)	`set_scenario_status=`
`result.test_case` / `result.test_case_exists`	`result.scenario` / `result.scenario_exists`

The legacy flat scenario shape (messages=, checks=, demo_output=) is also deprecated in favour of interactions=. See chat-shaped arguments above for the mapping.

In audit logs, the endpoint accepts entity_type="scenario" and "scenario_evaluation", while stored events keep the values "test_case" and "test_case_evaluation" in search results.

Deprecated: project scenarios renamed to prompt presets

The project-level “Scenarios” (persona and behaviour templates) are now Prompt Presets. The old Hub endpoints were removed, which is why SDK 3.1.x breaks against Hub v3. In SDK 3.2, the old methods still work against the new endpoints with a DeprecationWarning:

Deprecated (still works)	Use instead
`hub.projects.scenarios.*`	`hub.projects.prompt_presets.*`
`hub.datasets.generate_scenario_based(scenario_id=)`	`hub.datasets.generate_preset_based(prompt_preset_id=)`

# Hub v2 (SDK 3.1)
dataset = hub.datasets.generate_scenario_based(
    project_id=project_id,
    agent_id=agent_id,
    scenario_id=scenario_id,
    dataset_name="Generated suite",
    n_examples=10,
)

# Hub v3 (SDK 3.2)
dataset = hub.datasets.generate_preset_based(
    project_id=project_id,
    agent_id=agent_id,
    prompt_preset_id=prompt_preset_id,
    dataset_name="Generated suite",
    n_examples=10,
)

types.Scenario now means the dataset item (formerly the test case). The prompt preset types are PromptPreset and PromptPresetPreview. The DatasetGenerateScenarioBasedParams type is removed, import DatasetGeneratePresetBasedParams instead.

See Projects & Prompt Presets for the new API.

Fixing a broken CI pipeline

If your CI started failing after the Hub upgrade, work through this checklist:

Pin the SDK to 3.2.0 or later in your requirements.
Search your scripts for old check identifiers (correctness, metadata, string_match) and replace them with the new names from the table above. Also rename reference to reference_text on semantic_similarity checks.
Check Conformity and Groundedness parameters. Prefer hub_conformity and hub_groundedness; the short names are aliases, and fetched results return the hub_ identifiers. Use params={"rule": ...} for scenario Conformity checks; existing rules lists still work. Set target_key explicitly if you need a target other than the default.
Prefix custom check references. The Hub renamed your existing custom checks to custom_<identifier>. Update scripts that reference them by the old identifier.
Update uploaded dataset files (hub.datasets.upload() JSON/JSONL): the records may keep the legacy shape. Update correctness, metadata, and string_match; Conformity and Groundedness aliases are accepted.
Treat any remaining UnprocessableEntityError (422) as a validation message from the Hub. The error body names the rejected identifier or param and often suggests the correct check.