Overview

Task-oriented guides for writing tests with Giskard Checks. These guides assume you’re familiar with scenarios and checks. Each guide focuses on getting a specific job done.

Run Tests with pytest Configure pytest-asyncio and write async test functions for Giskard Checks scenarios

Simulate Users Use UserSimulator to drive multi-turn tests with LLM-generated user inputs and personas

Debug with Spy Inspect function calls during scenario execution using WithSpy to diagnose unexpected behavior

Testing Structured Outputs Validate Pydantic models, JSON objects, and nested fields using Equals, FnCheck, and JSONPath extraction

Batch Evaluation Run the same scenario pattern across many inputs and aggregate pass/fail results

Stateful Checks Track uniqueness, counts, and cross-scenario consistency with checks that accumulate state across runs

Custom Checks Build domain-specific validation logic using FnCheck, Check subclasses, metrics, and LLM-based checks

Custom trace types Subclass Trace, set trace_type and annotations, and render conversation-style transcripts with Rich

CI/CD Integration Run Giskard Checks in GitHub Actions with environment variable setup and pytest invocation