Overview
Task-oriented guides for writing tests with Giskard Checks. These guides assume you’re familiar with scenarios and checks. Each guide focuses on getting a specific job done.
Run Tests with pytest Configure pytest-asyncio and write async test functions for Giskard Checks scenarios
Simulate Users Use UserSimulator to drive multi-turn tests with LLM-generated user inputs and personas
Debug with Spy Inspect function calls during scenario execution using WithSpy to diagnose unexpected behavior
Testing Structured Outputs Validate Pydantic models, JSON objects, and nested fields using Equals, FnCheck, and JSONPath extraction
Batch Evaluation Run the same scenario pattern across many inputs and aggregate pass/fail results
Stateful Checks Track uniqueness, counts, and cross-scenario consistency with checks that accumulate state across runs
Custom Checks Build domain-specific validation logic using FnCheck, Check subclasses, metrics, and LLM-based checks
Custom trace types Subclass Trace, set trace_type and annotations, and render conversation-style transcripts with Rich
CI/CD Integration Run Giskard Checks in GitHub Actions with environment variable setup and pytest invocation