# Giskard Documentation

> Giskard is an AI testing and evaluation platform. This is the official documentation site for Giskard products.

## For LLMs

- **Agent skill discovery index:** https://docs.giskard.ai/.well-known/agent-skills/index.json
- **Full content in a single file:** https://docs.giskard.ai/llms-full.txt
- Every documentation page is also available as raw Markdown by appending `.md` to any URL. All links below point directly to the Markdown version.

## About Giskard

Giskard provides tools for testing, evaluating, and red-teaming LLM-based applications. The company offers an enterprise platform (Giskard Hub) and an open-source Python library (Giskard Checks) for AI quality assurance.

- Website: https://giskard.ai
- Documentation: https://docs.giskard.ai
- GitHub: https://github.com/Giskard-AI/giskard-oss
- Discord: https://discord.com/invite/ABvfpbu69R

## Products

### Giskard Hub
Enterprise platform for testing, evaluating, and red teaming agentic applications. It complements the open-source library and goes further on every axis. Its scans detect more vulnerabilities and business failures than the open-source scan, and continuous red teaming runs them on a schedule with alerting. Teams share projects, datasets, and evaluation dashboards under role-based access. Business users create and annotate tests in a web UI without code, while developers automate the same work through the SDK. It also adds synthetic test generation from knowledge bases, tool-calling tests, evaluation comparison and scheduling, SSO, 2FA, audit trails, and SOC 2 compliance. Test datasets import from the open-source library, so teams start local and scale up.

Typical entry points:

- Hub UI Introduction: https://docs.giskard.ai/hub/ui.md
- Hub SDK Quickstart: https://docs.giskard.ai/hub/sdk/quickstart.md
- Open Source vs Hub: https://docs.giskard.ai/start/comparison.md

### Giskard OSS
Open-source Python library (the `giskard` package, Apache-2.0) for testing, evaluating, and red teaming agentic applications, such as LLM-based chatbots, RAGs, and classification systems. It has two parts, Checks and Scan.

Checks is pytest-native behavioral testing. A scenario is one test case sent to the agent. A check is a rule its reply must satisfy, written in Python or in natural language and graded by an LLM judge. Checks pass or fail, so a suite can gate a pull request like unit tests. The target under test is any async callable, from any framework. Judged checks call your own LLM provider with your keys.

Scan, installed through the `scan` extra, is red teaming. `vulnerability_scan()` probes an agent with generated adversarial inputs and returns a re-runnable suite for regression testing. `third_party_scan()` runs garak or deepteam against the same agent.

Install with `pip install "giskard[scan,openai]"` (Python 3.12+). Swap `openai` for `google`, `anthropic`, `azure`, or `litellm`. Use the library for tests that run in CI. Use the Hub for a UI, shared datasets, more powerful checks and scanners, and continuous red teaming and evaluation.

Typical entry points:

- Install & Configure: https://docs.giskard.ai/oss/checks/installation.md
- Quickstart: https://docs.giskard.ai/oss/checks/quickstart.md
- Write Your First Test: https://docs.giskard.ai/oss/checks/tutorials/your-first-test.md
- Run Tests with pytest: https://docs.giskard.ai/oss/checks/how-to/run-in-pytest.md
- Scan for Vulnerabilities: https://docs.giskard.ai/oss/solutions/scan-vulnerabilities.md

### Giskard Research
Open research on AI safety and security, including the Phare multilingual benchmark, RealHarm incident dataset, and RealPerformance functional issues dataset.

## Documentation Structure

### Overview
- Welcome to Giskard: https://docs.giskard.ai/.md
- Open Source vs Hub: https://docs.giskard.ai/start/comparison.md

### Knowledge Glossary
- AI Testing & Evaluation Glossary: https://docs.giskard.ai/start/glossary.md
- AI Testing Methodologies: https://docs.giskard.ai/start/glossary/testing-methodologies.md

### Glossary — Business Failures
- Overview: https://docs.giskard.ai/start/glossary/business.md
- Addition of Information: https://docs.giskard.ai/start/glossary/business/addition-of-information.md
- Business Out of Scope: https://docs.giskard.ai/start/glossary/business/business-out-of-scope.md
- Denial of Answers: https://docs.giskard.ai/start/glossary/business/denial-of-answers.md
- Hallucination: https://docs.giskard.ai/start/glossary/business/hallucination.md
- Moderation Issues: https://docs.giskard.ai/start/glossary/business/moderation-issues.md
- Omission: https://docs.giskard.ai/start/glossary/business/omission.md

### Glossary — Security Vulnerabilities
- Overview: https://docs.giskard.ai/start/glossary/security.md
- Prompt Injection: https://docs.giskard.ai/start/glossary/security/injection.md
- Harmful Content: https://docs.giskard.ai/start/glossary/security/harmful-content.md
- Information Disclosure: https://docs.giskard.ai/start/glossary/security/information-disclosure.md
- Output Formatting: https://docs.giskard.ai/start/glossary/security/formatting.md
- Robustness: https://docs.giskard.ai/start/glossary/security/robustness.md
- Stereotypes & Discrimination: https://docs.giskard.ai/start/glossary/security/stereotypes.md

### Glossary — LLM Benchmarks
- Overview: https://docs.giskard.ai/start/glossary/llm-benchmarks.md
- Reasoning and Language: https://docs.giskard.ai/start/glossary/llm-benchmarks/reasoning-and-language.md
- Math Problems: https://docs.giskard.ai/start/glossary/llm-benchmarks/math-problems.md
- Coding: https://docs.giskard.ai/start/glossary/llm-benchmarks/coding.md
- Conversation and Chatbot: https://docs.giskard.ai/start/glossary/llm-benchmarks/conversation-and-chatbot.md
- Safety: https://docs.giskard.ai/start/glossary/llm-benchmarks/safety.md
- Domain-Specific: https://docs.giskard.ai/start/glossary/llm-benchmarks/domain-specific.md

### Hub UI
- Introduction: https://docs.giskard.ai/hub/ui.md
- Track Changes with Audit Logs: https://docs.giskard.ai/hub/ui/audit-logs.md
- Continuous Red Teaming: https://docs.giskard.ai/hub/ui/continuous-red-teaming.md
- User Management: https://docs.giskard.ai/hub/ui/user-management.md

### Hub UI — Setup
- Setup Overview: https://docs.giskard.ai/hub/ui/setup.md
- Projects: https://docs.giskard.ai/hub/ui/setup/projects.md
- Agents: https://docs.giskard.ai/hub/ui/setup/agents.md
- Knowledge Bases: https://docs.giskard.ai/hub/ui/setup/knowledge-bases.md

### Hub UI — Datasets
- Datasets Overview: https://docs.giskard.ai/hub/ui/datasets.md
- Import Scenarios: https://docs.giskard.ai/hub/ui/datasets/import.md
- Create Manual Scenarios: https://docs.giskard.ai/hub/ui/datasets/manual.md
- Prompt Preset Scenarios: https://docs.giskard.ai/hub/ui/datasets/prompt-preset.md
- Knowledge Base Scenarios: https://docs.giskard.ai/hub/ui/datasets/knowledge-base.md

### Hub UI — Evaluations
- Evaluations Overview: https://docs.giskard.ai/hub/ui/evaluations.md
- Run and Review: https://docs.giskard.ai/hub/ui/evaluations/create.md
- Compare Evaluations: https://docs.giskard.ai/hub/ui/evaluations/compare.md
- Schedule Evaluations: https://docs.giskard.ai/hub/ui/evaluations/schedule.md

### Hub UI — Scan
- Scan Overview: https://docs.giskard.ai/hub/ui/scan.md
- Launch a Scan: https://docs.giskard.ai/hub/ui/scan/launch-scan.md
- Review Scan Results: https://docs.giskard.ai/hub/ui/scan/review-scan-results.md

### Hub UI — Vulnerability Categories
- Attack Categories Overview: https://docs.giskard.ai/hub/ui/scan/vulnerability-categories.md
- Harmful Content Generation: https://docs.giskard.ai/hub/ui/scan/vulnerability-categories/harmful-content-generation.md
- Internal Information Exposure: https://docs.giskard.ai/hub/ui/scan/vulnerability-categories/internal-information-exposure.md
- Prompt Injection: https://docs.giskard.ai/hub/ui/scan/vulnerability-categories/prompt-injection.md
- Data Privacy Exfiltration: https://docs.giskard.ai/hub/ui/scan/vulnerability-categories/data-privacy-exfiltration.md
- Training Data Extraction: https://docs.giskard.ai/hub/ui/scan/vulnerability-categories/training-data-extraction.md
- Excessive Agency: https://docs.giskard.ai/hub/ui/scan/vulnerability-categories/excessive-agency.md
- Hallucination: https://docs.giskard.ai/hub/ui/scan/vulnerability-categories/hallucination.md
- Denial Of Service: https://docs.giskard.ai/hub/ui/scan/vulnerability-categories/denial-of-service.md
- Brand Damaging And Reputation: https://docs.giskard.ai/hub/ui/scan/vulnerability-categories/brand-damaging-and-reputation.md
- Legal And Financial Risk: https://docs.giskard.ai/hub/ui/scan/vulnerability-categories/legal-and-financial-risk.md
- Misguidance And Unauthorized Advice: https://docs.giskard.ai/hub/ui/scan/vulnerability-categories/misguidance-and-unauthorized-advice.md

### Hub UI — Annotate
- Annotate Overview: https://docs.giskard.ai/hub/ui/annotate.md
- Metrics, Failure Categories & Tags: https://docs.giskard.ai/hub/ui/annotate/overview.md
- Task Management: https://docs.giskard.ai/hub/ui/annotate/task-management.md
- Review Test Results: https://docs.giskard.ai/hub/ui/annotate/review-test-results.md
- Modify Test Cases: https://docs.giskard.ai/hub/ui/annotate/modify-test-cases.md

### Hub UI — Release Notes
- Release Notes: https://docs.giskard.ai/hub/ui/release-notes.md

### Hub SDK
- Introduction: https://docs.giskard.ai/hub/sdk.md
- Quickstart: https://docs.giskard.ai/hub/sdk/quickstart.md
- Core Concepts: https://docs.giskard.ai/hub/sdk/concepts.md
- API Reference: https://docs.giskard.ai/hub/sdk/reference.md
- Migration Guide: https://docs.giskard.ai/hub/sdk/migration.md
- Release Notes: https://docs.giskard.ai/hub/sdk/release-notes.md

### Hub SDK Guides
- Projects: https://docs.giskard.ai/hub/sdk/guides/projects.md
- Evaluations: https://docs.giskard.ai/hub/sdk/guides/evaluations.md
- Datasets & Checks: https://docs.giskard.ai/hub/sdk/guides/datasets-and-checks.md
- Vulnerability Scanning: https://docs.giskard.ai/hub/sdk/guides/scans.md
- Agents & Knowledge Bases: https://docs.giskard.ai/hub/sdk/guides/agents-and-knowledge-bases.md
- Playground Chats: https://docs.giskard.ai/hub/sdk/guides/playground-chats.md
- Tasks: https://docs.giskard.ai/hub/sdk/guides/tasks.md
- Audit Logs: https://docs.giskard.ai/hub/sdk/guides/audit.md

### Giskard Checks (Open Source)
- Giskard Library: https://docs.giskard.ai/oss.md
- Agent Skills: https://docs.giskard.ai/oss/agent-skills.md
- Contribute to Giskard: https://docs.giskard.ai/oss/contributing.md
- What are Giskard Checks?: https://docs.giskard.ai/oss/checks.md
- Install & Configure: https://docs.giskard.ai/oss/checks/installation.md
- Quickstart: https://docs.giskard.ai/oss/checks/quickstart.md

### Checks Tutorials
- Tutorials Overview: https://docs.giskard.ai/oss/checks/tutorials.md
- Your First Test: https://docs.giskard.ai/oss/checks/tutorials/your-first-test.md
- Your First LLM Call: https://docs.giskard.ai/oss/checks/tutorials/single-turn.md
- Multi-Turn Scenarios: https://docs.giskard.ai/oss/checks/tutorials/multi-turn.md
- Dynamic Scenarios: https://docs.giskard.ai/oss/checks/tutorials/dynamic-scenarios.md
- Test Suites: https://docs.giskard.ai/oss/checks/tutorials/test-suites.md

### Checks How-to Guides
- How-to Overview: https://docs.giskard.ai/oss/checks/how-to.md
- Run Tests with pytest: https://docs.giskard.ai/oss/checks/how-to/run-in-pytest.md
- Simulate Users: https://docs.giskard.ai/oss/checks/how-to/simulate-users.md
- Spy on Internal Calls: https://docs.giskard.ai/oss/checks/how-to/spy-on-calls.md
- Testing Structured Outputs: https://docs.giskard.ai/oss/checks/how-to/structured-output.md
- Batch Evaluation: https://docs.giskard.ai/oss/checks/how-to/batch-evaluation.md
- Stateful Checks: https://docs.giskard.ai/oss/checks/how-to/stateful-checks.md
- Custom Checks: https://docs.giskard.ai/oss/checks/how-to/custom-checks.md
- CI/CD Integration: https://docs.giskard.ai/oss/checks/how-to/ci-cd.md
- Custom Trace Types: https://docs.giskard.ai/oss/checks/how-to/custom-trace.md

### Checks API Reference
- Reference Overview: https://docs.giskard.ai/oss/checks/reference.md
- Core API: https://docs.giskard.ai/oss/checks/reference/core.md
- Checks: https://docs.giskard.ai/oss/checks/reference/checks.md
- Scenarios: https://docs.giskard.ai/oss/checks/reference/scenarios.md
- Testing Utilities: https://docs.giskard.ai/oss/checks/reference/testing-utils.md
- Generators: https://docs.giskard.ai/oss/checks/reference/generators.md
- Utilities: https://docs.giskard.ai/oss/checks/reference/utils.md
- Settings: https://docs.giskard.ai/oss/checks/reference/settings.md

### Checks Explanation
- Explanation Overview: https://docs.giskard.ai/oss/checks/explanation.md
- Core Concepts: https://docs.giskard.ai/oss/checks/explanation/core-concepts.md
- When to Use Which Check: https://docs.giskard.ai/oss/checks/explanation/when-to-use-which-check.md
- Async Design & pytest: https://docs.giskard.ai/oss/checks/explanation/async-and-pytest.md
- JSONPath in Checks: https://docs.giskard.ai/oss/checks/explanation/jsonpath-in-checks.md

### Checks Use Cases
- Use Cases Overview: https://docs.giskard.ai/oss/checks/use-cases.md
- RAG Evaluation: https://docs.giskard.ai/oss/checks/use-cases/rag-evaluation.md
- Testing Agents: https://docs.giskard.ai/oss/checks/use-cases/testing-agents.md
- Chatbot Testing: https://docs.giskard.ai/oss/checks/use-cases/chatbot-testing.md
- Content Moderation: https://docs.giskard.ai/oss/checks/use-cases/content-moderation.md

### Solutions (Open Source)
Task-shaped quickstarts that sit above the Checks and Scan sections.
- Solutions: https://docs.giskard.ai/oss/solutions.md
- Quality Assessment: https://docs.giskard.ai/oss/solutions/quality-assessment.md
- Scan Vulnerabilities: https://docs.giskard.ai/oss/solutions/scan-vulnerabilities.md
- Check Agentic Systems: https://docs.giskard.ai/oss/solutions/check-agentic-systems.md

### Giskard Scan (Open Source)
- Giskard Scan: https://docs.giskard.ai/oss/scan.md
- Install & Configure: https://docs.giskard.ai/oss/scan/installation.md

### Scan Tutorials
- Tutorials Overview: https://docs.giskard.ai/oss/scan/tutorials.md
- Your First Scan: https://docs.giskard.ai/oss/scan/tutorials/your-first-scan.md
- From Red-Team Finding to Regression Test: https://docs.giskard.ai/oss/scan/tutorials/redteam-to-regression.md
- Write a Custom Scenario Generator: https://docs.giskard.ai/oss/scan/tutorials/custom-scenario-generator.md

### Scan How-to Guides
- How-to Overview: https://docs.giskard.ai/oss/scan/how-to.md
- Wrap Your Agent for the Scan: https://docs.giskard.ai/oss/scan/how-to/wrap-your-agent.md
- Customize a Scan: https://docs.giskard.ai/oss/scan/how-to/customize-a-scan.md
- Tune a Scan Run: https://docs.giskard.ai/oss/scan/how-to/tune-scan-options.md
- Run the Scan in CI: https://docs.giskard.ai/oss/scan/how-to/scan-in-ci.md
- Scan With Your Own Dataset: https://docs.giskard.ai/oss/scan/how-to/dataset-generators.md
- Save and Version a Scan Suite: https://docs.giskard.ai/oss/scan/how-to/save-and-version-suites.md
- Run a Quality Scan: https://docs.giskard.ai/oss/scan/how-to/quality-scan.md
- Run Garak and DeepTeam Scanners: https://docs.giskard.ai/oss/scan/how-to/third-party-scanners.md

### Scan API Reference
- Reference Overview: https://docs.giskard.ai/oss/scan/reference.md
- Scan API: https://docs.giskard.ai/oss/scan/reference/scan-api.md
- Generators: https://docs.giskard.ai/oss/scan/reference/generators.md
- Knowledge Base: https://docs.giskard.ai/oss/scan/reference/knowledge-base.md

### Scan Explanation
- Explanation Overview: https://docs.giskard.ai/oss/scan/explanation.md
- How the Scan Works: https://docs.giskard.ai/oss/scan/explanation/how-scan-works.md
- What the Scan Looks For: https://docs.giskard.ai/oss/scan/explanation/threat-taxonomy.md