Skip to content
Giskard Documentation
Overview Hub UI Hub SDK Open Source
GitHubDiscord

Open Source vs Hub

Giskard offers two solutions for LLM agent testing and evaluation, each designed for different use cases and requirements.

Giskard Hub is our enterprise platform with advanced collaboration features, while Giskard Open Source is our free Python library for individual developers and researchers.

This guide will help you understand the differences and choose the right solution for your needs.

Feature comparison

Section titled “Feature comparison”
FeatureGiskard Open SourceGiskard Hub
Core Testing
Security vulnerability detectionBasic coverageState-of-the-art detection
Business failure detectionBasic coverageState-of-the-art detection
Continuous red teaming❌ Not available✅ Full support
Tool/function calling tests❌ Not available✅ Full support
Custom tests✅ Full support✅ Full support
Local evaluations✅ Full support✅ Full support
Team Collaboration
Multi-user access❌ Single user only✅ Full team support
Access control❌ Not available✅ Role-based access
Project management❌ Local only✅ Centralized
Dataset sharing❌ Local only✅ Team-wide
Automation & Monitoring
Scheduled evaluation runs❌ Not available✅ Fully supported
Evaluation comparison dashboard❌ Not available✅ Fully supported
Alerting❌ Not available✅ Configurable alerts
Performance tracking❌ Local only✅ Historical data
Enterprise Security
SSO (Single Sign-On)❌ Not available✅ SSO support ↗
2FA (Two-Factor Authentication)❌ Not available✅ 2FA support ↗
Audit trails❌ Not available✅ Full compliance ↗
SOC 2 compliance❌ Not available✅ SOC 2 certified ↗
Dedicated support & SLAs❌ Community only✅ Enterprise-grade ↗

When to use Giskard Open Source

Section titled “When to use Giskard Open Source”

Perfect for:

  • Individual developers and data scientists
  • Prototyping and research projects
  • CI/CD pipelines in development environments
  • Teams just starting with AI testing
  • Projects with budget constraints

What you get:

  • Full access to our basic testing capabilities
  • Local control over your data and models
  • No external dependencies or data sharing
  • Community support and open-source contributions

When to upgrade to Giskard Hub

Section titled “When to upgrade to Giskard Hub”

Consider upgrading to an enterprise subscription when you need:

  • Continuous red teaming - Automated testing and alerting
  • Team collaboration and business user enablement – Collaborate across technical and business teams: enable business users to contribute through annotations, prioritize actions based on test results, and access intuitive testing dashboards
  • Custom checks and result categorization – Create your own tests and automatically categorize test results for deeper, customizable analysis
  • Enterprise security features - SSO (Single Sign-On), SOC 2 compliance, and 2FA (Two-Factor Authentication) for robust access control and regulatory requirements
  • Compliance - Audit trails and access control requirements
  • Scale - Managing multiple projects and models with specific permissions by users and roles

Optional upgrade path

Section titled “Optional upgrade path”

The transition from Open Source to Giskard Hub is designed to be seamless. You can start with Open Source and gradually migrate to Hub as your team grows.

  1. Start with Open Source - Build your testing foundation locally with Giskard Checks
  2. Add Hub SDK - Import datasets from Open Source to Hub
  3. Gradual migration - Move more workflows to Hub as your project complexity grows
  4. Full Giskard Hub adoption - Leverage all Giskard Hub features for maximum efficiency

Getting started

Section titled “Getting started”