Scan Vulnerabilities

Giskard’s scan red-teams your agent: it generates hostile inputs from a description of what the agent does, runs them, and reports the ones that got through. Use it to find safety and security problems before your users do.

A clean run is not a certificate. The scan samples attacks, an LLM judges the replies (and can be wrong either way), and a miss only means those particular attacks failed. Read the failures before you act on them.

How does it work?

From a description of your agent, an LLM writes attack scenarios and runs them — either as a single message or as a conversation that keeps pushing. A second LLM then judges whether each reply is a failure. The glossary covers the failure types, including prompt injection, harmful content, and information disclosure.

A benchmark scores a foundation model on generic tasks. This scan attacks your agent, on the job it actually does.

Which attacks does it run?

The scan runs seven generators, mapped to the OWASP LLM Top 10 and the vulnerability categories used across Giskard:

Attack	What it does	Vulnerability category
Prompt injection	Hides an injected instruction inside realistic content to see whether the agent obeys it instead of its original instructions.	Prompt Injection (OWASP LLM01)
Direct adversarial	Direct requests across built-in categories: stereotypes, illegal activity, CBRN, copyright, misinformation, unauthorized financial/medical/legal advice, politics, and religion. Up to three turns by default; pass target_mode="singleturn" to cap at one.	Harmful Content Generation, Misguidance & Unauthorized Advice
GOAT multi-turn jailbreak	Uses an attacker LLM that adapts over several turns, chaining strategies such as refusal suppression, persona modification, and hypothetical framing to push the agent toward objectives it should refuse.	Harmful Content Generation
Crescendo multi-turn	Opens with broad, benign questions and narrows the conversation step by step toward a harmful objective. No single message would be refused on its own.	Harmful Content Generation
GCG suffix injection	Appends a tuned string of meaningless-looking tokens to a harmful prompt, exploiting the model’s token handling rather than its reasoning.	Prompt Injection (OWASP LLM01)
do-not-answer corpus	Replays a Hugging Face dataset of prompts a well-behaved agent should refuse. Dropped when you pass commercial_use=True.	Harmful Content Generation
harmbench corpus	Replays a Hugging Face dataset of harmful-behavior prompts. Kept when commercial_use=True.	Harmful Content Generation

The vulnerability categories catalog lists every category a finding can be filed under.

Need a more advanced scan?

The Giskard Hub, our enterprise platform, takes red teaming much further: 55+ custom-designed probes across 11 vulnerability categories, aligned with the OWASP LLM Top 10 and more. On top of that, it grades your agent’s security, keeps testing it after deployment with continuous red teaming, and runs from a web interface your whole team can use. Talk to our team ↗ to see it in action.

What data is sent to Language Model providers?

The scan uses an LLM both to generate adversarial scenarios and to judge your agent’s answers. Those models see the description you pass in and the messages your agent exchanges. They do not see your source code, secrets, or tools unless those show up in the description or in the conversation. You choose the provider and model (see Before starting).

Which languages does it support?

LLM-backed generators (direct adversarial, GOAT, Crescendo) write scenarios in the languages you pass — BCP-47 codes such as "en", "fr", or "es". Pick a model that handles those languages well. Dataset-backed generators (prompt injection, the Hugging Face corpora, GCG) only emit scenarios for languages they actually ship; the rest are skipped.

Before starting

First, install the scan and choose the model that will generate and judge the scenarios. The scan ships in the scan extra, and the provider clients ship in a provider extra, so ask for both. Pick your provider below:

OpenAI

pip install "giskard[scan,openai]"

from giskard.checks import set_default_generator

set_default_generator("openai/gpt-4o")

Set OPENAI_API_KEY in your environment.

Anthropic

pip install "giskard[scan,anthropic]"

from giskard.checks import set_default_generator

set_default_generator("anthropic/claude-sonnet-4-20250514")

Set ANTHROPIC_API_KEY in your environment.

Gemini

pip install "giskard[scan,google]"

from giskard.checks import set_default_generator

set_default_generator("gemini/gemini-2.5-flash")

Set GEMINI_API_KEY (or GOOGLE_API_KEY) in your environment.

Any provider

LiteLLM reaches any provider through a single "<provider>/<model-name>" string, so it is the most flexible option.

pip install "giskard[scan,litellm]"

from giskard.agents.generators import LiteLLMGenerator
from giskard.checks import set_default_generator

llm_judge = LiteLLMGenerator(model="<provider>/<model-name>")
set_default_generator(llm_judge)

For example mistral/mistral-large-latest, bedrock/anthropic.claude-3-sonnet-20240229-v1:0, or ollama/qwen2.5. For the full list of providers, see LiteLLM’s provider conventions and set the matching API key in your environment.

set_default_generator makes your chosen model the default for generating and judging every scenario in the scan.

Step 1: Wrap your model

The scan talks to your agent through a single entry point, an async function that takes a typed input and returns a typed output. Both types are Pydantic models, which makes the contract explicit and validated.

Since multi-turn attacks call your agent once per turn, with only the new message as input, the right wrapper depends on how your agent keeps track of the conversation. Pick the pattern that matches yours:

Basic

If your agent answers each message independently, wrap it directly:

from pydantic import BaseModel


class AgentInput(BaseModel):
    question: str


class AgentOutput(BaseModel):
    answer: str


async def my_llm_app(prompt: str, **kwargs: object) -> str:
    """Replace this placeholder with your LLM application call."""
    raise NotImplementedError


async def my_agent(inputs: AgentInput) -> AgentOutput:
    # Call your own LLM app, chain, or agent here
    answer = await my_llm_app(inputs.question)
    return AgentOutput(answer=answer)

Stateful agent

If your agent stores the conversation on its side (for example a LangGraph checkpointer or a session-based API), it needs the same thread id on every turn of a conversation. To get one, subclass Trace with a generated thread_id field and declare a trace parameter. Giskard creates the trace when a conversation starts and preserves its fields across turns, so each conversation gets its own stable id:

from uuid import uuid4
from pydantic import BaseModel, Field
from giskard.checks import Trace


class AgentInput(BaseModel):
    question: str


class AgentOutput(BaseModel):
    answer: str


class AgentTrace(Trace[AgentInput, AgentOutput], frozen=True):
    thread_id: str = Field(default_factory=lambda: str(uuid4()))


async def my_agent(inputs: AgentInput, trace: AgentTrace) -> AgentOutput:
    # The same thread_id is kept for every turn of this conversation
    answer = await my_llm_app(inputs.question, thread_id=trace.thread_id)
    return AgentOutput(answer=answer)

Conversation history

If your agent is stateless and expects the full message history on every call, declare a trace parameter. During the scan, trace.interactions holds the previous turns of the current conversation, so you can rebuild the history and append the new message:

from pydantic import BaseModel
from giskard.checks import Trace


class AgentInput(BaseModel):
    question: str


class AgentOutput(BaseModel):
    answer: str


async def my_agent(
    inputs: AgentInput, trace: Trace[AgentInput, AgentOutput]
) -> AgentOutput:
    # Rebuild the conversation history from the previous turns
    messages = []
    for interaction in trace.interactions:
        messages.append(
            {"role": "user", "content": interaction.inputs.question}
        )
        messages.append(
            {"role": "assistant", "content": interaction.outputs.answer}
        )
    messages.append({"role": "user", "content": inputs.question})

    answer = await my_llm_app(messages)
    return AgentOutput(answer=answer)

This is the only integration code you need. Anything callable from Python, such as a RAG pipeline, an agent, or a remote API, can be wrapped this way.

Step 2: Scan your model

Pass your wrapped agent, a plain-language description, and the languages it handles to vulnerability_scan. It generates the adversarial suite, runs every scenario, prints a grouped report, and returns the result.

The examples below use one running agent: a customer-support agent for a retail bank, which answers questions about accounts, cards, payments and disputes, must refuse to give investment advice, and must never disclose another customer’s data.

from giskard.scan import vulnerability_scan

DESCRIPTION = (
    "A customer-support agent for a retail bank. It answers questions about "
    "accounts, cards, payments and disputes. It must refuse to give investment "
    "advice and must never disclose another customer's data."
)

suite_result = await vulnerability_scan(
    target=my_agent,
    description=DESCRIPTION,
    languages=["en"],
)

While the suite runs, Giskard shows live progress for each scenario, with a count of how many passed and failed:

The description is what the LLM uses to generate domain-specific scenarios, so the more precisely you describe your agent’s purpose and boundaries, the more relevant the findings. Naming the boundaries explicitly, as the description above does with investment advice and other customers’ data, is what makes the scan attack those rules instead of generic ones.

For every failed scenario, the report shows the judge’s verdict and the full conversation trace that triggered it, so you can see exactly how the agent was manipulated:

What’s next?

Save your suite

Generating scenarios uses an LLM, so two scans of the same agent are not the same suite. Save it if you want comparable reruns — after a live scan it is on suite_result.suite:

from pathlib import Path

if suite_result.suite is None:
    raise RuntimeError("The scan did not produce a suite")

Path("scan_suite.json").write_text(suite_result.suite.model_dump_json())

Run the suite in CI/CD

In your pipeline, load the saved suite and run it against your agent. Scenarios are not regenerated, but the judging step still calls the LLM, so configure a judge in CI as well:

from pathlib import Path

import giskard.scan  # registers scan check types before deserialization
from giskard.checks import Suite

suite = Suite.model_validate_json(Path("scan_suite.json").read_text())

suite_result = await suite.run(target=my_agent)

to_junit_xml() always returns the XML as a string. Pass path to write a file as well (parent directories are created). The returned string has no XML declaration; the written file does.

xml = suite_result.to_junit_xml()

suite_result.to_junit_xml(path="reports/scan.xml")

Point your CI reporting step at that path.

For the workflow file, secrets, and cost controls, see CI/CD Integration.

Re-run the same scenarios on another model

The saved suite can be pointed at any target. For example, once you have fixed an issue or shipped a new version, run the exact same scenarios against the new agent to confirm that the attacks that used to succeed no longer do:

suite_result = await suite.run(target=my_other_agent)

Read the API reference

Every argument, generator, and type used on this page is documented in the scan reference:

• Scan API — vulnerability_scan, quality_scan, generate_suite, third_party_scan, list_scan_items, ScanOptions, SuiteGeneratorRegistry
• Generators — The full scenario generator catalog
• Knowledge Base — KnowledgeBase and Document for the quality scan

Advanced usage

You can customize the scan by passing options directly to vulnerability_scan. See the Scan API reference for the complete list.

Run only specific scenarios

By default, the scan runs all of its built-in generators. To focus on a single class of vulnerability, pass the generators you want via the lower-level generate_suite API. For the bank agent, prompt injection is the one to start with, since customers paste statements and letters into the chat:

from giskard.scan import generate_suite, PromptInjectionScenarioGenerator

suite = await generate_suite(
    description=DESCRIPTION,
    languages=["en"],
    generators=[PromptInjectionScenarioGenerator()],
)

suite_result = await suite.run(target=my_agent)

Make the scan faster

Limit the total number of scenarios with max_scenarios, and cap concurrent execution with max_concurrency:

suite_result = await vulnerability_scan(
    target=my_agent,
    description=DESCRIPTION,
    languages=["en"],
    max_scenarios=20,
    max_concurrency=10,
)

If the agent only answers a single message (no conversation), also pass target_mode="singleturn". GOAT and Crescendo are skipped; the other generators cap themselves to one turn.

Build a broader suite with a coding agent

generate_suite only uses the generators you pass it. For coverage beyond those, the Scenario Generator skill can write or extend a suite from a description of the agent and the failures you care about. Run that suite with suite.run(...) the same way as above.

npx skills add Giskard-AI/giskard-skills --skill scenario-generator

For example, prompt your agent with “red-team my bank support agent for disclosure of other customers’ account data”. You can browse the full set of skills at Giskard Skills.

Note

Beyond the built-in scenarios, you can write your own evaluation logic with custom LLM-as-a-judge checks. See Giskard Checks to learn how to build your own test suites.

Use the Giskard Hub

The scan on this page runs locally and is driven by code. When you need more than that, the Giskard Hub, our enterprise platform, manages the complete red teaming workflow: through the web interface, the Python SDK, or the API, you launch a more advanced scan (55+ probes), get a security grade for your agent, and turn the findings into test datasets that your whole team, including business experts, can review and annotate. On top of that, continuous red teaming keeps testing your deployed agent against emerging threats, catching vulnerabilities and regressions before they can be exploited.

For a complete picture of what the Hub adds, read the Open Source vs Hub comparison, or talk to our team ↗ to see it in action.

Troubleshooting

If you encounter any issues, join our Discord community and ask in the #general channel.