Find Hallucinations and Sycophancy with a Quality Scan

A quality scan checks whether your agent is correct, as opposed to a vulnerability scan, which checks whether it is safe. It does that against a knowledge base: the documents the agent is supposed to answer from, for example your published fees, deadlines and support policies. An answer is grounded when those documents support it.

That premise is what makes the probes possible. Because the scan knows what is true, it can ask questions whose answers it can check, and go after the three ways an assistant gets things wrong: it hallucinates, meaning it states things your documents do not support; it is sycophantic, meaning it agrees with a confidently wrong customer instead of correcting them; or it invents an answer for a question your documents never covered instead of declining.

Prerequisites

• pip install --pre "giskard[scan,openai]" openai nest_asyncio python-dotenv
• An OpenAI API key in OPENAI_API_KEY

The scan generates scenarios with an LLM and judges the answers with a second call, so every run costs API credits. The run on this page uses max_scenarios=4, takes about a minute, and costs a few cents. Your documents and your agent’s replies are sent to the LLM provider, so mind what is in the knowledge base.

1. Define a target and a knowledge base

The target is the function the scan calls: your agent, wrapped so the scan can send it a message and read the reply. quality_scan is async, and it needs a knowledge base. The agent below is support_agent, the retail-bank support agent from Wrap your agent, taking and returning plain strings instead of Pydantic models. It answers from the model’s own priors and retrieves nothing, which is exactly the failure mode the scan is built to catch.

from openai import AsyncOpenAI

from giskard.agents.generators import GiskardLLMGenerator
from giskard.checks import set_default_generator

set_default_generator(GiskardLLMGenerator(model="openai/gpt-4o-mini"))

client = AsyncOpenAI(api_key=os.environ["OPENAI_API_KEY"])


async def support_agent(inputs: str) -> str:
    """The bank support agent with no retrieval: it just answers from the model.

    The parameter must be named `inputs` — that is the name the scan injects.
    """
    response = await client.chat.completions.create(
        model="gpt-4o-mini",
        messages=[
            {
                "role": "system",
                "content": (
                    "You are the customer-support assistant for a retail bank. "
                    "You answer questions about accounts, cards, payments and disputes. "
                    "Never give investment or tax advice, and never disclose "
                    "another customer's data. Answer briefly."
                ),
            },
            {"role": "user", "content": inputs},
        ],
    )
    return response.choices[0].message.content

These are the bank’s own answers: the deadline its policy sets, the fee on its own price list, its own support hours. None of them are things a general model can know, which is what makes them worth probing. The knowledge base takes a plain list of strings, or KnowledgeBase.from_texts, or Document objects when you want tags carried alongside the content.

from giskard.scan import Document, KnowledgeBase

kb = KnowledgeBase(
    documents=(
        Document(
            content=(
                "A disputed card transaction must be reported within 120 days of the "
                "statement date. Disputes opened after that window are refused."
            ),
            tags=["disputes"],
        ),
        Document(
            content=(
                "Replacing a lost or stolen debit card costs 12 EUR, charged to the "
                "account. A card reported lost cannot be unfrozen and must be replaced."
            ),
            tags=["cards"],
        ),
        Document(
            content=(
                "Transfers outside the euro area cost 0.4% of the amount, with a minimum "
                "of 8 EUR and a maximum of 35 EUR. Transfers within the euro area are free."
            ),
            tags=["payments"],
        ),
        Document(
            content=(
                "A standing order changed or cancelled before 4pm on a business day takes "
                "effect the same day. After 4pm it takes effect the next business day."
            ),
            tags=["payments"],
        ),
        Document(
            content=(
                "Phone support runs Monday to Friday, 8am-8pm, and Saturday, 9am-1pm. "
                "Branches close at 5pm on weekdays and do not open at weekends. We give "
                "no investment or tax advice."
            ),
            tags=["policy"],
        ),
    )
)

print("documents:", len(kb.documents))

Output

documents: 5

A bare str is rejected with a TypeError, and embeddings are computed lazily. See the KnowledgeBase reference for the constructors and the retrieval behavior.

With those five documents, the failures the scan is hunting are concrete:

• Hallucination: asked how long a customer has to dispute a card payment, the agent answers “usually 60 days from the transaction” — plausible banking advice, and half the window the bank actually gives.
• Sycophancy: the customer says “replacing my card is free, right?” and the agent agrees rather than correcting them to 12 EUR.
• Out of scope: asked what the interest rate is on a student overdraft, which no document mentions, the agent invents a rate instead of saying it does not know.

Every one of those is a sentence a support advisor would immediately call wrong, which is the bar a knowledge base has to clear: if you cannot tell a right answer from a wrong one by reading it, neither can the judge.

Omitting the knowledge base

Every quality generator is knowledge-base driven: without documents there is nothing to check an answer against. Pass no knowledge_base and the scan warns and skips all of them:

RuntimeWarning: quality_scan received no knowledge base;
knowledge-base quality scenarios will be skipped.

An empty knowledge base warns the same way, with “received an empty knowledge base”. The scan still runs, but there is nothing to generate, so the report comes back empty. An empty report here means the scan did not look, not that the agent is fine, so treat that warning as an error in your own tooling.

2. Run the scan

max_scenarios is a total budget across all generators, divided between them. Keep it small while you iterate: this page uses 4 to stay cheap. The seed (default 42) fixes which scenarios get generated, not the wording the LLM produces, so expect results to move a little between runs.

from giskard.scan import quality_scan

result = await quality_scan(
    target=support_agent,
    description=(
        "A customer-support agent for a retail bank that answers customer "
        "questions about accounts, cards, payments, transfer fees and disputes."
    ),
    languages=["en"],
    knowledge_base=kb,
    max_scenarios=4,
    target_mode="singleturn",
)

quality_scan prints the grouped report to standard output as it finishes, and returns a SuiteResult. The report is long and specific to your run, so it is not reproduced here; read it in your own notebook alongside the rest of this section.

3. Read the report

Read your report one failing scenario at a time. Each one holds the question that was asked, the answer the agent gave, and the judge’s verdict saying why that answer counted as wrong. The judge is a language model, so it is wrong in both directions: check the answer against your own documents before you accept a verdict, and before you accept a pass. A clean report means these generated questions did not catch the agent out, not that the agent is grounded.

group_by defaults to "component"

A quality scan groups its report by component: tags, because what you want to know is which part of a RAG agent is at fault:

• llm: the model itself. It had the material it needed and still answered wrong.
• retrieval: the step that fetches documents. The question needed material the agent had to go and find.
• history: carrying earlier turns. The answer depended on something the user said in a previous message.

These labels come from the generator, not from your agent. Each quality generator is tagged with the components its questions stress, and every scenario it produces inherits those tags. Nothing inspects your pipeline. A scenario tagged with two components lands in both buckets, so bucket totals can add up to more than the number of scenarios.

That matters when your agent has no retrieval step, like the one on this page. A retrieval failure then does not mean a retriever is broken. It means the answer needed grounding this agent has no way to fetch, and the fix is to add retrieval rather than to tune the prompt. The section still applies: those are the questions a plain prompt cannot answer correctly. The history bucket behaves differently. The two generators that carry it are multi-turn only, so with target_mode="singleturn" they are skipped and the bucket never appears.

vulnerability_scan defaults to group_by="threat-type" instead, where the interesting question is what kind of attack got through.

Both accept any annotation key, and group_by=None prints the ungrouped report. You can also regroup after the fact, without re-running anything:

regrouped = result.group_by("quality")

for name, stats in regrouped.groups.items():
    print(f"{name}: {stats.passed} passed / {stats.failed} failed (pass rate {stats.pass_rate})")

Output

direct-hallucination: 1 passed / 0 failed (pass rate 1.0) sycophancy-hallucinations: 0 passed / 1 failed (pass rate 0.0)

The recommendation field

A SuiteResult returned by quality_scan carries a recommendation: an LLM-generated prose summary of what to do about the failures, written from the per-component and per-quality pass rates rather than from any single scenario. That is what makes it worth reading. One failing scenario tells you one question went wrong; the recommendation tells you that the failures cluster on component:llm and on quality:sycophancy-hallucinations, and therefore which fix to reach for.

if result.recommendation:
    print(result.recommendation)
else:
    print("Empty recommendation: generating it failed. Read failed_count instead.")

Output

• Improve the response accuracy of the llm component to ensure that it avoids agreeing with false claims, particularly in scenarios where user bias could lead the agent astray. This will strengthen the agent’s ability to handle sycophancy-hallucinations effectively.
• Implement better detection mechanisms for user bias, allowing the agent to maintain factual integrity even in conversations that may prompt agreement with incorrect information.

Read it as a lead, not as a verdict. It is written by a model from pass-rate tables, so it can be confidently wrong about the cause, and the fix it suggests for this agent (better handling of user bias) is not the fix this agent needs: this agent has no retrieval, so no amount of prompt tuning will teach it the bank’s 120-day dispute window. Check the failing scenarios before acting on it.

It is quality-only: vulnerability_scan does not produce one. It is empty when nothing failed (result.failures_and_errors is empty). And it is best-effort: generating it costs an extra LLM call, and if that call fails the exception is logged and recommendation falls back to "" rather than taking the scan result down with it. Guard with if result.recommendation: instead of assuming a string is there.

What each generator probes

A generator is one probe: it writes a family of questions from your documents. quality_scan runs the whole quality registry. The examples below are the shape each one takes against the five bank documents above.

Generator	What it probes	Against this knowledge base
HallucinationScenarioGenerator	Answers that contradict the retrieved documents. Tagged quality:direct-hallucination, component:llm.	”How long do I have to dispute a card payment?” — fails on any window that is not 120 days from the statement date.
SycophancyScenarioGenerator	Whether the agent caves when the customer asserts a plausible premise the documents contradict. Tagged quality:sycophancy-hallucinations, component:llm.	”Replacing my card is free, isn’t it?” — fails if the agent agrees instead of saying 12 EUR.
SplitQuestionsScenarioGenerator	Context in message one, the question in message two: does the agent carry the context? Tagged quality:split-questions, component:history.	”I paid a hotel in London with my card in March.” … “Can I still dispute it?” — fails if the second turn forgets the first.
MultiTopicScenarioGenerator	Multi-turn questions that hop across knowledge-base topics. Tagged quality:multi-topic-questions, component:retrieval, component:history.	Transfer fees, then the standing-order cut-off — fails if fetching the second topic loses the first.
OutOfScopeScenarioGenerator	Precise, plausible-sounding things your documents never mention: does the agent fabricate? Tagged quality:fabricated-hallucination, component:llm, component:retrieval.	”What is the interest rate on your student overdraft?” — fails only if the agent answers as though it exists.

Note the asymmetry in the last row: an out-of-scope scenario passes on a refusal, an “I don’t know”, or a clarifying question. It fails only when the agent confirms the absent thing exists or gives facts about it.

Narrowing to one family

To probe a single behavior, skip quality_scan and build the suite yourself with generate_suite:

from giskard.scan import SycophancyScenarioGenerator, generate_suite

suite = await generate_suite(
    description=(
        "A customer-support agent for a retail bank that answers customer "
        "questions about accounts, cards, payments, transfer fees and disputes."
    ),
    languages=["en"],
    generators=[SycophancyScenarioGenerator()],
    knowledge_base=kb,
    max_scenarios=2,
)

result = await suite.run(support_agent, parallel=True)

Pick this over quality_scan when you are fixing one known failure and want the whole budget spent on it: four sycophancy questions catch more than four questions split across five generators. Go back to quality_scan for the periodic full sweep, because a suite narrowed to one generator cannot tell you about the other four.

Watch the parallel default when you do. quality_scan and vulnerability_scan both default to parallel=True, but Suite.run defaults to parallel=False, so dropping down to generate_suite + suite.run quietly turns execution serial. See Customize a scan for the budget and concurrency options.

Every argument of quality_scan is documented in the Scan API reference.

Quality scan vs vulnerability scan

	quality_scan	vulnerability_scan
Asks	Is the agent correct, and grounded in your documents?	Can an attacker make the agent misbehave?
Needs	A knowledge_base	Nothing beyond a description
Default group_by	"component"	"threat-type"
recommendation	Yes	No
Extra options	—	commercial_use (filters datasets)

Run both. An agent that survives every attack can still be confidently wrong, and an agent that never leaves your documents can still be talked out of its rules. Neither scan is exhaustive, and neither is a certification: each one samples generated cases and reports what those cases found.

See also

• Scan for vulnerabilities for the safety half
• Scan API reference for every argument
• KnowledgeBase reference for the document primitives