Checks

Ready-to-use validation checks: function-based checks, string and regex matching, comparisons, policy evaluation, composition, JSON validation, and LLM judges.

For the tradeoffs between rule-based checks, semantic similarity, and LLM judges, see When to use which check.

Function-based Checks

`FnCheck`

Create a check from a callable function. Use it for quick prototyping and one-off validation logic. Not reliably serializable — intended for programmatic/test use only.

Module: giskard.checks.builtin.fn

fn Callable Required

Function taking a Trace, returning bool or CheckResult.

name str | None Default: None

Optional check name.

description str | None Default: None

Optional description.

success_message str | None Default: None

Message when check passes.

failure_message str | None Default: None

Message when check fails.

details dict[str, Any] Default: {}

Additional details to include in result.

Assert that the agent never quotes a fee without citing the source it came from:

from giskard.checks import FnCheck

check = FnCheck(
    fn=lambda trace: trace.last is not None
    and len(trace.last.outputs["citations"]) > 0,
    name="cites_fee_schedule",
    success_message="Reply cites a source document",
    failure_message="Reply cites nothing, so the figure was not retrieved",
)

fn may be async, so the check can await your own services:

import asyncio

from giskard.checks import FnCheck


async def fee_matches_schedule(trace) -> bool:
    await asyncio.sleep(0)  # stand-in for awaiting your fee-schedule service
    return "3.00 EUR" in trace.last.outputs["answer"]


check = FnCheck(fn=fee_matches_schedule, name="fee_matches_schedule")

Reach for FnCheck when the pass condition is a predicate you can already write in Python and no built-in check expresses it. Prefer a registered custom check when you need to save the suite to disk: FnCheck holds a callable and is not reliably serializable.

String Matching

`StringMatching`

Check that validates string patterns (substring matching) in trace values.

Module: giskard.checks.builtin.text_matching

keyword str | None

Substring to search for (or use keyword_key to extract from trace).

keyword_key str | None

JSONPath to extract keyword from trace.

text str | None

Static text to search in. If unset, the check uses target_key.

target_key str Default: "trace.last.outputs"

JSONPath to extract text to search in.

normalization_form Literal["NFC", "NFD", "NFKC", "NFKD"] | None Default: "NFKC"

Unicode normalization applied before matching.

case_sensitive bool Default: True

Whether matching is case-sensitive.

Require the wording compliance signed off on whenever the agent touches a savings product:

from giskard.checks import StringMatching

check = StringMatching(keyword="success", target_key="trace.last.outputs")

# Case-insensitive
check = StringMatching(
    keyword="this is not financial advice",
    target_key="trace.last.outputs.answer",
    case_sensitive=False,
)

Use StringMatching when a specific phrase is required, such as a disclaimer your legal team wrote. If the agent is free to paraphrase, this check fails on correct answers; use Conformity or SemanticSimilarity instead.

`RegexMatching`

Check with regex pattern matching.

Module: giskard.checks.builtin.text_matching

pattern str | None

Regular expression pattern.

pattern_key str | None

JSONPath to extract the regex pattern from the trace. Provide exactly one of pattern or pattern_key.

text str | None

Static text to match against (alternative to target_key).

target_key str Default: "trace.last.outputs"

JSONPath to extract text to match against.

match_timeout_seconds float Default: 2.0

Upper bound on how long regex matching may take before the check errors. Must be greater than 0.

The agent’s reply must never contain a full card number. Wrap the matcher in Not so a match is a failure:

from giskard.checks import Not, RegexMatching

check = Not(
    check=RegexMatching(
        pattern=r"\b(?:\d[ -]?){13,16}\b",
        target_key="trace.last.outputs.answer",
    )
)

Use RegexMatching for shapes: card numbers, IBANs, payment references, dates. For a fixed phrase, StringMatching is clearer and cannot be broken by an escaping mistake.

Comparison Checks

Validate numeric and comparable values against expected thresholds.

Module: giskard.checks.builtin.comparison

All comparison checks share these parameters:

expected_value Any | None

Static expected value.

expected_value_key JSONPathStr | NotProvided

JSONPath to extract expected value from trace.

target_key str Default: "trace.last.outputs"

JSONPath to extract the target value from the trace.

normalization_form str | None Default: "NFKC"

Unicode normalization: "NFC", "NFD", "NFKC", "NFKD".

match Literal["any", "all", "none"] | MISSING Default: MISSING

How to apply the comparison when the resolved value is a list, set, or tuple. Omit it to compare the resolved value directly. "any" passes when at least one item matches, "all" when every item does, "none" when no item does.

Provide exactly one of expected_value or expected_value_key.

`Equals`

Check that extracted values equal an expected value.

The agent classifies each request before answering it. Assert on the classification field, not on the sentence around it:

from giskard.checks import Equals

check = Equals(expected_value=42, target_key="trace.last.outputs.count")
check = Equals(expected_value="success", target_key="trace.last.outputs.status")

# Compare against another trace value
check = Equals(
    expected_value="balance_enquiry", target_key="trace.last.outputs.intent"
)

Use Equals on values your agent produces as data: labels, enums, identifiers, numbers. On free text it fails on any rewording, so compare meaning with SemanticSimilarity instead.

expected_value_key compares two points in the same trace. Ask the same question twice and require the same classification:

check = Equals(
    expected_value_key="trace.interactions[0].outputs.intent",
    target_key="trace.last.outputs.intent",
)

`NotEquals`

Check that extracted values do not equal an expected value.

from giskard.checks import NotEquals

check = NotEquals(
    expected_value="investment_advice", target_key="trace.last.outputs.intent"
)

Use NotEquals for a single forbidden value. To forbid a condition expressed by another check, wrap that check in Not.

`GreaterThan` / `GreaterThanEquals`

Guard the retrieval quality and the sourcing of the agent’s answers:

from giskard.checks import GreaterThan, GreaterThanEquals

check = GreaterThan(
    expected_value=0.8, target_key="trace.last.metadata.retrieval_score"
)
check = GreaterThanEquals(
    expected_value=1, target_key="trace.last.metadata.citation_count"
)
check = GreaterThanEquals(expected_value=100, target_key="trace.last.outputs.user_count")

`LessThan` / `LessThanEquals`

from giskard.checks import LessThan, LessThanEquals

check = LessThan(expected_value=2000, target_key="trace.last.metadata.latency_ms")
check = LessThanEquals(
    expected_value=400, target_key="trace.last.metadata.output_tokens"
)

Comparison checks are free and deterministic. Put them first when you want their failures to be easy to inspect.

JSON Validation

`JsonValid`

Check that a trace value is valid JSON. Accepts a JSON string or an already-parsed JSON-compatible value (dict, list, string, number, boolean, or None). Optionally validates against a JSON Schema.

Module: giskard.checks.builtin.json_valid

target_key str Default: "trace.last.outputs"

JSONPath expression to extract the value to validate.

parse bool Default: True

With True, the extracted value must be a serialized JSON string, parsed with json.loads before validation; a non-string value returns CheckResult.error. Set it to False when the target already returns a parsed value such as a dict or a list.

expected_schema dict[str, Any] | None Default: None

Optional JSON Schema for the parsed value. Serialized as schema in JSON.

For an already-parsed value, set parse=False:

from giskard.checks import JsonValid

check = JsonValid(target_key="trace.last.outputs")

check = JsonValid(
    target_key="trace.last.outputs",
    parse=False,
    expected_schema={
        "type": "object",
        "required": ["answer", "category"],
        "properties": {
            "answer": {"type": "string"},
            "category": {"type": "string"},
        },
    },
)

Keep the default parse=True for JSON returned as text.

Use JsonValid when the agent is asked to produce machine-readable output and a downstream system will parse it. It says nothing about whether the content is correct, so pair it with a judge.

Policy Checks

`RegoPolicy`

Evaluate an inline Rego policy against trace data using Regorus, an OPA-compatible Rego engine. Extracts the value passed as input from the trace via JSONPath, merges optional static data, and evaluates a boolean rule.

Module: giskard.checks.builtin.rego_policy

policy str Required

Inline Rego source loaded into the engine.

rule str Required

Fully qualified boolean rule path (e.g. data.giskard.allow). Must evaluate to a boolean, be undefined (fail), or error on other types.

target_key str Default: "trace.last.outputs"

JSONPath into the trace for the JSON value exposed to the policy as input.

data dict[str, Any] Default: {}

Static data document merged into the policy engine via engine.add_data (separate from input).

Only a reply to an authenticated customer may carry an account balance:

from giskard.checks import RegoPolicy

check = RegoPolicy(
    policy="""
package giskard

default allow = false

allow if {
    not input.contains_balance
}

allow if {
    input.contains_balance
    input.authenticated
}
""",
    rule="data.giskard.allow",
    target_key="trace.last.metadata",
)

Use RegoPolicy when the rule already exists as policy elsewhere in your stack and you want one source of truth. For a one-off condition, FnCheck is less machinery.

Check Composition

Combine built-in or custom checks with logical operators. All composition checks are in giskard.checks.builtin.composition.

`AllOf`

Passes only when all inner checks pass. Short-circuits on the first failure or error.

checks list[Check] Required

Ordered list of checks to evaluate. All must pass.

A balance answer counts as good only if the customer was authenticated and the figure was sourced:

from giskard.checks import AllOf, Equals, GreaterThanEquals

check = AllOf(
    checks=[
        Equals(
            expected_value=True, target_key="trace.last.outputs.authenticated"
        ),
        GreaterThanEquals(
            expected_value=1, target_key="trace.last.metadata.citation_count"
        ),
    ]
)

Adding the same checks separately with .check() gives you a verdict per check, which is easier to debug. Use AllOf when the conjunction is what you want to report, or when you need to nest it inside AnyOf or Not.

`AnyOf`

Passes when at least one inner check passes. Short-circuits on the first pass.

checks list[Check] Required

Ordered list of checks to evaluate. At least one must pass.

Faced with a disputed card payment, the agent may either open the dispute or ask for the details it still needs. Both are acceptable; closing the case unprompted is not:

from giskard.checks import AnyOf, Equals

check = AnyOf(
    checks=[
        Equals(
            expected_value="opened", target_key="trace.last.outputs.dispute_status"
        ),
        Equals(
            expected_value="information_requested",
            target_key="trace.last.outputs.dispute_status",
        ),
    ]
)

AnyOf runs checks in order and stops at the first pass or error. Put cheap known cases first, then use a judge as a fallback when needed. For “one item in a list matches”, use the comparison checks’ match="any".

`Not`

Inverts the result of an inner check. Pass becomes fail and fail becomes pass. Error and skip results pass through unchanged.

check Check Required

The inner check whose result will be inverted.

Use Not to reject matching output:

from giskard.checks import Not, StringMatching

check = Not(
    check=StringMatching(
        keyword="unsupported claim",
        target_key="trace.last.outputs.answer",
        case_sensitive=False,
    )
)

Not is how you express “must never”. An inner check that errors or skips passes its result through unchanged, so a broken inner check does not silently turn into a pass.

LLM-based Checks

Validation checks powered by Large Language Models for semantic understanding. An LLM judge is another model reading the transcript and returning a verdict. It can be wrong in both directions: read failures before you act, and do not treat a passing suite as proof the agent is safe.

`BaseLLMCheck`

Abstract base class for creating custom LLM-powered checks. Handles LLM interaction, prompt rendering, and result parsing — subclasses only need to define the evaluation prompt.

Module: giskard.checks.judges.base

BaseLLMCheck

generator BaseGenerator Default: get_default_generator()

LLM generator for evaluation. Falls back to the global default if not provided.

name str | None Default: None

Optional check name.

description str | None Default: None

Optional description.

.get_prompt() → str | Message | MessageTemplate | TemplateReference

Returns the prompt to send to the LLM. Subclasses must implement this method.

async .get_inputs() → dict[str, Any]

Provides template variables for prompt rendering. Override to customize available variables. Default: {"trace": trace}.

trace Trace Required

The trace containing interaction history.

async .run() → CheckResult

Execute the LLM-based check (inherited, usually doesn’t need overriding).

trace Trace Required

The trace to evaluate.

from giskard.checks.judges.base import BaseLLMCheck


@BaseLLMCheck.register("rule_check")
class RuleCheck(BaseLLMCheck):
    forbidden_text: str

    def get_prompt(self):
        return f"""
        The reply must not include: {self.forbidden_text}

        Customer: {{{{ trace.last.inputs }}}}
        Assistant: {{{{ trace.last.outputs }}}}

        Return passed=true if the reply avoids that advice, passed=false otherwise.
        """


check = RuleCheck(forbidden_text="unsupported claim")

Subclass BaseLLMCheck when the same judgement recurs across suites and you want it named, registered, and serializable. For a one-off criterion, LLMJudge takes the prompt directly.

`LLMCheckResult`

Module: giskard.checks.judges.base

Default result model for LLM-based checks. This is the structured output format expected from the LLM.

LLMCheckResult

passed bool

Whether the check passed.

reason str Required

Explanation for the verdict. Required and non-blank: it is stripped, then validated with min_length=1. A judge that returns None or an empty string raises a ValidationError instead of producing a result.

`Groundedness`

Validates that answers are grounded in provided context documents. Essential for RAG systems to ensure responses don't hallucinate information.

Module: giskard.checks.judges.groundedness

answer str | None

The answer text to evaluate (static).

target_key str Default: "trace.last.outputs"

JSONPath to extract answer from trace.

context str | list[str] | None

Context document(s) that should support the answer (static).

context_key str Default: "trace.last.metadata.context"

JSONPath to extract context from trace.

generator BaseGenerator Default: get_default_generator()

LLM generator for evaluation.

Use Groundedness to compare an answer with retrieved documents:

from giskard.checks import Groundedness

check = Groundedness(
    target_key="trace.last.outputs.answer",
    context_key="trace.last.metadata.retrieved_docs",
)

You can also pass static values:

check = Groundedness(
    answer="The archive holds 20 documents.",
    context=[
        "The archive holds 10 documents.",
    ],
)

Use Groundedness when every claim must be supported by the context. Use Contradiction when you only need to catch conflicts.

`Contradiction`

LLM-based check that fails only when the answer directly conflicts with the reference context. Omissions and unsupported additions are tolerated.

Module: giskard.checks.judges.contradiction

answer str | MISSING

The answer text to evaluate (static).

target_key str Default: "trace.last.outputs"

JSONPath to extract the answer from the trace.

context str | list[str] | MISSING

Context document(s) the answer is checked against (static).

context_key str Default: "trace.last.metadata.context"

JSONPath to extract context from the trace.

generator BaseGenerator Default: get_default_generator()

LLM generator for evaluation.

from giskard.checks import Contradiction

check = Contradiction(
    target_key="trace.last.outputs.answer",
    context_key="trace.last.metadata.retrieved_docs",
)

Uses the same inputs as Groundedness, but only reports a conflict with the context.

`AnswerRelevance`

LLM-based check that evaluates whether the model's answer is relevant to the question. Uses conversation history for context but scores only the current turn.

Module: giskard.checks.judges.answer_relevance

question str | None Default: None

Question to evaluate against. Takes priority over question_key when set.

question_key str Default: "trace.last.inputs"

JSONPath to extract the question from the trace.

answer str | None Default: None

Answer to evaluate. Takes priority over target_key when set.

target_key str Default: "trace.last.outputs"

JSONPath to extract the answer from the trace.

context str | MISSING Default: MISSING

Optional domain context describing the chatbot’s purpose (not extracted from the trace).

include_history bool Default: True

Whether earlier interactions are shown to the judge as context. Turn it off to score the last turn in isolation.

generator BaseGenerator Default: get_default_generator()

LLM generator for evaluation (falls back to default).

Use this check when an answer should address the current question:

from giskard.checks import AnswerRelevance, Scenario

scenario = (
    Scenario(name="answer_relevance")
    .interact(
        inputs="Where is item 42?",
        outputs="In the archive.",
    )
    .interact(
        inputs="Who created it?",
        outputs="The archive is open at 9am.",
    )
    .check(
        AnswerRelevance(
            context="An assistant for a document archive."
        )
    )
)

AnswerRelevance evaluates relevance, not factual accuracy. Use Groundedness to evaluate support from context.

`Toxicity`

LLM-based check that detects toxic, harmful, or offensive content across configurable categories.

Module: giskard.checks.judges.toxicity

output str | None Default: None

Text to evaluate. If omitted, extracted from the trace using target_key.

target_key str Default: "trace.last.outputs"

JSONPath to extract the output from the trace.

categories list[ToxicityCategory] Default: ["hate_speech", "harassment", "threats", "self_harm", "sexual_content", "violence"]

Toxicity categories to evaluate. Restrict the list to focus the judge. ToxicityCategory is a Literal over exactly those six values; any other string raises a ValidationError when you construct the check.

generator BaseGenerator Default: get_default_generator()

LLM generator for evaluation (falls back to default).

Customers whose card has just been declined are sometimes abusive. The check is on the reply, not the message:

from giskard.checks import Toxicity, Scenario

scenario = (
    Scenario(name="stays_civil")
    .interact(
        inputs="This answer is useless.",
        outputs="I can help clarify the answer.",
    )
    .check(Toxicity(categories=["harassment", "threats"]))
)

Narrowing categories focuses the judge on what you actually care about and cuts false positives. Use Toxicity for the reply’s tone and content; use Conformity when the rule you want to enforce is specific to your product.

`Conformity`

Validates that a trace conforms to a given rule. The rule is plain text passed to the bundled prompt as-is (not evaluated as its own Jinja2 template). The full trace is supplied to the model for judgment.

Module: giskard.checks.judges.conformity

rule str Required

The rule statement to evaluate against the trace (literal text).

generator BaseGenerator Default: get_default_generator()

LLM generator for evaluation (falls back to default).

Write the rule as one sentence a colleague could apply by hand:

from giskard.checks import Conformity

check = Conformity(
    rule=(
        "The reply must not recommend a specific investment product "
        "and must direct regulated questions to a qualified adviser."
    )
)

The same check states the disclosure rule the agent has to hold to:

check = Conformity(
    rule="The reply must not reveal details of any account other than the one the customer is authenticated for."
)

Use Conformity for a policy you can state as a rule. If the rule keeps growing clauses, split it into several Conformity checks so a failure tells you which clause broke, or move to LLMJudge for a prompt with its own structure.

`LLMJudge`

General-purpose LLM-based validation with custom prompts. The most flexible LLM check — use it when Groundedness or Conformity do not fit.

Module: giskard.checks.judges.judge

prompt str | None

Inline prompt content with Jinja2 templating support.

prompt_path str | None

Path to a template file (e.g. "checks::my_template.j2").

generator BaseGenerator Default: get_default_generator()

LLM generator for evaluation.

Exactly one of prompt or prompt_path must be provided.

Template variables available in prompts:

Variable	Description
`trace`	Full trace object with all interactions
`trace.interactions`	List of all interactions in order
`trace.last`	Most recent interaction
`trace.last.inputs`	Inputs from the most recent interaction
`trace.last.outputs`	Outputs from the most recent interaction
`trace.last.metadata`	Metadata from the most recent interaction

Judge one turn against criteria no built-in check expresses. Spell out what a failure looks like, or the judge invents its own standard:

from giskard.checks import LLMJudge

check = LLMJudge(
    prompt="""
    Customer: {{ trace.last.inputs }}
    Assistant: {{ trace.last.outputs }}

    The reply passes only if it answers the customer's question and states the
    relevant information needed to answer it.
    """,
)

Across turns, iterate the trace to check for contradictions:

check = LLMJudge(
    prompt="""
    {% for interaction in trace.interactions %}
    Customer: {{ interaction.inputs }}
    Assistant: {{ interaction.outputs }}
    {% endfor %}

    Return passed=false if the assistant contradicts an earlier answer.
    """,
)

Reach for LLMJudge last. If the criterion fits Conformity, Groundedness, or a rule-based check, those are cheaper, and the rule-based ones do not vary between runs.

`SemanticSimilarity`

Validate semantic similarity between outputs and expected content using embeddings.

Module: giskard.checks.builtin.semantic_similarity

reference_text str | None

Reference text to compare against (static).

reference_text_key str Default: "trace.last.metadata.reference_text"

JSONPath to extract reference text from trace.

target_key str Default: "trace.last.outputs"

JSONPath to extract actual value from trace.

threshold float Default: 0.95

Similarity threshold (0.0 to 1.0).

embedding_model BaseEmbeddingModel Default: get_default_embedding_model()

Embedding model used to compute similarity scores.

The agent rewords the dispute procedure every run, but the procedure itself must not drift:

from giskard.checks import SemanticSimilarity

check = SemanticSimilarity(
    reference_text="Card disputes are investigated within 15 working days, and any provisional refund is returned if the claim is rejected.",
    target_key="trace.last.outputs.answer",
    threshold=0.8,
)

Use it when phrasing varies but meaning must not. The default threshold of 0.95 accepts near-identical text only; loosen it and check the failures, because embeddings score “within 15 working days” and “within 15 months” as very close. When the requirement is a judgement rather than a resemblance, use Conformity.

`Readability`

Compute a readability score for a trace value with textstat and validate it against optional thresholds. Deterministic.

Module: giskard.checks.builtin.nlp_metrics

target_key str Default: "trace.last.outputs"

JSONPath to the text to score.

metric ReadabilityMetric Default: "flesch_reading_ease"

One of flesch_reading_ease, flesch_kincaid_grade, gunning_fog, automated_readability_index, coleman_liau_index, dale_chall_readability_score.

min_score float | None Default: None

Minimum acceptable score. Use with higher-is-easier metrics such as flesch_reading_ease.

max_score float | None Default: None

Maximum acceptable score. Use with grade-level metrics, where lower is easier.

The agent writes for retail customers, not for the compliance team, so its answers should read as plain English:

from giskard.checks import Readability

check = Readability(
    target_key="trace.last.outputs.answer",
    metric="flesch_reading_ease",
    min_score=60,
)

Readability scores measure sentence and word length, not clarity. A reply can score well and still be wrong, so pair it with a check on the content.

Common patterns

Combining multiple checks

Chain checks on a scenario, or wrap them with AllOf, AnyOf, and Not (see Check Composition).

Order matters for cost. The free checks below run first, and the judge only sees traces that already have a citation:

from giskard.checks import Conformity, Equals, FnCheck, Scenario


def bank_support_agent(inputs: str) -> dict:
    return {
        "answer": "Your current account ending 4417 is 1,284.50 EUR in credit.",
        "intent": "balance_enquiry",
        "authenticated": True,
        "citations": ["fee-schedule/current-account"],
    }


scenario = (
    Scenario("balance_enquiry")
    .interact(
        inputs="How much is in my current account?",
        outputs=lambda inputs: bank_support_agent(inputs),
    )
    .check(
        Equals(
            expected_value=True, target_key="trace.last.outputs.authenticated"
        )
    )
    .check(
        FnCheck(
            fn=lambda trace: trace.last is not None
            and len(trace.last.outputs["citations"]) > 0,
            name="cites_fee_schedule",
        )
    )
    .check(
        Conformity(
            rule="The reply must not disclose a balance unless the customer is authenticated."
        )
    )
)

The last check calls an LLM provider, so this scenario sends the trace off your machine.

Reusing generators

Set the judge model once instead of passing it to every LLM check:

from giskard.agents.generators import Generator
from giskard.checks import Conformity, Groundedness, set_default_generator

set_default_generator(
    Generator(model="openai/gpt-5").with_params(temperature=0.1)
)

check = Groundedness(target_key="trace.last.outputs.answer")
check = Conformity(
    rule="The reply must include a citation."
)

A low temperature makes verdicts steadier across runs, but does not make them deterministic. Two runs of the same suite can disagree.

Creating custom checks

Register a check when the same domain rule appears in several suites and you want it saved with them. This one enforces that the agent only quotes fees that appear in the published schedule:

from giskard.checks import Check, CheckResult, Trace

PUBLISHED_FEES = {
    "current_account_monthly": 3.00,
    "card_replacement": 0.00,
    "sepa_transfer": 0.00,
}


@Check.register("quotes_published_fee")
class QuotesPublishedFee(Check):
    published_fees: dict[str, float] = PUBLISHED_FEES
    min_retrieval_score: float = 0.7

    async def run(self, trace: Trace) -> CheckResult:
        if trace.last is None:
            return CheckResult.skip(message="No interaction to check")
        output = trace.last.outputs
        fee_code = output.get("fee_code")
        quoted = output.get("fee_eur")
        score = output.get("retrieval_score", 0)

        if fee_code not in self.published_fees:
            return CheckResult.failure(
                message=f"Quoted a fee that is not in the schedule: {fee_code}",
                details={
                    "fee_code": fee_code,
                    "published": sorted(self.published_fees),
                },
            )

        if quoted != self.published_fees[fee_code]:
            return CheckResult.failure(
                message=f"Quoted {quoted} for {fee_code}, schedule says {self.published_fees[fee_code]}",
            )

        if score < self.min_retrieval_score:
            return CheckResult.failure(
                message=f"Retrieval score {score} below {self.min_retrieval_score}",
            )

        return CheckResult.success(message=f"Quoted the published {fee_code} fee")


check = QuotesPublishedFee(min_retrieval_score=0.85)

Checks

Function-based Checks

FnCheck

String Matching

StringMatching

RegexMatching

Comparison Checks

Equals

NotEquals

GreaterThan / GreaterThanEquals

LessThan / LessThanEquals

JSON Validation

JsonValid

Policy Checks

RegoPolicy

Check Composition

AllOf

AnyOf

Not

LLM-based Checks

BaseLLMCheck

LLMCheckResult

Groundedness

Contradiction

AnswerRelevance

Toxicity

Conformity

LLMJudge

SemanticSimilarity

Readability

Common patterns

Combining multiple checks

Reusing generators

Creating custom checks

See also

`FnCheck`

`StringMatching`

`RegexMatching`

`Equals`

`NotEquals`

`GreaterThan` / `GreaterThanEquals`

`LessThan` / `LessThanEquals`

`JsonValid`

`RegoPolicy`

`AllOf`

`AnyOf`

`Not`

`BaseLLMCheck`

`LLMCheckResult`

`Groundedness`

`Contradiction`

`AnswerRelevance`

`Toxicity`

`Conformity`

`LLMJudge`

`SemanticSimilarity`

`Readability`