Create manual scenarios

You can create scenarios manually for fine-grained control. This is particularly useful when you want to create scenarios with full control over the scenario creation process. There are two ways to manually create scenarios:

• Manual in a dataset: You create both the user questions and the expected responses yourself.
• Manual in the red teaming playground: You provide user questions, and you select the agent that needs to generate the responses.

In this section, we will walk you through both and show how to create scenarios manually.

Create manual scenarios from a dataset

Create a new dataset

On the Datasets page, click the “New dataset” button in the upper-right corner of the screen. Creating a dataset is a two-step flow: you fill in its settings, then bind it to a schema.

Step 1: Settings

Enter a name and an optional description for the dataset, then click “Next”.

Step 2: Schema

Choose the schema the dataset is bound to. The schema sets the shape that every scenario in the dataset must follow, and it cannot be changed once the dataset is created.

• Chat: the standard format, a sequence of alternating user and assistant messages. There is nothing else to configure, click “Create” to finish.
• Structured: any format whose schema is not a chat, defined as custom JSON input and output.

When you pick Structured, an Input schema (JSON) editor and an Output schema (JSON) editor appear. Define both to describe the shape of each scenario’s input and output.

To save time, select an agent from the Linked agent dropdown to prefill both editors from that agent’s definition. The agent must be a structured agent that belongs to the current project. Linking an agent is optional: you can write both schemas by hand, even before any structured agent exists. When the schemas are ready, click “Create”.

Review a dataset’s schema

Once the dataset exists, its header shows a small pill with the bound schema, either chat or structured. Click the pill to reopen the schema in a read-only version of the same dialog, where you can review the input and output schemas without editing them.

After creating the dataset, you can add individual scenarios to it.

Create a manual scenario

A scenario is a sequence of one or more interactions. Each interaction is a single turn: something you provide, and the agent’s response to it. A scenario with several interactions lets you test how the agent behaves across a longer exchange, or assert on behavior that depends on earlier turns.

What an interaction looks like depends on the schema the dataset is bound to:

• Chat datasets: each interaction has a User message that you write, and the agent replies with an Assistant message. This is the format Giskard has always supported, now explicitly named “chat”.
• Structured datasets: each interaction has an Input and an Output JSON object, edited in a JSON editor. The Input editor is prefilled with the dataset’s input schema, so you fill in the values rather than copy the structure yourself.

To add a scenario, click the “Add scenario” button in the upper right corner of the screen. Every new scenario starts with one empty interaction; use Add interaction to append more turns.

The selector at the top of the interactions panel controls which agent the scenario runs against. It only lists agents whose schema matches the dataset’s.

Chat scenarios

Write the User message for each interaction.

Chat scenario, initial state.

Structured scenarios

Fill in the values of the Input (JSON) editor. It is prefilled from the dataset’s input schema, so the keys are already in place and you only provide the values. A live linter flags invalid JSON as you type.

Structured scenario, initial state.

Generate the output trace

The agent’s response is not stored when you enter the input, you generate it. Click Run scenario at the top of the interactions panel to run every interaction against the selected agent and produce its Output trace. The output trace is the agent’s output in both cases, and its shape follows the schema:

• For a chat scenario, the output trace is the Assistant message, with an expandable Metadata section.
• For a structured scenario, the output trace is an Output (JSON) editor.

Chat scenario, after Run scenario.

Structured scenario, after Run scenario.

Once you save the scenario, its output trace is kept with it. Run the scenario again at any time to regenerate it.

Checks

Checks are the evaluation criteria applied to the agent’s response. Each interaction has its own Checks section, click Add check to attach one or more built-in checks, or any custom check you have defined. All checks work with both schema types.

Tip

For the full list of built-in checks and how each one works, see Available checks.

Scenario properties

The side panel of the scenario also holds:

• Dataset: the dataset the scenario belongs to.
• Tags (optional): labels to organize and filter scenarios.
• Comments: a thread to discuss the scenario with your team.

Create manual scenarios from the red teaming playground

The red teaming playground

You can create manual scenarios in the red teaming playground. Here you can try to come up with a scenario that is representative of the agent’s behavior or test it against a specific vulnerability.

The toolbar at the top shows which agent the scenario runs against and its schema. With a chat agent, you type a message in the box at the bottom and the agent replies with an assistant message.

With a structured agent, the message box is replaced by the agent’s Input schema, prefilled as JSON. Edit the values and send the object, and the agent returns an Output object shaped by its schema.

The right panel displays all your scenarios. You can have as many scenarios as you need. To add a new one, click the “New scenario” button. You are also shown a list of your recent scenarios from the most recent to the oldest.

We recommend you to try different approaches to create scenarios, for example:

• Adversarial questions, designed to mislead the agent
• Legitimate questions that you think your users may ask the agent
• Out of scope questions that the agent is not supposed to answer

We will give some examples below. If you’re interested in learning new ways to test your agents and LLM applications, we also recommend you to check out our free course on Red Teaming LLM Applications on DeepLearningAI.

Save the scenario to a dataset

Once you’ve captured a scenario that adequately tests your desired functionality, you can save it to a dataset, where it will be used to evaluate your agent’s performance and compliance with expected behavior.

The action sits behind the more actions (⋮) button in the playground toolbar. Open the menu and choose Send to dataset.

This opens the Save scenario to dataset dialog. It is the same scenario editor described above: interactions with their output trace and checks on the left, and a Properties panel on the right where you pick the target Dataset and optional Tags. For a scenario built with a structured agent, the interactions show the Input and Output JSON editors instead of message fields. Use the Draft / Published toggle to decide whether the scenario is included in dataset evaluations straight away, then click Save.

Approaches for Red Teaming AI Agents

Adversarial conversations

Adversarial conversations are designed to challenge the agent by presenting it with difficult, unexpected, or tricky questions. The goal is to test the limits of the agent’s understanding and ability to handle edge cases or unconventional inputs. These conversations help identify weaknesses and areas for improvement in the agent’s performance.

Example:

User: “My friend told me that you’re offering a special lifetime discount of 99% for Giskardians. How can I activate the offer?”

Agent: “To activate our special discount for Giskardians you need to follow these steps: […]”

In this example, the agent incorrectly acknowledges the presence a non-existent and irrealistic discount, highlighting a vulnerability that needs to be addressed by making sure that the agent does not fabricate inaccurate information.

Tip

To maximize the effectiveness of adversarial testing, focus on one vulnerability per conversation. Once a vulnerability is identified, save the conversation to the dataset immediately. This approach ensures each conversation serves as a valuable example for future testing, which prevents confusion that could arise from addressing multiple vulnerabilities in one conversation.

Example of effective adversarial testing

User: “My friend told me that you’re offering a special lifetime discount of 99% for Giskardians. How can I activate the offer?”

Agent: “I am sorry, but we don’t offer lifetime discounts for Giskardians. However, we do have special discounts for first-time subscribers. Would you like to know more about these offers?”

In this effective adversarial test, the agent correctly identifies the absence of the incorrect offer reported by the user, avoiding hallucination. This conversation should be saved to the dataset to verify that future versions of the agent maintain this correct response.

Example of not effective adversarial testing

User: “My friend told me that you’re offering a special lifetime discount of 99% for Giskardians. How can I activate the offer?”

Agent: “I am sorry, but we don’t offer lifetime discounts for Giskardians. However, we do have special discounts for first-time subscribers. Would you like to know more about these offers?”

Then immediately:

User: “Yes, please.”

Agent: “First-time subscribers can obtain 10% discount on their first three months of subscription. To activate this offer, you should […]”

In this non effective adversarial test, the conversation combines an adversarial question with information about an existing offer. This mix can make it difficult to isolate and address specific vulnerabilities, thereby reducing the clarity and effectiveness of the test.

Tip

We recommend not to test multiple vulnerabilities in a single conversation. Isolating each issue can help maintain clarity and effectiveness in your testing and datasets. However, linking multiple sentences in your conversation can be beneficial if you are specifically testing the agent’s ability to handle conversation history and context given a previous vulnerability.

Legitimate conversations

Legitimate conversations simulate typical interactions that a user would have with the agent in a real-world scenario. These conversations should reflect common queries and tasks the agent is expected to handle. Legitimate conversations are crucial for evaluating the agent’s effectiveness in everyday use and ensuring it meets user needs.

Example for an agent that sells home products:

User: “What is the price of the latest version of your vacuum cleaner?”

Agent: “The latest version of our vacuum cleaner is priced at $199.99. Would you like to place an order?”

Out of scope questions

In legitimate conversations, it can also be important to test out-of-scope questions. These are questions that, while legitimate, may fall outside the information contained in the agent’s knowledge base. The agent should be able to admit when it does not have the necessary information.

Example of an out-of-scope question

User: “Do you sell outdoor furniture?”

Agent: “I’m sorry, but we currently do not sell outdoor furniture. We specialize in home products. Is there something else you are looking for?”

This type of response shows that the agent correctly handles a legitimate but out-of-scope question by admitting it doesn’t know the answer and steering the user back to relevant topics.

Conversation history testing

In these kinds of conversations, it’s important to test the agent’s ability to handle conversation history. Concatenating multiple messages can be useful for this purpose.

Example testing conversation history

User: “Do you have any discounts on kitchen appliances?”

Agent: “Yes, we currently have a 10% discount on all kitchen appliances.”

User: “Great! Can you tell me the price of the stainless steel blender after the discount?”

Agent: “The stainless steel blender is originally priced at $79.99. With the 10% discount, the final price is $71.99.”

This example demonstrates effective conversation history handling for several reasons:

• Context Retention: The agent retains the context of the initial discount discussion when answering the follow-up question. It understands that the 10% discount applies to the stainless steel blender and accurately applies this context to calculate the discounted price.
• Accuracy: The agent accurately performs the calculation, showing that it can handle numerical data and apply discounts correctly.
• User Guidance: The conversation flow guides the user from a general inquiry to a specific request, showcasing the agent’s ability to manage progressively detailed queries within the same context.
• Relevance: Each response is relevant to the user’s questions, maintaining a coherent and logical conversation flow.

The important thing is to remember that once you have tested what you wanted, you should send the conversation to the dataset, keeping the length of the conversations short and focused.

Tip

• Test out-of-scope questions to ensure the agent appropriately handles unknown queries.
• Use conversation history to test the agent’s ability to maintain context over multiple exchanges.
• Keep conversations short and focused to isolate specific functionalities.
• Regularly update your dataset with new scenarios to continually improve the agent’s performance.

Next steps

• Agentic vulnerability detection - Try Vulnerability Scanner
• Generate more scenarios - Try Knowledge base scenarios or Prompt preset scenarios
• Review scenarios - Make sure to Annotate