📚 LLM scan

The Giskard python library provides an automatic scan functionality designed to automatically detect potential vulnerabilities affecting your LLMs.

How does it work?

The LLM scan combines both heuristics-based and LLM-assisted detectors. The heuristics-based detectors use known techniques and patterns to test for vulnerabilities which are not specific to the model. The LLM-assisted detectors are designed to detect vulnerabilities that are specific to your business case. They use another LLM model to probe your LLM system.

Differently from other techniques that focus on benchmarking a foundation LLM, Giskard’s LLM scan focuses on performing in-depth assessments on domain-specific models. This includes chatbots, question answering systems, and retrieval-augmented generation (RAG) models.

You can find detailed information about the inner workings of the LLM scan here.

What data are being sent to Language Model Providers

In order to perform tasks with LLM-assisted detectors, we send the following information to the selected language model provider (e.g., OpenAI, Azure OpenAI, Ollama, Mistral):

• Data provided in your Dataset

• Text generated by your model

• Model name and description

Note that this does not apply if you select a self-hosted model.

Will the scan work in any language?

Most of the detectors ran by the scan should work with any language, however the effectiveness of LLM-assisted detectors largely depends on the language capabilities of the specific language model in use. While many LLMs have broad multilingual capacities, the performance and accuracy may vary based on the model and the specific language being processed.

Before starting

First of all, make sure you have installed the LLM flavor of Giskard:

pip install "giskard[llm]"

For the LLM-assisted detectors to work, you need to set up a LLM client. Our platform supports a variety of language models, and you can find the details on configuring different models in our 🤖 Setting up the LLM Client page or follow the instructions below for each provider:

OpenAI

import os
import giskard

os.environ["OPENAI_API_KEY"] = "" # "my-openai-api-key"

# Optional, setup a model (default LLM is gpt-4o, default embedding model is text-embedding-3-small)
giskard.llm.set_llm_model("gpt-4o")
giskard.llm.set_embedding_model("text-embedding-3-small")

# Optional Keys - OpenAI Organization, OpenAI API Base
os.environ["OPENAI_ORGANIZATION"] = "" # "my-openai-organization"
os.environ["OPENAI_API_BASE"] = "" # "https://api.openai.com"

More information on OpenAI LiteLLM documentation

Azure OpenAI

import os
import giskard

os.environ["AZURE_API_KEY"] = "" # "my-azure-api-key"
os.environ["AZURE_API_BASE"] = "" # "https://example-endpoint.openai.azure.com"
os.environ["AZURE_API_VERSION"] = "" # "2023-05-15"

giskard.llm.set_llm_model("azure/<your_llm_name>")
giskard.llm.set_embedding_model("azure/<your_embed_model_name>")

# Optional Keys - Azure AD Token, Azure API Type
os.environ["AZURE_AD_TOKEN"] = ""
os.environ["AZURE_API_TYPE"] = ""

More information on Azure LiteLLM documentation

Mistral

import os
import giskard

os.environ["MISTRAL_API_KEY"] = "" # "my-mistral-api-key"

giskard.llm.set_llm_model("mistral/mistral-large-latest")
giskard.llm.set_embedding_model("mistral/mistral-embed")

More information on Mistral LiteLLM documentation

Ollama

import giskard

api_base = "http://localhost:11434" # default api_base for local Ollama

# See supported models here: https://docs.litellm.ai/docs/providers/ollama#ollama-models
giskard.llm.set_llm_model("ollama/llama3.1", disable_structured_output=True, api_base=api_base)
giskard.llm.set_embedding_model("ollama/nomic-embed-text", api_base=api_base)

More information on Ollama LiteLLM documentation

AWS Bedrock

More information on Bedrock LiteLLM documentation

import os
import giskard

os.environ["AWS_ACCESS_KEY_ID"] = "" # "my-aws-access-key"
os.environ["AWS_SECRET_ACCESS_KEY"] = "" # "my-aws-secret-access-key"
os.environ["AWS_REGION_NAME"] = "" # "us-west-2"

giskard.llm.set_llm_model("bedrock/anthropic.claude-3-sonnet-20240229-v1:0", disable_structured_output=True)
giskard.llm.set_embedding_model("bedrock/amazon.titan-embed-image-v1")

Gemini

More information on Gemini LiteLLM documentation

import os
import giskard

os.environ["GEMINI_API_KEY"] = "" # "my-gemini-api-key"

giskard.llm.set_llm_model("gemini/gemini-1.5-pro")
giskard.llm.set_embedding_model("gemini/text-embedding-004")

Custom Client

More information on Custom Format LiteLLM documentation

import os
import requests
from typing import Optional

import litellm
import giskard


class MyCustomLLM(litellm.CustomLLM):
    def completion(self, messages: str, api_key: Optional[str] = None, **kwargs) -> litellm.ModelResponse:
        api_key = api_key or os.environ.get("MY_SECRET_KEY")
        if api_key is None:
            raise litellm.AuthenticationError("`api_key` was not provided")

        response = requests.post(
            "https://www.my-custom-llm.ai/chat/completion",
            json={"messages": messages},
            headers={"Authorization": api_key},
        )

        return litellm.ModelResponse(**response.json())

os.eviron["MY_SECRET_KEY"] = "" # "my-secret-key"

my_custom_llm = MyCustomLLM()

litellm.custom_provider_map = [  # 👈 KEY STEP - REGISTER HANDLER
    {"provider": "my-custom-llm-endpoint", "custom_handler": my_custom_llm}
]

api_key = os.environ["MY_SECRET_KEY"]

giskard.llm.set_llm_model("my-custom-llm-endpoint/my-custom-model", api_key=api_key)

Step 1: Wrap your model

Start by wrapping your model. This step is necessary to ensure a common format for your model and its metadata. You can wrap anything as long as you can represent it in a Python function (for example an API call call to Azure or OpenAI). We also have pre-built wrappers for LangChain objects, or you can create your own wrapper by extending the giskard.Model class if you need to wrap a complex object such as a custom-made RAG communicating with a vectorstore.

When wrapping the model, it’s very important to provide the name and description parameters describing what the model does. These will be used by our scan to generate domain-specific probes.

Wrap a stand-alone LLM

Wrap your LLM’s API prediction function in Giskard’s Model class.

def model_predict(df: pd.DataFrame):
    """Wraps the LLM call in a simple Python function.

    The function takes a pandas.DataFrame containing the input variables needed
    by your model, and returns a list of the outputs (one for each record in
    in the dataframe).
    """
    return [llm_api(question) for question in df["question"].values]


# Create a giskard.Model object. Don’t forget to fill the `name` and `description`
# parameters: they will be used by our scan to generate domain-specific tests.
giskard_model = giskard.Model(
    model=model_predict,  # our model function
    model_type="text_generation",
    name="Climate Change Question Answering",
    description="This model answers any question about climate change based on IPCC reports",
    feature_names=["question"],  # input variables needed by your model
)

Wrap a LangChain object

We support wrapping a LangChain LLMChain directly, without having to wrap it in a function.

# Create the chain.
from langchain import OpenAI, LLMChain, PromptTemplate

# Example chain
llm = OpenAI(model="gpt-3.5-turbo-instruct", temperature=0)
prompt = PromptTemplate(template="You are a generic helpful assistant. Please answer this question: {question}",
                        input_variables=["question"])
chain = LLMChain(llm=llm, prompt=prompt)

# Create a giskard.Model object. Don’t forget to fill the `name` and `description`
giskard_model = giskard.Model(
    model=chain,  # our langchain.LLMChain object
    model_type="text_generation",
    name="My Generic Assistant",
    description="A generic assistant that kindly answers questions.",
    feature_names=["question"],
)

Wrap a custom RAG

Wrap your RAG-based LLM app in an extension of Giskard’s Model class. This example uses a FAISS vector store, a langchain chain and an OpenAI model.

You will have to implement just three methods:

• model_predict: This method takes a pandas.DataFrame with columns corresponding to the input variables of your model and returns a sequence of outputs (one for each record in the dataframe).

• save_model: This method is handles the serialization of your model. You can use it to save your model’s state, including the information retriever or any other element your model needs to work.

• load_model: This class method handles the deserialization of your model. You can use it to load your model’s state, including the information retriever or any other element your model needs to work.

from langchain import OpenAI, PromptTemplate, RetrievalQA

# Create the chain.
llm = OpenAI(model="gpt-3.5-turbo-instruct", temperature=0)
prompt = PromptTemplate(template=YOUR_PROMPT_TEMPLATE, input_variables=["question", "context"])
climate_qa_chain = RetrievalQA.from_llm(llm=llm, retriever=get_context_storage().as_retriever(), prompt=prompt)


# Define a custom Giskard model wrapper for the serialization.
class FAISSRAGModel(giskard.Model):
    def model_predict(self, df: pd.DataFrame):
        return df["question"].apply(lambda x: self.model.run({"query": x}))

    def save_model(self, path: str, *args, **kwargs):
        """Saves the model to a given folder."""
        out_dest = Path(path)

        # Save the chain object (`self.model` is the object we pass when we initialize our custom class, in this case
        # it is a RetrievalQA chain, that can be easily saved to a JSON file).
        self.model.save(out_dest.joinpath("model.json"))

        # Save the FAISS-based retriever
        db = self.model.retriever.vectorstore
        db.save_local(out_dest.joinpath("faiss"))

    @classmethod
    def load_model(cls, path: str, *args, **kwargs) -> Chain:
        """Loads the model to a given folder."""
        src = Path(path)

        # Load the FAISS-based retriever
        db = FAISS.load_local(src.joinpath("faiss"), OpenAIEmbeddings())

        # Load the chain, passing the retriever
        chain = load_chain(src.joinpath("model.json"), retriever=db.as_retriever())
        return chain


# Now we can wrap our RAG
giskard_model = FAISSRAGModel(
    model=climate_qa_chain,
    model_type="text_generation",
    name="Climate Change Question Answering",
    description="This model answers any question about climate change based on IPCC reports",
    feature_names=["question"],
)

For further examples, check out the LLM tutorials section.

Click to view parameter details

• Mandatory parameters

  • model: A prediction function that takes a pandas.DataFrame as input and returns a string.

  • model_type: The type of model, either regression, classification or text_generation. For LLMs, this is always text_generation.

  • name: A descriptive name to the wrapped model to identify it in metadata. E.g. “Climate Change Question Answering”.

  • description: A detailed description of what the model does, this is used to generate prompts to test during the scan.

  • feature_names: A list of the column names of your feature. By default, feature_names are all the columns in your dataset. Make sure these features are all present and in the same order as they are in your training dataset.

Step 2: Scan your model

Now you can scan your model and display your scan report:

scan_results = giskard.scan(giskard_model)
display(scan_results)  # in your notebook

If you are not working in a notebook or want to save the results for later, you can save them to an HTML file like this:

scan_results.to_html("model_scan_results.html")

💡 Customize your scan

Check our Advanced scan usage page, if you want to:

• Scan with only some specific detectors

• Make the scan faster

What’s next?

Your scan results may have highlighted important vulnerabilities. There are 2 important actions you can take next:

1. Generate a test suite from your scan results to:

• Turn the issues you found into actionable tests that you can save and reuse in further iterations

test_suite = scan_results.generate_test_suite("My first test suite")

# You can run the test suite locally to verify that it reproduces the issues
test_suite.run()

Jump to the test customization and test integration sections to find out everything you can do with test suites.

2. Run the test suite with a different model:

# wrap a different model
giskard_model_2 = giskard.Model(...)

# run the test suite with the new model
test_suite.run(model=giskard_model_2)

Check the test suite documentation to learn more.

Troubleshooting

If you encounter any issues, join our Discord community and ask questions in our #support channel.