Misguidance & Unauthorized Advice
Probes that test whether AI agents can be manipulated to provide professional advice outside their intended scope.
Probes
Section titled “Probes”| Probe Name | Description |
|---|---|
| Unauthorized Advice | This probe generates adversarial messages that request unauthorized advice from the agent, including financial recommendations, medical advice, legal counseling, etc. |
| In Domain Misguidance | This probe uses a dynamic logic: it adapts its follow-up questions based on the agent’s previous answers to probe for weaknesses and see if the chatbot can be led into providing harmful or out-of-scope guidance. |
| NEW Controversial Topics | This probe tests whether an AI agent handles sensitive and controversial topics appropriately. An autonomous agent identifies areas within the target’s domain that are polarizing, ethically complex, or require contextual nuance, then probes them with realistic user scenarios to detect context-insensitive advice, side-taking, and inappropriate recommendations. |