🔬Testing

Now that you have an AI agent set up, it is time to test out the agent to make sure that it actually behaves the way you intended.

Iteration

• Ideally, you would start testing your AI agent as soon as you have a simple and short prompt so that you can learn what topics of conversation are likely to come up

• Make sure to test different approaches your trainees might take to see how the AI agent responds

  • Be aggressive, yell and shout

  • Act nervous, trip over words

  • Ask the agent what it thinks should happen

  • Purposefully answer incorrectly or say the wrong thing for your situation

  • Act like a complete novice who has no idea what they’re doing

• You will likely create a list of conditions that happen in the conversation that you’ll want to address as part of the prompt

• As you continue testing, you may come up with a list of questions to ask the AI agent to make sure it is still responding correctly

Consistency

• Make sure that the AI agent stays in the correct role through the whole simulation

• Make sure that your instructions focus on what you want to happen, rather on what you don’t want to happen, as the AI agent can sometimes get confused

  • “Don’t talk about a pink elephant” might actually cause the AI to mention elephants more than normal, instead use more proactive instructions like “if animals are mentioned bring the focus back onto the primary topic of conversation”

• Does the AI agent try to steer the conversation back towards its defined goals no matter how the trainee responds?

• Can you get the AI to say inappropriate or incorrect things?

• When asked for information present in the prompt, does the agent provide the information accurately?

Humanity

• Does the agent speak how you want them to speak?

  • As you test out the conversation, you may find specific mannerisms are overrepresented, or you may want to offer more ways to say the same thing in the “Common Responses” box.

• Does the AI agent pronounce things correctly?

  • You can fix this by using phonetic spelling for words

• Does the AI agent exhibit the desired emotions?

  • Add conditions to trigger emotions when specific conversational topics appear