Audit advanced Works with: Any AI

Chatbot Conversation Flow Audit

Table output Has example

Check bot answers, handoffs and dead ends.

The Open buttons send the prompt text as currently shown below to that site only when you click. Fill in the placeholders first.

[VERIFY] is an editorial flag, not a blank to fill. Wherever the prompt shows [VERIFY], check that item against a reliable source before you use it.

Audit our support chatbot. Intents, sample conversations, fallback messages, handoff rules and failure examples: [CHATBOT_DATA]. Customer complaints about the bot: [COMPLAINTS]. Hours when humans are available: [HUMAN_HOURS].

Test conversations in writing: a clear question, an unclear one, an angry one and one involving sensitive data. Check accuracy of answers, loops, clarity that the customer is talking to a bot, ease of reaching a human, and handling of personal data.

Return a table with Scenario, Bot behaviour, Problem and Fix, a score from 1 to 10 for helpfulness and honesty, and rules for when the bot must hand over. Disclosure rules for automated systems are [VERIFY] for [JURISDICTION].

Also suggest regular review of conversation logs for failed intents, a policy for keeping personal data out of transcripts, and a clear phrase that tells customers they can ask for a person at any time. Finish with the single most urgent fix.

Then prepare a test and improvement plan with governance. Write forty test conversations grouped by intent, language style, error type and sensitivity, and the expected result for each. Define rules for handover: when the bot must offer a person, how context is passed, and what happens outside staffed hours. Provide a content review process in which every answer in the bot is owned, dated and linked to a source policy. Specify safeguards for personal data, including not asking for sensitive information and masking in logs. Add a monthly review of unresolved conversations, thumbs-down feedback and phrases customers use that the bot does not recognise. Draft an honest introduction message stating that it is an automated assistant and what it can and cannot do. Describe measures that reflect customer success, not just containment. Disclosure and privacy rules are [VERIFY] for [JURISDICTION].

Finish with a record template for tracking each finding through to completion, with columns for owner, status, date fixed and how the fix was verified.

Fill in before use

Optional: type your details and the prompt above updates. Nothing is saved or sent.

Replace each bracketed placeholder with your own details, then paste into Any AI or another assistant.

Scenario table (invented bot)
Scenario: "I want a refund." Bot behaviour: lists the policy, then asks the same question twice. Problem: loop. Fix: offer a form after one failed turn.
Scenario: "My card was charged twice." Bot answers about plans. Problem: wrong intent. Fix: route to a person.
Scenario: customer types an address and phone number. Bot stores it in logs. Problem: personal data. Fix: mask.
Scenario: "Speak to a human." No route outside 9 to 5. Problem: dead end. Fix: show hours and an email form.
Score for helpfulness and honesty: 4. Rule: hand over after two misunderstandings.

• audit