> ## Content Index > Fetch the complete content index at: https://kandid.ai/blog/llms.txt > Use this file to discover other available public pages before exploring further. # Ecommerce Chatbot Setup: Review Bot Responses Before Publishing Changes - URL: https://kandid.ai/blog/ecommerce-chatbot-setup-review-bot-responses-before-publishing-changes/ - Published: 2026-10-18T18:18:47.000Z - Updated: 2026-10-18T18:18:47.000Z - Author: Pulkit Garg - Tags: Shopify, Customer Support, Chatbot Setup, Ecommerce Chatbots > **Quick Summary:** Every chatbot edit should be reviewed before going live, since a single unchecked answer can promise a refund or delivery term the store cannot honor. The fix is to tier responses by risk, hold high-risk topics like refunds and damaged deliveries to a named approver and human handoff, and check every draft against a verified 50-100 question answer bank. Re-run that bank after any prompt, catalog, or policy change and keep a versioned change log so bad replies can be rolled back fast. Kandid fits this flow because it lets Shopify teams log handoffs and keep chatbot replies aligned with live store data. A merchant edits one line in a returns prompt, ships it unreviewed, and three weeks later the bot promises store credit that no longer exists. A simple ecommerce chatbot response review process stops this. This guide shows how to build an ecommerce chatbot response review process with risk tiers, a test question bank and named approvers, so your ecommerce chatbot response review process catches bad answers before shoppers do. ## Step 1: Run a Risk-Tiered Review of Every Bot Response Not every bot answer carries the same weight. A wrong shipping estimate costs you a sale; a wrong return promise can cost you a chargeback. Review effort should follow that difference. ### Classify Responses by Risk Before Editing Sort every response your bot can give into three tiers. **High risk** covers anything with money or policy attached: refunds, returns, delivery promises, pricing, discounts, subscription terms. A human must approve every edit here. **Medium risk** covers product guidance, sizing, and recommendations. One reviewer checks these against live catalog data. **Low risk** covers greetings, small talk, and phrasing. A single pass of edits is enough. Write the tiers into a simple sheet, assign a named approver to each, and make tier checks part of your standard [chatbot setup checklist](https://kandid.ai/blog/ecommerce-chatbot-setup-checklist-for-shopify-store-launches/). Approval ends when the right person signs off, not when the draft "looks fine". ### Compare Each Draft Against a Verified Answer Bank Build a question bank of your 50 to 100 most common shopper questions with the approved answer for each. When a new or edited response comes up, check it word by word against that source. Policies change, stock moves, and AI drafts drift; the bank is your single point of truth. If a draft disagrees with the bank, the bank wins until you update it deliberately. Never approve a claim you cannot trace to a store policy page or catalog record. ## Step 2: Gate Sensitive Topics Behind Human Handoff A chatbot should answer what it knows and stop where money or trust is at stake. Refunds, damaged items, and disputes drive most escalations, and research suggests a large share of handoffs lose context when the transfer is set up poorly. Build the gates first, before any new answers go live. ![Flowchart routing refund requests to a human agent](https://assets.snowseo.com/organization-813abc9e-b233-44b2-ae76-4bd670b7e4d1/brand-R0iMcAIFDKu8Brq0IQGR6SjFzQ1H1NWI/library/ai-images/ai-image-1792347485818-g7w5im.webp) Flowchart routing refund requests to a human agent ### Topics That Should Never Auto-Publish Keep these off the bot's approved answer list and route them to a person every time: - Refunds, chargebacks and disputed payments - Damaged or missing deliveries - Legal, health or safety claims about products - Requests to speak with a human - Angry or repeated messages, a sign the bot is stuck ### Configure an Approval Trigger and Fallback Message Set three triggers: an explicit "talk to a person" request, low answer confidence, and keyword rules for the topics above. Per escalation guidance, keyword rules work best when combined with confidence and sentiment, not used alone. Then write the fallback message. It should confirm the transfer, say who will reply and when, and never send the shopper back to the start. Test each trigger with a real question before you publish. If your platform, like Kandid, lets you log handoffs, review them weekly to spot weak answers. ## Step 3: Document the Change, Version the Prompt, and Re-Test ![Person editing prompt code on laptop with version log notebook](https://assets.snowseo.com/organization-813abc9e-b233-44b2-ae76-4bd670b7e4d1/brand-R0iMcAIFDKu8Brq0IQGR6SjFzQ1H1NWI/library/ai-images/ai-image-1792347495922-tvmjfr.webp) Person editing prompt code on laptop with version log notebook ### Keep a Short, Honest Change Log Write down what you changed, why, who approved it, and when it went live. Keep it in a shared sheet with the old and new prompt side by side. If a reply goes wrong two weeks later, you can roll back in minutes instead of guessing what the bot used to say. Version numbers matter more than pretty notes. ### Re-Run the Answer Bank on a Set Cadence After any prompt edit, product launch, or policy change, re-test every answer in your 50 to 100 question bank before publishing. Then repeat monthly, because store data and offers drift. Log which replies failed and fix those first. ![Kandid homepage](https://assets.snowseo.com/organization-813abc9e-b233-44b2-ae76-4bd670b7e4d1/brand-R0iMcAIFDKu8Brq0IQGR6SjFzQ1H1NWI/screenshots/screenshot-homepage.png?v=1782889980333) Kandid homepage Every bot edit carries risk, so review your answers before shoppers do. See how [Kandid](https://kandid.ai/?ref=kandid.ai) helps Shopify teams keep chatbot replies accurate and on-brand. ## Frequently Asked Questions ### Q1: How should teams review ecommerce chatbot responses before publishing updates? Classify each response by risk tier, check drafts against a verified answer bank, and require named approver sign-off for high-risk topics like refunds and delivery promises before going live. ### Q2: Which chatbot topics should always route to a human? Refunds, chargebacks, damaged deliveries, legal or safety claims, and any request for a person. Route angry or repeated messages to staff too, since they usually signal a stuck bot. ### Q3: How often should you re-test your chatbot's answers? Re-run your full question bank after any prompt edit, product launch, or policy change, then monthly. Store data and offers drift, so log failed replies and fix them first. ## Conclusion Treat every bot edit as a small release: tier responses by risk, check drafts against your question bank, gate sensitive topics behind handoff, and re-test on a set cadence before shoppers find the errors.