Improve your chat

The improvement inbox turns conversation evidence into proposed changes that your team can review, test, approve and publish. It can help improve answer quality and lead collection. It does not train model weights or automatically publish changes.

Tenant owners and tenant admins use AI → Improvements for their tenant. Global Admins use AI & knowledge → Tenant improvements, choose a Target tenant, and review that tenant's improvements. Evidence, evaluations and releases remain scoped to the selected tenant.

Where suggestions come from

The system flags negative visitor ratings, workflow or tool failures, delivery failures and inactive pending forms or workflows. It also samples completed successful conversations, so reviews include working examples. Analysis is limited to 100 distinct conversations per tenant per UTC day, with bounded retries.

An automated finding is a suggestion to investigate, not a verified fact. Read the conversation and supporting evidence before making a change. Generating a draft is a separate reviewer action.

Review, evaluate and publish

Open an inbox item and inspect its evidence, category and status. Use Reject with a reason if it does not warrant a change.
Click Capture current baseline to capture the supported configuration and agent scope. If the scope has changed, use Rebase to current scope and clear approval, then prepare a new draft and evaluation. A baseline captured now does not establish which settings an older conversation used.
Click Generate draft, then review Edit candidate. Check the Change type, complete agent instructions or callable workflow offer instructions, and any proposed knowledge correction. Use Save draft and clear prior approval after editing.
Verify knowledge sources using Verify source for this candidate when applicable. A plausible answer or generated reference is not sufficient evidence.
Prepare the Regression suite. Add review templates supplies starting cases; adapt them to your tenant's facts and actual workflows. Supply verified expected answers, deterministic assertions and at least one Held-out case, then Verify and save case. Held-out cases are excluded from draft generation.
Click Evaluate. Model-dependent cases run three times against both the baseline and candidate. Critical checks must pass on every candidate run, quality must meet or exceed the baseline, and errors fail evaluation. Inspect the results before approving.
Click Approve, then Publish to activate the approved, evaluated change. Approval alone does not change live chat behavior.

Editing a candidate clears its evaluation and approval. Changed scope or new evidence can make an earlier evaluation stale and block publishing until it is reviewed again. Accuracy and safety checks take precedence over conversion gains.

Supported changes and limits

Supported executable candidates include agent instructions, verified knowledge corrections within supported document or website scopes, and offer instructions for captured callable workflows. Callable workflows are still subject to their supported node types.

Some dependencies cannot yet be captured for a reproducible improvement evaluation. These include mutable parent workflow graphs, MCP tools, commerce or email retrieval, external user information, webhooks and integration-dependent skills. A configured AI guardrail decision model can also prevent baseline capture until its dependencies can be versioned. If capture reports an unsupported scope, ask your platform administrator to review it; preserve the safeguards your chat needs.

Workflow graph changes use the existing workflow editor: review the proposal, edit and save the workflow, save its attachment and publish the agent, then capture a fresh baseline. Engineering proposals are review-only; the inbox does not execute generated code or deploy software.

Releases and rollback

Published improvement releases apply to future chats. Existing conversations and running workflow calls keep their captured release settings. If active guardrails no longer match a captured release, runtime checks can block it pending staff recovery.

Use Roll back on a published proposal to return to its baseline ancestor release in the same scope. Future chats use that release; existing chats keep their prior settings. Rollback does not undo emails, CRM updates or other external actions already performed. This release rollback is separate from Versions → Set as Current in the agent editor.

Review outcomes and read the metrics

Under Review a production outcome, select Conversation evidence and record whether the answer is grounded, contains an unsupported claim or repeated clarification, and handles handoff appropriately. Click Save reviewed outcome. Staff reviews provide verified judgments; automated findings do not become reviewed accuracy scores by themselves.

For qualification, choose Also review qualification, enter the Tenant qualification criteria, and record whether the conversation is eligible and qualified under those criteria. Criteria are versioned so incompatible definitions are not combined into one rate.

Recorded outcomes · last 30 days reports observed outcomes for the release:

Offer acceptance uses accepted offers divided by recorded offers.
Form completion uses valid submissions divided by collection starts.
Delivery success uses confirmed delivery outcomes for submitted conversations.
Qualification uses reviewed eligible conversations under the same criteria version.
Reviewed accuracy includes its reviewed sample size. Missing judgments or delivery receipts remain unknown, rather than being treated as zero.
AI credits and model-call latency cover correlated recorded calls; uncorrelated older calls are not counted.

Synthetic evaluation runs are excluded from production outcomes. These metrics describe recorded results; they do not establish that a release caused a conversion increase. Use Chat test for the visitor experience as well as regression evaluation for candidate comparisons.

Remove evidence

Purge evidence and derived copies removes the selected evidence and its derived copies and invalidates dependent candidates, evaluations and releases. Treat it as a removal action, not a way to reject a suggestion. Automatic evidence expiry runs only when an explicit retention policy is configured.