评测对话流程 · Dialogue Evaluator

它解决什么问题

搭好 AI 对话流程就上线,常在前几轮跑偏:答非所问、重复提问、卡在死循环、用户暴躁时不会安抚。直接让 AI 自查,会给自己打高分。

这个技能做什么

把整段对话按轮次拆开评测:每轮检查理解是否准确、回复是否推进目标、转折是否平滑;按「目标达成 / 信息完整 / 语气得体 / 效率」打分,输出问题定位与具体改法(改提示词还是改流程)。

为什么比裸提示词强

• 按轮评测:问题定位到具体哪一轮。
• 区分原因:是提示词问题还是流程问题。
• 可复测:同一套评分卡反复用。

典型用例

• 上线前验收客服/销售对话机器人。
• 调提示词后做前后对比评测。
• 把差评对话复盘成改进清单。

局限

需要你提供对话记录;评分卡需按业务目标定制,主观语气项请人工复核。

English · 英文介绍

The problem

Once the AI ​​dialogue process is set up and launched, it often goes astray in the first few rounds: answering questions that are wrong, repeating questions, getting stuck in an endless loop, and not appeasing users when they are irritable. Just let the AI ​​self-check and it will give itself a high score.

What it does

The entire conversation is divided into rounds for evaluation: each round checks whether the understanding is accurate, whether the reply advances the goal, and whether the transition is smooth; score according to "goal achievement/complete information/appropriate tone/efficiency" and output the problem location and specific correction methods (change prompt words or process).

Why is it better than naked prompt words?

• Round-by-round evaluation: The question is located in a specific round.
• Distinguish the reason: is it a prompt word problem or a process problem.
• Retestable: The same set of scorecards can be used repeatedly.

Typical use cases

• Accept customer service/sales dialogue bots before going online.
• After adjusting the prompt words, do a before and after comparison evaluation.
• Review negative review conversations into an improvement list.

Limitations

You are required to provide conversation records; the scorecard needs to be customized according to business goals, and subjective tone items must be reviewed manually.