ACE

Question

We use ACE but some of the questions are not accurately grading. For example: When we verify a member there are two different ways. We can accept a text code or ask set questions. The bot seems to grade the questions being asked as a no instead of a yes.

Has anyone else run into this and what was the work around you used as my managers are spending a lot of time manually reviewing this to ensure the scoring is accurate. We have built in a variance for the scoring so that on the days we cannot go in and review its not negatively effecting the employees overall score during year end reviews.

BeckyH · Accepted Answer

​@EduquetteGood catch. Splitting it into two questions would just create a new problem, since ACE doesn't support "only one of these needs to be Yes" logic. Whichever method wasn't used would score "No" anddrag down the total.Try rewording it as one question instead:"Did the agent verify the caller's identity by either a text code or the security questions?"On uploading your procedures, that's not possible yet, but it's an open request on the Ideas portal- please do upvote. Right now wording and manual overrides are your only levers.On learning timeframe and note detail - loop in your CSM or open a case asking specifically how overrides factor into future scoring.Let us know if the reworded question helps!

BeckyH · Answer

Hey, ​@EduquetteThanks for raising this.Here are a couple of things worth trying:Split the compound question. Our scorecard-writing guidance flags "either/or" questions as a common cause of AI mis-scoring. Instead of one question covering both verification paths, try two separate Yes/No questions (one for text code, one for security questions), and treat either "Yes" as passing.See Intro to Scorecards in ACE for the full list of question-writing tips.Keep overriding it manually for now. Per our KBs, manual corrections on the Details page actually feed back into ACE and are meant to improve grading accuracy on that specific question over time. This means your team's manual reviews aren't just a stopgap, they're training the model.Confirm the question type is set to Yes/No. Rating-type questions aren't AI-scored at all and always require manual review.If splitting the question doesn't resolve it, open a support caseand reference this specific scorecard question by name. And consider posting it on the Ideas portal too so Product has visibility into the pattern. Happy to keep troubleshooting here if you try the split and it's still inconsistent!

Sign up

Login with SSO

Login to the community

Login with SSO