“AI can make mistakes. Please check important information.” This or something like it appears below many chat windows. The notice is accurate, but it offers no help when an error actually occurs, since it shows no way to deal with a wrong answer.
Yet the topic is not new. At least two well-known sets of guidelines explicitly address the error case.
What the guidelines say
In 2019, Microsoft published 18 guidelines for human-AI interaction (Amershi et al., CHI 2019). One of the four groups is dedicated to the moment when the system is wrong. Among other things, it includes:
- Support efficient correction. It should be easy to edit or correct something when the system is wrong.
- Scope services when in doubt. If the system is unsure what is meant, it should ask or do less instead of guessing.
- Make clear why the system did what it did.
Google's People + AI Guidebook has its own chapter, “Errors + Graceful Failure”, including on what users can do after an error.
What follows for design
I derived the following points from this; they are not a quoted requirement:
- Correct individual statements. A thumbs down for the entire answer does little good when a single number in two hundred words is wrong. The correction should be able to target the relevant spot.
- Show sources and make them rateable. If you can see what a statement is based on, you can check it. This includes feedback such as “does not support the statement.”
- Undo actions step by step. If an agent has already acted on a wrong assumption, for example filled in fields or changed data, it must be possible to undo this item by item.
- Hand over to a human. In legally sensitive or safety-critical moments, the interface leads to a person without the user having to start over.
Be careful with confidence indicators
The obvious idea would be to simply mark uncertain passages. That assumes the model knows how certain it is. A study by Xiong et al. (ICLR 2024) finds that language models tend to overestimate their own certainty when asked about it. Larger models do somewhat better, without the problem being solved.
External signals such as missing sources, contradictory results, or an ambiguous input are more reliable. Anyone who displays a confidence level should be able to show where it comes from.
From the series “Design Systems and AI”. Previous: Five patterns for AI interfaces that already exist.
Want to build a digital product with a system?
Start a project