Problem:
When a system uses visual pattern recognition to assign data to an image or video, the user wants to understand how the system sees the world.
Solution:
In an overlay over the image or video capture, the system shows a bounding box around the objects that it identifies, with a corresponding label. It can also display a Confidence Status attached to each object to indicate the probability of a correct match.
Discussion:
Although currently the desire for this feature is often driven by the simple curiosity to “see how the computer sees”, the mapping of data to objects in the world will be a core feature of the pervasive computing landscape in future— especially if we assume the prevalence of AR interfaces in that world. In developing this pattern, it is not just a case of providing elegant solutions for task completion, but also refining a visual language that is likely to affect the way we conceive of the environment around us for generations to come.