System Feedback

leighbryant Sun, 11/15/2020 - 00:00

Topic

Problem:

The algorithm may take a while to deliver a result. While this is happening, the user wants to know what is taking place.

The AI-powered application provides both a progress bar and a more granular explanation for the time it is taking in note-form below the bar.

Solution:

The system provides granular progress messages during calculations.

Discussion:

This is important both for the transparency of the system—to reassure the user that the system is operating and hasn't stalled—and to create the impression of an intelligent system with many moving parts. In some cases where there is no actual processing delay, it may even be beneficial to simulate one (as with Demonstrating Thinking). By contrast, sometimes it may undermine the overall impression of the system to expose its workings and over-explain (in which case Appropriate Magic might be deployed).

Before & After Comparison

mattiealston — Fri, 15 May 2020 00:00:00 +0000

Before & After Comparison

Application

mattiealston Fri, 05/15/2020 - 00:00

Topic

Problem:

The user has submitted an artifact for processing, be it an image, sound clip, or video file, and wants to understand fully what effect the processing has had.

This demo from Nvidia is so convincing that you might not even believe the "After" image had been edited if you couldn't compare it to the "Before"

Solution:

The system presents the user with their initial artifact beside the processed version for comparison. On a desktop app, this may mean a simple split display. On a mobile app where screen real-estate is more of a concern, this could take the form of a slider that allows the user to swipe between before and after images, or a similar interactive device.

Discussion:

Although a very straightforward pattern, it is one that can be overlooked by designers. Losing sight (or sound) of the original content can present serious problems for the user in evaluating the effectiveness of the processing, or, if the effect is subtle, to even verify that processing has correctly taken place. The immediacy of comparison is key to reducing the cognitive load required of the user. In many cases even when the original content is presented with the processed version, the user is required to click in and out of each version via preview thumbnails, which makes direct comparison problematic. Likewise for sound clips or video, being able to quickly swap between before and after clips as they play (without restarting) aids the immediacy of comparison.

Confidence Status

leighbryant — Sat, 15 Feb 2020 00:00:00 +0000

Confidence Status

Application

leighbryant Sat, 02/15/2020 - 00:00

Topic

Problem:

Users want to know how accurate results are when the system presents the results of a calculation.

The wording “Seats may be available” was chosen carefully and deliberately to suggest there is a degree of uncertainty in the train busy-ness prediction.

Solution:

The system provides a visual indication of the confidence status of the calculation (as a percentage, by graphical indicator, or through another format).

Discussion:

When the operation of an algorithm is opaque to the user, it is hard for that user to assess how much trust to place in the calculation. Transparency in the algorithm is vital for the user to build a relationship of trust in the system. This transparency includes not just details of how the algorithm arrives at its results, but also honesty about its shortcomings. Displaying a confidence status allows the user to differentiate between reliable and unreliable outcomes, and to make more informed choices. It also works to mitigate the negative impression around inaccurate results and to amplify the feeling of success around accurate ones.

Other Examples:

A lookalike app displays a confidence status with each potential match to show how similar the system thinks its AI prediction is.

Another image recognition AI app displays a score for certainty of assessment.

Explicit Training

leighbryant — Tue, 15 Jan 2019 00:00:00 +0000

Explicit Training

Application

leighbryant Tue, 01/15/2019 - 00:00

Topic

Problem:

When a machine learning system is being trained through the user's repeated inputs, the user wants to know that this is the case.

This lookalike application explicitly informs users that their image will be used for training the image processing services.

Solution:

The system tells the user when it's taking inputs for training purposes, especially if those inputs are unrelated to the user's intentions.

Discussion:

When the user has a long term relationship with a machine learning software, one in which the software shows an improvement over time, it's important to establish the expectations around improvement up front. It's also useful for mitigating some of the negative impressions that can be formed if the AI is still "in training" and imprecise, acknowledging inaccuracy in a way that makes it clear that improvement can be expected. This also goes to the heart of issues around transparency— if the system is capturing data from the user, then it should communicate why such data is needed. This pattern is especially relevant for chatbots and other conversational UIs, where the user may be involved in a long term, conversation-based relationship with the system.

Input & Output Comparison

mattiealston — Sun, 15 Jul 2018 00:00:00 +0000

Input & Output Comparison

Application

mattiealston Sun, 07/15/2018 - 00:00

Topic

Problem:

When the AI acts on variables that are inputted by the user, the user wants to compare what they originally inputted to the resulting output, in order to better understand how they can achieve the output they desire.

The GauGAN app generates photorealistic landscapes from the user's doodles. The input and output comparison is vital to understand how this works.

Solution:

The system presents a summary of the user's input next to the resulting output. As the output is already translating the user’s input into something new, the input feedback itself should closely resemble the user’s input in format, e.g. if they have entered data as variables to be converted into graphics, the input feedback should be variables (not graphics).

Discussion:

This pattern assumes that the user is submitting data for processing and waiting on a response — obviously in systems where controls directly affect the output via real time tweaking, as long as those controls are visible (e.g. are not automatically hidden in a collapsed panel once applied), they suffice as input feedback.

In more conventional, non-AI applications, it can be redundant to replay the user’s input to them, as their mental model of cause and effect in such cases is strong and they fully grasp how their input relates to output. In AI operations however, the AI can act in unanticipated ways where it is harder for the user to understand what control they have on it, and to know how to guide it towards their desired results. Keeping the input in the same context as the output allows the user to use trial and error exploration over repeated operations to develop their understanding of the system.

A similar pattern is Before & After Comparison, and in practice there may be some crossover between these two. That said, there is an important distinction — Before & After Comparison allows a user to inspect two objects to extrapolate the effect of the AI on the processed version and validate its success, whereas Input & Output Comparison is more focused on allowing the user to establish a sense of control over the operation by building a picture of cause and effect.

Motion Tracking Feedback

mattiealston — Tue, 15 May 2018 00:00:00 +0000

Motion Tracking Feedback

Application

mattiealston Tue, 05/15/2018 - 00:00

Topic

Problem:

When an algorithm processes the user’s bodily position in real time via pattern recognition, the user wants to understand what the system sees and how they can affect that.

In the Move Mirror application, it is helpful for the user to see what the computer sees so they can explore poses to find different matching images.

Solution:

In an overlay over the video capture, the system shows a simplified representation of the points on the body that it is tracking, often connected in a wireframe model of lines and nodes. This offers an intermediate abstraction between the user’s body and the underlying numerical variables that are actually used by the system.

Discussion:

For many AI systems, the key usability challenge is in how the user builds their mental model of what the system is doing. This is especially relevant when the system is capturing something concrete and intuitive from the user (e.g. the position of their body and limbs) and converting that input into intangible datapoint to process it or generate new outputs from it. While the intermediary visual neither really represents how the user thinks about their body nor really manifests the abstraction of variables under the surface of the software, it is an effective vehicle for communicating the translation of body to data that is happening in that moment.

Object Identification Feedback

mattiealston — Sun, 15 Apr 2018 00:00:00 +0000

Object Identification Feedback

Application

mattiealston Sun, 04/15/2018 - 00:00

Topic

Problem:

When a system uses visual pattern recognition to assign data to an image or video, the user wants to understand how the system sees the world.

Object identification as seen in demo of Tensorflow functionality.

Solution:

In an overlay over the image or video capture, the system shows a bounding box around the objects that it identifies, with a corresponding label. It can also display a Confidence Status attached to each object to indicate the probability of a correct match.

Discussion:

Although currently the desire for this feature is often driven by the simple curiosity to “see how the computer sees”, the mapping of data to objects in the world will be a core feature of the pervasive computing landscape in future— especially if we assume the prevalence of AR interfaces in that world. In developing this pattern, it is not just a case of providing elegant solutions for task completion, but also refining a visual language that is likely to affect the way we conceive of the environment around us for generations to come.

Progressive Feature Reveal

leighbryant — Fri, 15 Dec 2017 00:00:00 +0000

Progressive Feature Reveal

Application

leighbryant Fri, 12/15/2017 - 00:00

Topic

Problem:

The system can do a lot of interesting things. But the user doesn't want to be bombarded with information learning about them all at once.

An app offers progressively more information as the user gets further into the system's set up flow.

Solution:

The system slowly reveals features over the first phase of use.

Discussion:

As with Confirm Configuration and similar patterns, the designer should think of the features of a piece of software not just extending across a UI, but also in time. Not only what is displayed where, but what is displayed when and on what trigger? Whole sections of the app may initially be hidden and only activate later when relevant.

Training Progress Indicator

mattiealston — Wed, 15 Feb 2017 00:00:00 +0000

Training Progress Indicator

Application

mattiealston Wed, 02/15/2017 - 00:00

Topic

Problem:

When the algorithm relies on a particular user to train it in order to perform at optimal effectiveness, the user wants to know how much of the training has been completed.

In the generative music app "Boomy", the system needs to be trained to understand the user's preferences. The progress bar provides vital feedback on this.

Solution:

The system displays a progress bar, percentage complete, or similar device, to show how far the system is through the training process.

Discussion:

Many AI systems will be trained en masse, either behind the scenes with datasets curated by the designers or by collecting aggregate user data in live use, but some applications rely on each individual user training the system to complete their particular tasks. The system has to learn from their actual inputted material or own unique preferences. In these cases, there could be a separate training mode prior to task mode being unlocked, or alternatively the system trains through actual use, increasing its effectiveness through repeated operations. In either case, it is beneficial to communicate to the user how far they are through the process.

Anti-Pattern: Content Flagging Biases

mattiealston — Sat, 15 Aug 2015 00:00:00 +0000

Anti-Pattern: Content Flagging Biases

Application