Input Data

Confirm Configuration

leighbryant — Wed, 15 Jan 2020 00:00:00 +0000

Confirm Configuration

Application

Input Data

leighbryant Wed, 01/15/2020 - 00:00

Topic

Autonomy & Control

Problem:

The system requires the user to provide configuration details in order to work effectively. The user does not want to invest effort into this process.

In these two examples, the system uses AI to offer suggestions and prompts to set its functioning so the user can continue with a minimum of input effort.

Solution:

Rather than asking for all configuration details upfront, the system starts running on minimal input and sensible defaults, and organically prompts the user to provide more configuration details over time. This can be in the form of confirming defaults, e.g. "It looks like 3:32pm where you are. Is that correct?" or it can be asking for input on unknown variables.

Discussion:

A key difference for many AI-driven apps compared to non-AI is that user experience will evolve over time, as the user becomes more comfortable using the system and as the system gains in accuracy and capability. With this in mind as designers, the focus should be less on fully configuring the system upfront, and more on how it can become increasingly customized to the user's requirements in a staged process. This is especially relevant for a chatbot type app, where configuration questions can slide naturally into other conversations.

Crowdsourcing

leighbryant — Fri, 15 Nov 2019 00:00:00 +0000

Crowdsourcing

Application

Input Data

leighbryant Fri, 11/15/2019 - 00:00

Topic

Autonomy & Control

Problem:

The system has insufficient or low confidence in the data, it relies on explicit user inputs to make effective, real-time updates to the AI predictions. The user doesn’t want to put a lot of effort into this.

The Trainline app uses crowdsourcing to help make predictions around seat availability, keeping the interactions light so users are more inclined to participate.

Solution:

The system provides easy input options for users to contribute back (by choice) to the functioning of the AI and highlights the reasons for doing so.

Discussion:

Many algorithms rely on ongoing user updates to the information they work from in order to be effective. While some do this in the background (think Google Maps or Waze and their dependence on regular traffic updates, largely gleaned from the relative speed of drivers over a distance, to accurately predict travel times), other algorithms may need users to actively give feedback. By making their input both easy to offer and the reasons for doing so clear, the AI systems can bring the user into the process to ensure the outcomes are useful for all users.

Pattern submission via Jan Srutek

Data Being Shared Flag

mattiealston — Tue, 15 Oct 2019 00:00:00 +0000

Data Being Shared Flag

Application

Input Data

mattiealston Tue, 10/15/2019 - 00:00

Topic

Transparency & Trust

Problem:

When the system communicates data to third parties, the user wants to know what data is being shared and what is not.

As part of the set-up process, Youper flags what data is being shared.

Solution:

The system flags inputs to show this or otherwise tells the user what data is being shared and to whom.

Discussion:

Transparency shouldn't end with a vague story about data generically being stored and shared, rather it should be explicit about what particular data is being used for any particular purpose. Similarly, when data is not being shared, the user will also be reassured to know this.

Explaining Reductive Inputs

leighbryant — Fri, 15 Mar 2019 00:00:00 +0000

Explaining Reductive Inputs

Application

Input Data

leighbryant Fri, 03/15/2019 - 00:00

Topic

Fairness & Inclusiveness

Problem:

The user wants the system to respect their identity and not force them to conform to an identity that doesn't match their own simply to use the app.

By providing an explanation of why it requires a binary response, this health app acknowledges the identity of the individuals using it even when it has to limit them within the application.

Solution:

If the system does ask the user to provide a binary choice around identity—e.g. if a health app asks a user to choose between male and female—then the system explains why it necessary to reduce the options to only these choices, and qualifies what the implication of each is.

Discussion:

While ideally any app allows for the user to choose any gender, medical diagnosis normally requires a choice between binary male or female. The UI is not really the best place to discuss issues around the relationship between gender and sex and the social construction of both. Explaining the necessity of asking a reductive question in such a way as to make it clear that the system empathizes with the users concerns is tricky, yet it must try to do its best.

Inclusive Voice Recognition

mattiealston — Sat, 15 Sep 2018 00:00:00 +0000

Inclusive Voice Recognition

Application

Input Data

mattiealston Sat, 09/15/2018 - 00:00

Topic

Fairness & Inclusiveness

Problem:

Users with speech impairments would like to use voice interfaces, but the software often fails to recognize their speech.

Google's Project Euphonia provides voice recognition to users with speech impairments.

Solution:

The AI that powers an inclusive voice interface is trained with voices from users with speech impairments so that it can understand their inputs. This is achieved to a baseline level through collective training, in addition to training for each individual so as to optimize the system for their personal speech patterns.

Discussion:

Accessibility issues are well understood when it comes to traditional web applications, and the need for interfaces that are accessible to users with vision loss or physical impairments, for example, is widely acknowledged (even if fully accessible UIs are not provided as widely as they should be). By comparison, it can be easy to overlook the issues that users may have with new emergent interfaces, especially those that rely on AI or ML. Unlike the lack of alt text for screen readers, for instance, there are few indications to the able-bodied tester that the interface will fail for some users. Voice interfaces are a very useful case study in that regard — it is only through inclusive interfaces like Voiceitt or Google’s Project Euphonia, that we are made aware of how inadequate our standard provision may be.

Going further, we should not just consider how these projects can make ordinary software accessible to all users, we should embrace opportunities to leverage the unique properties of these emergent interfaces in order significantly improve the quality of life for people with speech impairments, such as those with cerebral palsy, amyotrophic lateral sclerosis (ALS), Parkinson’s Disease, brain cancer, or traumatic brain injury.

Privacy Reassurance

leighbryant — Mon, 15 Jan 2018 00:00:00 +0000

Privacy Reassurance

Application

Input Data

leighbryant Mon, 01/15/2018 - 00:00

Topic

Transparency & Trust

Problem:

Users want to know what happens to their data when it's captured by the system.

Microsoft's AI-powered caption bot has an explanatory box about what happens to the image after uploading outside of the primary captioning function.

Solution:

The system communicates to the user what data it’s capturing, how that data is stored, how long it's stored for, and what other systems the data is being communicated to.

Discussion:

This is especially important when it comes to data captured based on user behaviour rather than explicit input. Ideally the system captures the least amount of data to achieve its task and stores it for no longer than is necessary, but sometimes data needs to be retained (e.g. for that user to maintain a profile or to train the system—or in some cases, another system—in general). In these cases, a key aspect of transparency is understanding the mechanics of this data capture and storage.

Other Examples:

In this example, the user is given valuable insight into what happens to their data after a delete.

Qualitative Feedback for Training

leighbryant — Sun, 15 Oct 2017 00:00:00 +0000

Qualitative Feedback for Training

Application

Input Data

leighbryant Sun, 10/15/2017 - 00:00

Topic

Autonomy & Control

Problem:

When a system being trained by a user is not producing adequate outputs, more information is required to understand what's not working as expected or desired. The user does not want to invest a lot of effort in this process.

In the two images above, a user is given the option to provide feedback with the simple push of a button, and then the application makes feedback easy by auto-generating some options as well as providing an open text field for more information if desired.

Solution:

Triggered by a negative response or rating, the system prompts the user to provide qualitative feedback in the form of an explanation of what is unsatisfactory. This may be free text input or selection from preset options, and is usually done in combination with Quantitative Feedback for Training.

Discussion:

As the effort required here is relatively high for the user with no immediate reward, care should be taken to motivate the user accordingly. This could be the system advising that this feedback will help the system improve and thus establish the benefit to the user. Or the motivation could simply be a pleasant and rewarding interaction, e.g. an interesting conversation point with a chatbot.

Quantitative Feedback for Training

leighbryant — Fri, 15 Sep 2017 00:00:00 +0000

Quantitative Feedback for Training

Application

Input Data

leighbryant Fri, 09/15/2017 - 00:00

Topic

Autonomy & Control

Problem:

The system needs to be trained by the user. The user does not want to invest a lot of effort in this process.

By prompting for a quick thumbs up or thumbs down, the system can get quick and easy feedback from the user to train itself with.

Solution:

When providing an output, the system prompts the user to rate the quality of that output. This could be in the form of a simple thumbs up / down, or require a little more granular feedback in terms of a star rating or other score.

Discussion:

Whether this is fed directly back into training the algorithm or is captured for analysis outside the system will depend on the specifics of the implementation. If the feedback is being captured for analysis, then it makes sense to prompt the user to also provide Qualitative Feedback for Training, with that feedback mechanism triggered by a negative quantitative feedback.

Randomized Inputs

mattiealston — Tue, 15 Aug 2017 00:00:00 +0000

Randomized Inputs

Application

Input Data

mattiealston Tue, 08/15/2017 - 00:00

Topic

Autonomy & Control

Problem

Sometimes a system is so new to the user, or so complicated, that they don’t know where to start, especially in a system that algorithmically generates imagery or sound. Rather than learning how the system works, the user would like to just see it in action as quickly as possible.

Boomy generates new music based on random variables, quickly producing output which the user can later tweak.

Solution:

Instead of prompting the user to enter inputs one by one, a single click on a “generate” button seeds random values into the system and produces output based on those variables. The user may then be given the option to edit the results.

Discussion:

While this is a very effective pattern for first use and feature discovery, there is no reason to limit it to just that use case. In many instances, generative systems can produce pleasantly surprising results from random prodding, and allowing the user to “spin the wheel” facilitates playful exploration and serendipitous discovery. This may be more rewarding than directed or goal-orientated use, in certain instances.

Obviously the nature of the algorithm itself will determine how good the results are when generated from random values rather than considered ones, but even if the algorithm itself is prone to wildly variant results when operated via all possible variables, a randomized inputs feature can be tweaked to work within a sensible subset of variables that produces a good range of output.

Semantic Search

mattiealston — Mon, 15 May 2017 00:00:00 +0000

Semantic Search

Application

Input Data

mattiealston Mon, 05/15/2017 - 00:00

Topic

Transparency & Trust

Problem:

When the user is searching a catalogue of items, they want the system to return results based on that system’s understanding of the content of those items, not just via matching keywords contained in each.

Microsoft Academic search turns the user's input into semantic tags. Via machine learning, these are already associated to items it has indexed, allowing it to return more intelligent results than just via keywords.

Solution:

The system parses the user’s text input into semantic variables. Using machine learning-generated mapping and the ability for the system to learn about the content of items, it can use those semantic variables to return more meaningful results than just using keywords.

Discussion:

This is, of course, the true promise of all AI and ML powered software — rather than relying on precise input from the user to achieve optimal results, a system can take more ambiguous natural language, extrapolate the user’s intent from that, and combine that with an understanding of the actual meaning of the data. We are now seeing how effectively this can be implemented — one challenge will be in how quickly expert users who are used to “speaking computer” through decades of conditioning are able to adapt to software that is actually easier to use.