Display

Automated Alt Text

mattiealston — Sat, 15 Aug 2020 00:00:00 +0000

Automated Alt Text

Application

Display

mattiealston Sat, 08/15/2020 - 00:00

Topic

Fairness & Inclusiveness

Problem

Users with vision impairments want to understand what image content exists in the interface. The system designers want to facilitate this without manually tagging alt text for every image.

Facebook generates alt text as content tags, but also allows the user to edit these if needed.

Solution:

Image recognition automatically tags images with alt text based on what the AI identifies in that image.

Discussion:

While the majority of patterns catalogued in this library are focused on the benefits to the user, in some cases, the value of AI may not be recognized by the end user but rather by those responsible for designing and maintaining the system. In this case, the primary users of alt text will care little how it is generated, as long as exists. Designers, on the other hand, should use patterns like this to broaden our ideas as to where data can be sourced from and ask which is the best approach in each case: from the initial uploader or content manager, crowdsourced across users, or generated by the system itself?

Each has advantages as per the accuracy, authority, breadth and depth of the data created, but the AI approach is unique in terms of requiring the least human effort. That said, it doesn’t pay to be one hundred percent hands-off— automated content tags of any kind benefit from an additional layer of human intervention, where end users can ad-hoc moderate the data by flagging anything they believe is erroneous or offensive.

Dataset Explorer

mattiealston — Thu, 15 Aug 2019 00:00:00 +0000

Dataset Explorer

Application

Display

mattiealston Thu, 08/15/2019 - 00:00

Topic

Transparency & Trust

Problem:

The user wants to understand how the underlying data used in a system is structured, how extensive it is, and comprehend how it is used by the system.

The "Infinite Drum Machine" provides a playful way to explore the data (in this case different percussive sound samples clustered by audio similarity) .

Solution:

Alongside the core functionality provided by the system, there is an extra feature that allows the user to dive into the data itself and explore it from different angles, using filters and other controls to manipulate it. This may utilize an attractive Dataset Visualization pattern as part of this interface, that can provide a playful interaction in its own right. Alternatively, it might be a simple list or table, which holds less appeal for casual explorers (and indeed may be very daunting for them to deal with) but empowers serious users to deepen their understanding of, and expertise with, the system.

Discussion:

For some applications, exploring a dataset can be the very point of the system, and plentiful examples of these can be found. For others, where there is a dataset “under the hood” powering operations, it might not increase the utility of the application to allow users to explore the data, but it can be very effective at building trust in the system by exposing its inner workings. In addition, it can provide a compellingly pleasurable “bonus feature” that can drive additional engagement.

More Examples:

In addition to the core font finding feature, Fontjoy provides a dataset explorer that enables a deep dive into everything the system knows about fonts.

Dataset Visualization

mattiealston — Mon, 15 Jul 2019 00:00:00 +0000

Dataset Visualization

Application

Display

mattiealston Mon, 07/15/2019 - 00:00

Topic

Transparency & Trust

Problem:

The user wants to quickly understand the general shape of the dataset used by the system, the extent of it, and what patterns might exist within it.

The "Living Archive" visualization from Google is an insightful and attractive way to explore the catalogue of dance moves from choreographer Wayne McGregor.

Solution:

The system provides an attractive visualization that takes all the underlying data or a subset of it, and renders it into an easily comprehendible visual. This may be a static image or, if interactive, it may form part of a Dataset Explorer.

Discussion:

In general, users struggle to understand what it means when data powers a system, especially when the reality is that that data is just a list of abstract variables. Using a visualization to illustrate the nature of that data is an incredibly effective tool for helping users build a mental model of the system. In contrast to the Dataset Explorer pattern, where users want to understand and validate the data on a granular level, here even if that visualization is a big picture simplification or only portrays a small subset of data, it can still be a very valuable asset to share.

Demonstrating Thinking

leighbryant — Sat, 15 Jun 2019 00:00:00 +0000

Demonstrating Thinking

Application

Display

leighbryant Sat, 06/15/2019 - 00:00

Topic

Transparency & Trust

Problem: 

The user wants to know that the system has conducted a complex calculation. If the calculation occurs too quickly, they might not believe that the calculation has been completed correctly or has been extensive enough in considering all the variables.

The Articoolo app uses a progress bar and explanatory text to demonstrate the AI is "at work", so the user is given the sense that "thinking" is happening.

Solution: 

The system uses artificial wait times and progress messages to demonstrate that it is "thinking" and applying satisfactory effort to the calculation.

Discussion: 

While simplicity in an interface is often desirable, in some cases the appearance of complexity may be beneficial, even to the point of simulating some of that complexity. While the reality of a system's intelligence may result in effortless calculations and speedy response times, the perception of a system's intelligence is ironically often associated with the appearance of effort and slower responses.  

Dreamlike Novelty

mattiealston — Wed, 15 May 2019 00:00:00 +0000

Dreamlike Novelty

Application

Display

mattiealston Wed, 05/15/2019 - 00:00

Topic

Transparency & Trust

Problem:

Sometimes the user wants to be surprised, to be hit with a joyous wave of weirdness that disrupts their everyday existence and makes them question whether they are awake or are dreaming the strangest of dreams. They also want to laugh at bizarre notions, such is the nature of absurdist humour.

Artbreeder combines images of everyday objects to produce dreamlike creations.

Solution:

Machine learning apps can often output strange results, whether accidentally or by design — the interface should embrace this freakiness and share this with the user. This can be in the form of generative text, imagery, or video.

Discussion:

Many of us long to be tourists in the world of the weird. Now we can be confronted with and delighted by such alien creativity as emerges from machine minds. We have constructed a gateway into the domain of dreams!

The reality is that any machine learning tool is going to take odd paths regardless, so, where appropriate, make it a feature not a flaw to expose the oddities. Beware, though, that what one person considers to be charming Dreamlike Novelty, another may consider horrible Nightmare Fuel.

Manual Overrides

leighbryant — Fri, 15 Jun 2018 00:00:00 +0000

Manual Overrides

Application

Display

leighbryant Fri, 06/15/2018 - 00:00

Topic

Autonomy & Control

Problem:

The system provides one or more AI-generated responses, but the user isn't happy with any of them.

An avatar-creation flow offers AI-generated solutions as well as pre-populated solutions created by a designer without the aid of AI.

Solution:

In addition to dynamically generated AI-responses, the system provides a range of pre-made options not informed by AI to choose from.

Discussion:

Allowing designers to to pre-make alternate responses can be a useful addition to a system, since they're generally good at anticipating users' needs. There are a number of advantages to this approach: it's a good fallback if the AI fails to produce satisfactory quality; it can be a demonstration of the best of what the AI is capable of; and it allows the user to feel empowered choosing the AI-option knowing that they could otherwise opt out while still proceeding in the process.

Progressive Processing Display

leighbryant — Wed, 15 Nov 2017 00:00:00 +0000

Progressive Processing Display

Application

Display

leighbryant Wed, 11/15/2017 - 00:00

Topic

Transparency & Trust

Problem:

While processing an image, the user wants to see the results as quickly as possible, even if that means only seeing part of the image completed.

An example of progressive processing display using a progressive image upload, which shows a grid with different sections coming in clearer as the AI processes.

Solution:

The system processes and displays the image progressively, revealing greater detail or more sections as it is able to.

Discussion:

A pattern like this mitigates some of the issues around long processing times by providing an initial result in a shorter time, which ideally is enough of a preview for the user to choose to cancel if the output is undesirable. It also aids in transparency of operation by showing the algorithm in action rather than just the final output.

Anti-Pattern: Nightmare Fuel

mattiealston — Sun, 15 Mar 2015 00:00:00 +0000

Anti-Pattern: Nightmare Fuel

Application

Display

mattiealston Sun, 03/15/2015 - 00:00

Topic

Transparency & Trust

Problem:

An AI that generates imagery can be capable of producing some very unsettling material, especially when it combines otherwise innocent sources in unexpected ways. The user does not want to be freaked out by this.

Artbreeder combines images of innocent objects to produce horrible things.

Anti-Pattern Response:

The system generates bizarre and disturbing imagery, which can include distorted faces, scary chimeras, unpleasant mutations, unsettling textures, terrifying landscapes, and so on — imagery that is better suited to a horror film, but can even surpass what the human imagination can conjure up on its own. In most cases, a system generates this imagery without much warning to the user that this is a possibility.

Discussion:

The capacity for AI imagery to unnerve us is immense but is often only discussed as a byproduct of AI’s ability to surprise us with its weirdness. If a system generates imagery and the intent is not to disturb, there are some very deep psychological responses that we should be aware of and avoid if possible, or at least caution the user about — at best there is a risk of negative reaction to the apparently familiar being twisted into dreamlike shapes, but at worst there can be a visceral repulsion, a nausea-inducing or otherwise deeply unpleasant sensation of wrongness.

Of course, what we find desirable or not can be very subjective— what one person considers to be horrid Nightmare Fuel might be someone else’s pleasurable Dreamlike Novelty.