Research into notifications and control

Summary

I created a design system for displaying and presenting information to viewers that respects their autonomy in the context of the whirlwind apps vying for their attention.

Role & Activities

As design lead for this project I worked with a team of user researchers, product managers, designers, customer support experts, back-end engineers, and marketing professionals on this multi-year effort to understand and make incremental improvements to the notification experience.

Visual Design
Strategy
Wireframing
Facilitating workshops
User testing

Background

Notifications are intensely personal and incredibly fraught with risk. Wanting to understand how entertainment notifications impacted viewers, I worked with several product managers, UX researchers, CRM managers, customer support analysts, and other designers to get a full picture.


Opportunity/Risk

After launching a new interface with the launch of a Live TV service in 2017, Hulu turned off push notifications. While disabling a feature, we needed to create a new strategy around notifications and with that a new understanding of our viewers needs, habits, and other factors in their lives. Push notifications across entertainment apps generally focus on new content. User-generated content platforms focus on subscriptions to specific channels and traditional media platforms center their notifications around new offerings and time-based announcements. Making viewers aware of newly available content helps the businesses by re-engaging users and bringing additional engagement to the platform.

Competitive Analysis of profile layouts, notification experiences, and control options for users

Research

Working with Jill Strawbridge, Ph.D. I performed a literature review of external research on push notifications across a variety of devices and platforms. I synthesized this research to build a baseline of knowledge. A study the Android team at Google conducted on push notifications served as a north star of where features should lead.

Process

I worked with 3 UX researchers over the course of this multi-year project. Together we developed a series of goals and priorities based on our understanding of existing research uncovered in the literature review. We understood what questions we had about users in general and their notification reactions and priorities, but also understood that viewer interest and engagement alone would not determine the core user experience.

Coming from a background in marketing, I developed a partnership with Hulu’s CRM team, a team who created, sent, and analyzed communications to viewers through a variety of channels. I wanted to understand their motivations and goals. SInce push and in-app notifications come from marketing with the goal of increasing engagement and other metrics I wanted to find the intersection between the business’ goals and the user needs. This would allow us to work together with full transparency toward the same goal.

We had information around broad population use and experience with notifications but needed to conduct our own with current and potential Hulu users. Working with user researchers I designed a series of internal qualitative studies that would explore how mobile push notifications and in-app notifications fit into our viewers’ lives. Here I developed three core insights:

  1. Make it easy to turn notifications on and off
  2. Viewers need to feel notifications are relevant and they have control
  3. There is general agreement among viewers around notification categorization

Next we worked on testing a few hypotheses about specific notification types. We conducted a pre-study survey to personalize sample push notifications to each viewer to simulate how these notifications would work in production. We tested in-person with a small sample to get qualitative insights into anticipated interest in these notifications. This sentiment would help guide opt-out preferences

I worked with the backend services teams, the CRM team, and a product manager to understand the motivations and segmentation capabilities of proposed push notifications. We brainstormed possible groupings of notifications internally and cross-referenced with the sentiment from our research on push notifications to create a proposal for opt-in categories. I worked with a product copywriter to name and describe these categories.

Working with a different user researcher we designed two card sorting studies, one open and one closed, to understand if viewers grouped a sample of notifications using the same logic we had. We wanted to understand if viewers received a certain type of notification, would they know which setting to adjust to affect that type of notification. Similarly, we wanted to see how viewers would group these messages on their own to identify any patterns and departures from our proposal.

We found that viewers wanted control, the ability to know when things were new, and important security updates. I took this knowledge coupled with our extensive research on push notifications and selected a few key human values our team uses in the design process: autonomy and honesty. We used these principles in the design and prioritization process for next steps.

From here we posited that providing a place in-app to receive relevant information would provide the most benefit to the most number of users. Providing granular notification controls for in-app and push notification would give greater autonomy to users.

I conducted an extensive competitive analysis of over 75 apps and how they used in-app notification centers.

Working with an associate designer we sketched possibilities. We tested these designers via a remote survey and in-person usability test.

Outcome

My team was able to push for changes to the layout of the profile section in our mobile apps, using native split layouts on iOS to take advantage of screen real estate for make the destination more useful.

Communications performed between 16-24% better in an A/B test of viewers with the notification inbox and profile updates. Presence of the inbox and path to notifications also resulted in 1.5-2% sustained visit days per visitor and visit days per subscriber.