Designing with Data: Improving the User Experience with A/B Testing

by Rochelle King, Elizabeth F Churchill, Caitlin Tan

On the surface, design practices and data science may not seem like obvious partners. But these disciplines actually work toward the same goal, helping designers and product managers understand users so they can craft elegant digital experiences. While data can enhance design, design can bring deeper meaning to data.

This practical guide shows you how to conduct data-driven A/B testing for making design decisions on everything from small tweaks to large-scale UX concepts. Complete with real-world examples, this book shows you how to make data-driven design part of your product design workflow.


Understand the relationship between data, business, and design
Get a firm grounding in data, data types, and components of A/B testing
Use an experimentation framework to define opportunities, formulate hypotheses, and test different options
Create hypotheses that connect to key metrics and business goals
Design proposed solutions for hypotheses that are most promising
Interpret the results of an A/B test and determine your next move

Genres: Design, Technology, Nonfiction, Business

370 pages, ebook

First published April 22, 2014

Reviews

[Frank · Rating 2 out of 5]

This book took me a while to get through. It repeated itself a lot (the authors even admit it in various points), the whole camp metaphor didn't work at all for me (and the authors give it up partway through the book and don't keep it running), and it's very longwinded. I feel like this could have been a 10-page article or even an extended blog post.

[Kevin Duong · Rating 4 out of 5]

Great read, found a lot of value as a UI/UX Designer. This book gives you a basic overview of data terminology and how data is used at product companies. I appreciated the use of examples and explanations of how you as a designer/researcher/PM can leverage it. The principle and methods shared in this book pertain to A/B testing so don't expect to find much about other methods in here. Have not yet had a chance to implement the learnings but it seems proming. I would recommend other designers to give this a quick read if you're interested in learning more about how to use data to impact your design work.

[MADISON HEISE · Rating 5 out of 5]

Read this for a graduate class and really enjoyed it. Thought it gave a great overview of the importance of quantitative data in ux research and how designers and everyone on a product team can learn from gaining information from users from the lens of A/B testing. I liked that it was specifically through the lens of A/B testing as there are many different ways to collect quantitative data as a UX researcher and it could've been confusing. A/B testing is extremely valuable and also a huge commitment for the company/team and something I personally haven't conducted yet as a product designer but feel way more comfortable doing now since reading this book and since I already have background in other user research methods.

Its also super easy to read and I thought it was extremely thorough!

[Matti · Rating 3 out of 5]

This book had a lot of great points, and had a good framing around using the experimental method in a designers work. I particularly liked the points about finding your "data friends" in your org, the experiment examples from Netflix and Spotify, and was pleasantly surprised that they also explained the need for "thick data" to minimize the chance for error in quantitative testing.

But I thought the book was too verbose, and read more like it was for academics rather than designers. I'm happy I read it, and got a ton of great suggestions and references. But I wish I liked it more.

[Radoslava Koleva · Rating 4 out of 5]

Very informative even if a bit too repetitive.

Some concepts are repeated so many times in so many words that it makes the book level feel like total beginner course. At the same time at other times the material is written with the assumption that you have extensive experience with product design.

In any case I enjoyed the examples from data-driven companies like Spotify and Netflix and I definitely got a lot of ideas to apply to my own work as a Product Manager.

[Miglė Butkevičė · Rating 4 out of 5]

There are some really good insights about A/B tests and the importance of being data aware while working with product development, plus I enjoyed examples form Netflix, Spotify & other companies. But the book it’s too repetitive, you end up reading the same things over & over again.

[Vivek · Rating 4 out of 5]

Helps to build an in-depth understanding about A/B testing in the context of product design. Lots of real life examples from Stitch-fix, Google and Netflix with interesting insights. Could have been a 200 page book, instead of 300+.

[David · Rating 2 out of 5]

数据驱动设计