A/B Testing: The Most Powerful Way to Turn Clicks Into Customers

by Dan Siroker

A guide to delivering a better user experience through A/B testing"A/B Testing" outlines a simple way to test several different versions of a web page with live traffic, and then measure the effect each version has on visitors. Using A/B testing you can determine the most effective way to increase conversion rate--the rate at which visitors convert into customers. The average conversion rate across the web is only 2%. That means 98% of visitors to most websites don't end up converting. Using A/B testing, companies can improve the effectiveness of their marketing and user experience and in doing so can sometimes double or triple their conversion rates.

In the past, marketing teams were dependent on IT and engineering to implement A/B tests, and that proved to be a costly bottleneck. Today, the market for A/B testing is growing very fast, in no small part due to this transition from IT to marketing. Offers best practices and lessons learned from over 100,000 experiments run by over 2,000 Optimizely customersDetails a roadmap for how to use A/B testing to personalize your customer's web experience and a practical guide to start A/B testing todayAuthors Dan Siroker and Pete Koomen are cofounders of Optimizely.com, an user-friendly testing system that more than 2,000 organizations use

Marketers and web professionals will become obsolete if they don't embrace a data-driven approach to decision making. This book shows you how, no matter your technical expertise.

Genres: Business, Nonfiction, Technology, Design, Startup, Social Media

213 pages, Kindle Edition

First published January 1, 2013

About the author

Dan the CEO and co-founder of Optimizely. The original inspiration for Optimizely came from his experience as the Director of Analytics on the Obama 2008 campaign. Prior to Optimizely, Dan served as Product Manager for Google Chrome and AdWords. Dan has extensive experience speaking about the benefits of A/B testing including guest lectures at Stanford and Berkeley business schools and keynotes at SXSW, SES, Conversion Conference, TheNextWeb, and dozens of others. He has also been featured on Bloomberg, the Wall Street Journal, New York Times and Wired and recognized by Forbes as Top 30 under 30 in Technology in 2013

Reviews

[Stephanie Sun · Rating 3 out of 5]

In the intro to Lean Analytics, the authors admitted that they had difficulty convincing any startups to disclose their experiences as case studies for that book.

This book has the case studies (Obama 2008, Clinton-Bush Haiti, ABC Family, Lumosity, Formstack, Wikipedia, Cost Plus World Market, IGN, Etsy, and more) and is also better written and full of a lot more practical advice (including how to sell A/B testing within your organization), despite being a much shorter and faster read.

Highly recommended.

[Ricky · Rating 5 out of 5]

Unlike the few other business books I have read in my life this isn't just one chapter repeated over and over with a few simple word changes. The book actually does a great job helping people understand what A/B and multivariate testing are then moves on to explain the value proposition and provides valuable advice about how to get started and what to test.

I may be biased because I work at Optimizely, but I would be thrilled if all of our clients read this book before they started testing.

[Bing Wang · Rating 4 out of 5]

This is not a technical book instructing you how to do A/B testing. Instead, it is more like a career summary from an experienced professional. You may expect what A/B testing could do in various scenarios, what are the common mistakes, heart-breaking lessons and brilliant-ideas in his legendary A/B testing history. Still very insightful for a data scientist, product manger and even people who in charge of developing big strategies.

[Cliff Chew · Rating 3 out of 5]

This is a light introductory book to A/B testing. It covers some organizational concerns that beginners should take note, and also covers lightly on some of the theoretical concepts of A/B testing. It highlights many (I don't think it's all, but I don't know what I don't know) of the issues and concerns of starting and executing A/B tests in any organization, and I think overall, for the amount of time that you will require to read this book, I think it's still worth the read (This is really a rather light book to me).

That said, this book doesn't go deep enough if you want to know more about the technical implementations of A/B testing. This is a conceptual book on A/B testing for managers. This isn't a statistics book, nor an technical implementation book.

Note that this book is written by people from Optimizely, which is a tech firm that helps organizations run A/B tests. Don't worry about any hard-selling of their products, because they were very mindful of their own biases view regardless of their own products, and hence, didn't do much of that. That said, I feel that their blog posts have more interesting (I mean in-depth) A/B testing content that interested people who take note.

[Annabel Nguyen · Rating 3 out of 5]

Practical and down to earth knowledge for A/B testing. However, I think this book is like a bunch of (successful) case studies which helps to highlight the importance of A/B testing (if you already knew it, well, this book is hardly valuable much).

Instead of reading this book, you can go and read lots of articles about A/B testing similar like this on internet as well. Still, a good and easy read.

[Abinav · Rating 4 out of 5]

The art of A/B testing is mentioned in a glance at this book. For a number guy who believes numbers and data can rule the world this book reaffirms and restores faith in the process of analytics. Especially the Obama campaign and the use of A/B testing in it roused my curiosity and quenched it too.
A good read and simple read and very useful if implemented in user experience facing products.

[Vicky · Rating 4 out of 5]

An absolute must-read for tech/internet professionals. Beginner level, but has lots of good examples, and the appendix allows you to dig deeper. List of tests and success metrics can be used as reference.

[Shaw Lu · Rating 4 out of 5]

Short and concise cases.

A very good introduction to AB testing. Too much emphasis on political campaign and fundraising. Would love to see more in depth examples on how it works in big tech companies.

[Michael Corley · Rating 3 out of 5]

This book was not nearly as technical and diving into the design of the experiments as I would have hoped. It's a light read. I finished it very quickly.

It is good for getting idea's on what types of A/B tests to run, but not necessarily for getting into the detail of how to do them.

[Ihor Muzyka · Rating 3 out of 5]

Some interesting real-life use cases of A/B testing! Overall, I’d give it 4 stars—if it weren’t for the minor DataViz slips like using line graph to visualize categorical data. Not exactly what you’d expect in a book on analytics. :)

[Rohit Gupta · Rating 4 out of 5]

Good for beginners

I love that the book provides many real-world examples/case studies to communicate the application and value of A/B testing. This is a good book if you are looking to build good foundations on A/B testing, hypothesis driven thinking and analytical thinking.

[Jesse Richards · Rating 5 out of 5]

All business books should be written this well. Would have appreciated a bit more detail in Appendix 2.

[Preston · Rating 4 out of 5]

Loved it(: Yaaasss

[Edaena · Rating 5 out of 5]

Great introduction to A/B testing. This booked is packed with examples and explanations. It also talks about building a test culture in an organization.

[Jadwiga · Rating 2 out of 5]

Really basic stuff. As the stars say - it was ok. Nothing less, nothing more.

[Dawid Jurand Szkiełka · Rating 4 out of 5]

Basics . For beginners. Nice intro for A/B testing.

[Shiro · Rating 5 out of 5]

A must read for people in growth engineering teams.
Easy to read, full of ideas.

[Walter Ullon · Rating 3 out of 5]

Fifty-thousand view of what A/B testing is, why it is important, and how it is used by the likes of the Clinton-Bush foundation, ETSY, Chrome, IGN, and political campaigns.

Early on, the authors lay the ground rules and spend the rest of the book reinforcing their points through several relevant industry examples.

Read it if you need the basics and only the basics. Skip it if you need the nitty-gritty details.

Recommended!

[Alex · Rating 4 out of 5]

I thoroughly enjoyed the book A/B Testing because it was to the point and a great overview of why you would want to do A/B testing, what A/B testing can tell you, and how to get started.

I work in social media and I completely understand how important testing website design is and how it can covert people into customers. This book is one that you could hand to someone who was interested in how this would work and how to get started.

I found that I liked the summations at the end of each chapter to go over what I just read and put it into just a few words. Also, I thought the examples given in the book made it easy to understand why and how they tested them. Especially, the example from the Obama campaign. That one was very interesting because of what the data told them.

Overall, this is an easy read and makes me want to learn more about A/B testing.

[Jenny · Rating 3 out of 5]

This book helps people understand why A/B testing is important, but as a UX designer who already has some A/B testing knowledge, 1/3 of the book is meant for beginners as it repeats stuff I already know. However, I like the examples/case studies. It's easy to understand with the visuals and impressive stats.

Of course, a book written by the authors of Optimizely is meant to sell you the product. Thus I find the book too biased, or you can say framed. For example, in Chapter 7 - Deciding which way to go for A/B testing, the description for using Optimizely is a lot longer with a list of benefits of their product. Also, the examples in Chapter 11 Avoiding Pitfalls, gives an example on Dell testing with off-brand colors, but the chapter just ends there without explaining the results and whether or not Dell found anything useful by testing color variants.

[Niffer · Rating 4 out of 5]

I received this book through a Goodreads First Reads giveaway.

It's not often that I pick up a nonfiction book and find myself compelled to read it from the first few pages. But this book accomplished that. I was really struck by the logic of the author's thought processes and how his logic translated into practicality. I didn't agree with all of his points but overall I found this book to be a valuable read and I expect I will refer back to it in the future.

[Joc · Rating 4 out of 5]

This book can be read in a day or two, but i think its simple message and clear illustration of how A/B testing is supplanting web design strategies is deliberate. Definitely worth reading, you will come away with a conviction that digital marketing is undergoing democratization, no longer the cloistered playground of consultants and creative agencies.

[Nadya Tsech · Rating 3 out of 5]

If you've never heard about a/b testing this is an ideal book for you.
It explains principles with lots of real-life examples

[Teresa · Rating 4 out of 5]

I won this book free through Goodreads. I think that this is a great book for companies and businesses that need some help with marketing and needs to know how to attract customers to their business. This book gives you a step by step process on how to do it.

[Tony Sheng · Rating 3 out of 5]

Only for very very beginners. Read it in 30 minutes. The cases are fun but nothing groundbreaking. A testament to the popularity of A/B testing, I suppose, that the book feels so obvious and dated so soon after publication.

[deleted d · Rating 3 out of 5]

Very good advice, one that in my business I try to do every 3 days. Split testing can make a ton of difference and looking back I regret I didn’t know of it in my previous 2 businesses. If you have visitors to a product, online or not, A/B testing is a concept you must use!

[Maria-Paula · Rating 3 out of 5]

I learned some interesting things as I was reading the book. I would definitely recommend to anyone marketing online for their business. Received as an ARC.

[Emily Reeves · Rating 5 out of 5]

This books is a great 101 for those just starting to A/B test components of website design. I highly recommend it.

[Harry Sesna · Rating 4 out of 5]

Not a bad book at all. I enjoyed it and learned some new things about A/B testing.

[Hannah · Rating 4 out of 5]

Excellent introduction to the concept with concrete examples.