Question 1

Should I read this cover-to-cover, or can I dip in by topic?

Accepted Answer

Treat it as reference material rather than a linear read. The indexing is strong and most teams will need different chapters at different stages—use the pitfalls sections (multiple testing, sample ratio mismatch, novelty effects, network interference) when you encounter those problems in your own experiments, rather than trying to absorb everything upfront.

Question 2

Why is this more useful than a statistics textbook for running experiments?

Accepted Answer

Kohavi, Tang, and Xu have collectively run tens of thousands of experiments across Amazon, Microsoft, LinkedIn, and Google. The book distills that hard-won experience into a practical manual focused on the failure modes product teams actually discover painfully over years—it's written by people who've made and learned from those mistakes at scale.

Question 3

What makes this the go-to reference for A/B testing in product work?

Accepted Answer

It's the most authoritative book on experimentation currently in print for product direction. Rather than theoretical depth, it prioritizes the specific pitfalls and design decisions that determine whether your experiments give you reliable answers or fool you into wrong conclusions.