Over Thinking A/B Testing

Ruin A/B testing by overthinking it. 

A/B Testing Process

No one is thrilled by process. It slows you down. When looking for A/B testing wins, you want to reduce as much process as possible. For example, a process should preserve your velocity, organized cross functional teams, inspire collaboration and maintain the integrity of your A/B testing results. If any part of this slows you down, remove it. You should not be excited by your process in Airtable, Excel, JIRA or Asana. A process should be useful and invisible. Not something to display in a leadership meeting. That’s what results are for.   

A/B Testing List  

Are you proud of your long list of A/B testing ideas? You have sorted them in their proper hierarchy. Arguing for hours over the right prioritization process – RICE, ICE, PIE give yourself a pat on the back. You have overthought A/B testing. For example, a long list never gets done. Have you ever completed a list of 50 to-do items: never. A/B testing ideas should be a short list. With simple ideas. And learn from your first A/B test launched. Focus on the outcome. That will foster your next small list of A/B tests to launch. A long list means you are focused on your own vanity, and not your visitors. Grow up and do some real work. 

Voting on A/B Testing Ideas

Remove your spin and your ability to command a room – introduce voting into your A/B testing process. Voting takes time, it’s a process. Voting will slow you down. For example, A/B testing isn’t important enough to require a vote. It’s an A/B test, not an election. And when visitors experience recipe A or B, let them them cast vote that matters: a conversion. Eliminate voting – its a process. The fun is launching quickly. Task completion. And the outcome. Leadership is inspired by what is learned, not a tally of votes. Someone has a strong opinion on an A/B testing idea, great, it gets launched next. 

Centralized vs Decentralized A/B Testing Teams

4: How much time did you spend on the new org chart? If things are that big, bloated, convoluted and it requires a debate over centralized vs decentralized A/B testing teams, you are overthinking it. If someone can launchd A/B test, then go. Don’t think just launch. The organization will adapt to whatever method your velocity cares to keep up with. Don’t look back. Go.  

Designing for A/B Tests

5: If you worry about “off brand” or style guides you are overthinking A/B testing. For example, if something is off brand, good. If something looks inconsistent – even better – it’s a test. And if you think brand, consistency, more pop, and logos are so important – then test it. But don’t slow down the process. A/B test it and see if your visitors think it’s so bad. Design should not be the friction point of launching an A/B test. Keep going, stop overthinking and have some fun. Jesus.

Puffin.io is A/B Testing Software for Websites. Looking for help setting up your next A/B test? Let us know.