- The Flywheel
- Posts
- THE CRO TOP TEN #4
THE CRO TOP TEN
Conversion Rate Optimization ain’t easy.
But it can be a blast if you are doing the right things.
We spoke with RBL teammates and our very own head of all things Conversion Rate Optimization, Gary Marx to identify the CRO Top Ten!
Inspired by The Late Show with David Letterman ‘s Top Ten
The Big Takeaway (TLDR)
What is the most important factor in managing a successful conversion rate optimization (CRO) program for e-commerce brands? Spoiler Alert...
Aligning Expectations
Now let’s show Dave some love and count down to the most important factors in CRO, Experimentation and A/B testing:
#10 Balance - People do not realize how much CRO programs are a true balance of data, customer insights, e-commerce client input, experience and gut instinct. While it might sound scientific, great CRO practitioners know how to strike the balance between the data and the science and when to rely on what is best for the client, for the end user, and the situation.
There is great example of the need for this balance in Jeff Bezo’s 2016 letter to shareholders
“market research and customer surveys can become proxies for customers – something that’s especially dangerous when you’re inventing and designing products. “Fifty-five percent of beta testers report being satisfied with this feature. That is up from 47% in the first survey.” That’s hard to interpret and could unintentionally mislead.
Good inventors and designers deeply understand their customer. They spend tremendous energy developing that intuition. They study and understand many anecdotes rather than only the averages you’ll find on surveys. They live with the design.
I’m not against beta testing or surveys. But you, the product or service owner, must understand the customer, have a vision, and love the offering. Then, beta testing and research can help you find your blind spots. A remarkable customer experience starts with heart, intuition, curiosity, play, guts, taste. You won’t find any of it in a survey.”
#9 Clarity - Being crystal clear in verbal and written communication removing ambiguity and working with individuals on your team (especially in remote working environments) that can deliver clear messaging is massively important.
#8 Have fun! - We are not curing cancer here. But we are unlocking a ton of learnings by combining business, art, and science. When you are having fun you are more likely to do your best work, learn more, and gather improved insights from your colleagues, customers, and clients. Enjoy the process.
#7 Stats - You need to mind the data at the experiment level to ensure you are taking into consideration: Statistical Significance, Confidence Interval, Power, and Minimum Viable Effect. If principles are ignored it can lead to false positive or false negative conclusions.
#6 User Experience (UX) - Who are you doing this for? The customer, the user, right!? So ensuring your copy, headlines, trust symbols, testimonials, call to action, all speak to the appropriate user segments is going to allow you to achieve better outcomes. If you can tap into some UX pros on your team like we do or outside UX resources - it’s highly encouraged if not required.
#5 Data Analysis - Qualitative and quantitative data analysis, used to identify challenges, answer questions, feed experiment generation, and deliver solutions. Knowing the steps in the data process. We often break data analysis into chunks as follows:
Defining the challenge / question
Collecting the data
Cleaning the data
Analyzing the data
Visualize and Sharing data and results
Providing a recommendation
#4 Reinvent for large gains - We have shared this concept in recent tweets, it is probably one of our greatest call outs when counseling e-commerce clients about CRO and A/B testing. It was also perhaps my favorite takeaway from the Reforge Experimentation & Testing course:
While simplification and enhancement are important to secure lower cost/friction wins in your CRO program, there comes a time when you need to add more macro value and reinvent. This is when you gain the largest wins in CRO programs. You need your base hits but you also need an occasional home run!
#3 Speed Matters - You need to ensure you have testing velocity and are shipping enough tests each month. There is a great image from about ten years ago illustrating Twitter’s meteoric growth and one of the primary reasons was experiment volume:
If you are not running at least 4 tests per month per 100k uniques per month, you might not be running enough A/B tests and site experiments!
#2 Experiment Design - Experiments need to be high quality and well structured. Meaning the experiment needs to be informed by clean reliable quantitative and qualitative data. WHEN you run the experiment matters too!
To us it's not well structured if it's deployed at the wrong time! Your experiment roadmap (list of tests) needs to be prioritized based on real data. For example the ICE framework:
Source: Jexo
The hypothesis, assumptions, need to directly tie to the businesses North Star metric for growth. This also requires that the work product baked into the B / variant test that we estimate will improve performance versus the control has high quality images, speed, copy, and improvements that will resonate with the e-commerce brand requirements and ultimately the end user.
#1 Manage Expectations!
Ah yes the holy grail of the CRO Top Ten…at #1…expectations.
If you do not have alignment with the product and engineering teams that you work with that support, approve and interpret your CRO strategy, roadmap, and experimentation pipeline, then regardless of your win rate, your CRO program will be dead on arrival.
High EQ is required to unlock client feedback, have regular check ins, and allow for open ended unstructured feedback sessions to learn more about the internal communication requirements, communication styles, business structures and objectives, to ensure you are delivering communication, test result, and recommendations at the right time to the right stakeholders.
How you can use it
Pull up this post, jot down our top ten, and use it as a checklist when you are building a CRO program, or you are looking to improve and assess how things are performing each month!