The channel knockout
The gene knockout experiment can be easily repurposed on a growth team to measure channel effectiveness.
The channel knockout experiment
Borrowing from genetics to quickly measure channel effectiveness.
Many times startups are faced with incomplete data problems. The problem is that the data is incomplete and time is always against you. You have a few choices, wait until your data is great or great creative.
Let’s get creative.
Our DNA codes for about 100,000 different genes. These genes control everything from eye color, to disease predisposition. There is also a fascinating body of research around epigenetics that you should definitely read, but that is neither here nor there. We are talking about growth right?
Right.
Scientists are often presented with the same incompleteness of data problem. They have several genes but aren’t 100% sure which ones do what. What do they do? They systematically remove each gene and watch what happens. This is called a gene knockout.
Wait, this is about growth right?
Yes, here goes. Let’s say you are promoting an app in the App Store. While there are solutions like Tapstream and other iOS tracking applications, the data you get from Apple is less than complete. However, to successfully promote your application you are likely using the following channels to varying degrees:
- facebook cpi ads
- twitter cpi ads
- google cpi ads
- social
- in app promotion
- third party services
- blog outreach
- etc
The quickest way to measure effectiveness is to preform an effective channel knockout experiment. You can do so in three simple steps.
- Get a baseline by running all channels for 1–2 weeks. Your key metric is downloads / day across all channels.
- Stop all activity in a given channel for 48 hours.
- Calculate your difference and voila, you have a quick and dirty measure for the effectiveness of that channel + CPI if there is a cost associated with it.
Pro tips:
- Don’t do this on a Fri-Sat-Sun-Mon. Tues-Wed are your best days.
- Stop all activity in the channel. No cheating.
- If you do this with a paid channel. Take your cost savings for that 48 period and divide by the net loss in downloads to get an approximate CPI.
You are looking for a graph that looks like this.
The key to continuous and strong growth is identifying levers that you can pull when you need to. A channel knockout experiment is a fantastic way to ballpark your levers quickly and effectively.
Stay beautiful.
Kent Stop, Won’t Stop
