Hey all – just wrapped up experiment 3, and I’m ready to report on results:
In my last experiment, I had gotten my cost per conversion (conversion, in this case = people signing onto my email list, which then triggers them going through an auto sequence to drive them into starting interviews for the goodvalchal). Down to $8 by leading with a $1k grant offer. This was a key finding because this was a low enough cost to get a lot of people in my funnel to test a simplified process to get them filling out interviews. Key Goal: Create a process that was automated and scalable to get people filling out interviews.
Hypothesis 1: My email sequence is too long and impersonal. Make it more personal, include an autofill field for their name, get straight to the point, and we’ll exceed a 25% open rate on our email sequence.
Hypothesis 2: Focusing our ads exclusively on ones that mention a $1k grant and running Facebook ads will result in a cost per conversion that is less than $7.
Failure condition 1: Less than a 25% open rate.
Failure condition 2: Overall cost per conversion greater than $7
Result: Hypothesis 1 was confirmed. By shortening the email sequence, and getting to the point, we exceeded a 30% open rate. The emails were simple, basically saying, here’s where you get started on filling out interview forms, call me if you have trouble.
Hypothesis 2 was not confirmed. We focused on the $1k grant and dropped our cost per conversion to $8, a good number, but not our $7.
Thus, failure condition 1 was not met, but condition 2 was.
Conclusions: While we did most of what we wanted with conversions, this experiment shows a fatal flaw in my assumptions. I assumed that conversions would = action. They did not. Of the 50 people going through the automated sequence, I only see one who has filled out 3 interviews as a result. This was disheartening, as we nearly succeeded in every other way, but action is the only metric that really matters in this case and we failed to deliver.
Way forward: For experiment 4, I will spend no money. Conversions mean nothing without action taken by the person converting. Instead, I’m going to test the assumption that people who have paid us (even a nominal amount) to attend a webinar on ‘Lean Teachings for Social Good’ (which includes the mention of a $1k grant) will be more likely to do the interviews because they have skin in the game and have demonstrated commitment by spending 45 minutes and cash on a webinar. I’m going to use organic channels and the new email list I created from experiment 3 to market this.
Experiment 4 Hypothesis: By getting 25 people to pay $19-39 attend a 45 minute webinar on ‘Lean Methodology for Social Good’ I will get 50% of attendees to complete 25+ interviews within 15 days of the webinar.
Failure condition 1: Less than 25 attendees.
Failure condition 2: Less than 50% of attendees completing 25+ interviews within 15 days of webinar.