The client and brief are hypothetical; the research methods, market data, and analysis are real.

Customer Retention

Food Delivery

Bootcamp exercise | Hypothetical client: Eats-2-Go

Research Question

Our company Eats-2-Go is a small but popular food delivery service. We have strong acquisition numbers but struggle to retain customers long term. Despite frequent usage, many customers exhibit loyalty challenges such as switching between apps to chase discounts and discontinuing usage after promotions end.

This study on loyalty-enhancing strategies aims to two retention strategies and identify which is the most successful over a period of three months.

My Role

I conducted the full A/B test design independently, including survey design and analysis, two loyalty program designs, hypotheses, evaluation criteria and metrics of success, calculation of sample size, and synthesis of findings into strategic recommendations.

Approach

The first task was to design and field a customer survey identifying key drivers and barriers to customer loyalty. The survey findings and insights were then used to design two interventions designed to reduce customer churn. The second task was to test the interventions using an A/B test protocol.

Research methods

Literature review on factors affecting food delivery customer loyalty
Customer survey and analysis
A/B test of two interventions designed to improve retention
KPIs and success thresholds defined

Strategic goals

Retaining an existing customer is significantly more cost-effective than acquiring a new one, estimated at 5 times cheaper. Increasing our retention rate by just 5% can potentially lead to a 25% boost in profits. A critical quarterly goal is moving new users from their first to their third order (“rule of three”). Retention typically jumps to 71% after the third order, compared to only 56% after the second.

Three performance targets were defined:

Customer Retention Rate - 25%
Repeat Purchase Rate - 33%
Time Between Purchases - 21 days

A/B Test Protocol

An intervention was designed to improve retention among new and occasional users, rather than reward regular users:

A promotional offer of $10 off the customer’s order, offered upon completion of an order to be applied to the customer’s next purchase

Test and control groups will have same selection criteria, with users meeting criteria randomly assigned into each group (50/50) as orders are received:

First order ever placed
OR Time between orders is greater than 30 days
AND Cart value > $20

Hypotheses

The impact of this intervention will be evaluated based on quarterly customer retention KPIs:

H.0 The promotion will not impact customer retention metrics (null hypothesis)
H.1 The promotion will increase customer retention rate (CRR) by 10 pp (primary)
H.2 The promotion will increase repeat purchase rate (RPR) by 20% (secondary)
H.3 The promotion will decrease time between purchases (TBP) by 10 days (secondary)

Sample Size

A power analysis (two-proportion z-test) was used to calculate the minimum sample size for statistically significant results, 486 customers per group. We used a higher threshold for statistical power than is standard - 90% for effect detection; this is because each promotion costs $10 and we want to ensure that it is effective.

Uptake rate and cost estimate

A high uptake rate of 10% is expected due to targeted in-app placement and ease of redemption. The minimum sample size will be met, with some allowance for lower eligibility and uptake rates than expected.

Analysis of Results

Primary analysis will follow an intent-to-treat approach where all customers assigned to test and control groups will be included in the analysis, whether or not they used the promotion. This allows us to interpret the results to the intervention, without the influence of self-selection bias

Each of KPI will be calculated at the conclusion of the three-month intervention period and compared between the test group (n=2000) and control group (n=2000) using appropriate tests of statistical significance. Results will determine whether observed impacts were significant enough to be attributed to the intervention, or fall within the bounds of random chance.

Metrics of Success

The intervention will be considered a success for retention if H1 is proven correct, as the primary purpose of the promotion is to improve customer retention

If H2 and H3 are proven correct, these are considered success for revenues. Both are measures of purchasing activity and will be used to evaluate the intervention’s net impact on revenues.