Nudge is a data-driven organization. We take pride in ensuring our clients are aware of every facet of their Nudge programs by providing them with useful, clear, and actionable reporting on the progress of their campaigns. We want to make sure that our clients have the knowledge they need to make the smartest decisions for their business.

But we’re also focused on assessing our own impact on the businesses that subscribe to our platform. As Nudge’s Lead Data Scientist, one of my top objectives is to oversee the advanced analytics practice with an aim of uncovering and quantifying the “Nudge Effect” across various sectors. Put in a different way, my job is to figure out an investment into Nudge’s frontline enablement solution can impact a business for better or worse. 

Our approach to quantifying the “Nudge Effect”

We approached this problem as any reputable scientist would – from a place of neutrality. We weren’t looking to prove out the positive effects of Nudge, but rather to disprove that Nudge had no impact at all (i.e. the  null hypothesis). 

And despite the advanced nature of our quantitative approach, we also wanted to ensure that our reporting, story-telling and presenting would be accessible to all, regardless of technical capabilities. After all, we’re not creating reports for academics. All of our reporting is tailor-made with memorable narratives, captivating and informative visuals, and aesthetically pleasing designs. No one’s background or level of technical capabilities should be a hindrance to understanding our findings – and the subsequent discussions.

Our quest to quantify the “Nudge Effect” has progressed over the course of the last year and a half. During this time, we completed six in-depth analytics studies that have each sought to disprove the null hypothesis (that Nudge has no impact) from a different perspective. 

Methodology

Our methods were relatively simple. KPI metrics relating to performance and success were provided to us by our clients from the telecommunications, retail, and foodservice sectors. These KPI variables were audited, cleaned, and categorized into various types, like operational efficiencies and upsell/point-of-sale (PoS) behaviors.  Once the data had been validated and merged with our own platform usage metrics at the location level, we were finally ready for some stats. 

We had a hunch we wanted to validate: Locations that had more employees active on the Nudge app on a regular basis would naturally do better on other performance/success KPIs. The challenge was, how can we know for sure?

To test this hypothesis, we employed a series of generalized bivariate linear regression modeling techniques. You might remember regression models from high school or first-year undergrad courses. These models allow us to ask basic questions of what happens to performance when average Nudge usage increases at the store level.

Quick sidebar on generalized bivariate linear regression modeling for those of you (like my editor) that had no idea what I was talking about. Conceptually, the technique is simple. Essentially, we plot the distribution of two variables onto a scatter chart and draw a line of best fit. Despite its simplicity, this model (i.e. the straight line and its properties) provides a powerful way of interpreting a generalized relationship between two variables, and allows us to ask some basic questions. How tightly are these two variables related to one another (i.e. correlation)? When one variable increases, how does the other respond (i.e. direction of relationship)? And what proportion of a variability in a given KPI can be explained by Nudge usage? 

It should be noted that given our datasets were organized at the store level and spanned over a number of weeks, we were able to employ more advanced regression modeling techniques that accounted for repeat sampling across time. This was necessary to ensure we weren’t missing any underlying patterns that may not have been obvious at the aggregated level. However, regardless of which technique was employed we consistently found the same patterns – and came to the same conclusions. 

Results

In most cases, we were comfortably able to reject our null hypotheses – which, again, stated that Nudge usage had zero impact on KPIs at the store level. Our models revealed intricate, statistically significant and, at times, meaningfully strong relationships between Nudge usage and various success KPI metrics. 

Among the most notable findings was how Nudge usage relates to operational efficiencies and PoS KPIs. The table below shows a summary of these findings:

The Nudge effect can be quantified by the average effect size, or the point percentage difference between locations with all staff active on Nudge vs. locations with no staff active on Nudge. 

What we learned about the “Nudge Effect”

Our studies have uncovered strong evidence that investment into frontline communication with Nudge is linked to more effective institutional procedures and processes. Nudge’s real value becomes apparent when we look into its effects on operational efficiencies and effective task execution relating to how a location is organized, maintained, and managed on a day-to-day basis. It is these broader operations, facilitated with more effective communication that lead to better customer service journeys and stronger frontline retention. 

This was a fascinating discovery, especially paired with some of the recent data uncovered in The Deskless Report, our annual in-depth research into the state of the frontline world. For one thing, task efficiency is one of the top indicators of an enabled frontline. When asked how they know when their staff has what they need to succeed, 56% of corporate respondents said “efficient task/process execution.” Furthermore, 80% of all respondents, which included corporate leaders, frontline managers, and frontline workers, believe that task management tools can effectively impact frontline challenges and help workers thrive. 

Let’s take a deeper look at the patterns uncovered in some of the most notable instances of “The Nudge Effect.”

How Nudge usage relates to operational efficiency

The largest effect was found among fast food chain locations. Locations that had all employees active on the Nudge app on a weekly basis had service wait times reduced by an average of two minutes and 20 seconds (a ~47% reduction), presumably leading to a better customer experience overall. This effect was found independent of location size, meaning it did not matter if a location was a small three-person operation or a larger 30-person fast food power house. In either case, Nudge usage was objectively linked to customer experience and operational efficiency. Furthermore, Nudge usage was found to explain over 20% of the variation in customer service wait times. This may seem relatively low, but in the world of quantitative relations where random noise and incomplete or insufficient data often drowns out emerging patterns, it’s huge. 

How Nudge usage relates to retention

Another signal we uncovered was in a different – but arguably more important – facet of operational efficiency: employee retention. We found that in a foodservice provider that operated in retirement homes, locations with higher Nudge usage had significantly higher employee retention, explaining roughly 10% of the variability in our retention spanning a nine-month period. 

Although we can only refer to it as a correlation based on this study, we can theorize that retention is likely based on a variety of independent factors. A possible factor that may explain this causal relationship comes from The Deskless report, which asked frontline workers about the top drivers of frontline success and happiness. 46% of workers said a strong employee community drives happiness and success, and 41% said empathetic leadership, both of which can be indirectly achieved with Nudge – and in turn, may influence retention. 

How Nudge usage relates to upsell behaviors

The next batch of KPIs that we found Nudge usage was moderately related to can be classified as Upsells/Point-of-Sale (PoS) behaviour KPIs. These are metrics based on interactions that occur at the point-of-sale touchpoint. Examples include signing up customers to a loyalty program, or adding a promotional product relating to the customer basket. Roughly speaking, Nudge was found to explain between 4 to 14% of the variation observed in these KPIs – a small to moderate effect but statistically significant and worthy of consideration nonetheless. 

Real-world experimentation is required to elucidate the exact causal effect for this relationship, but that shouldn’t stop us from theorizing the why. At its core, Nudge is a frontline enablement platform that, when set up properly, (under the direction and guidance of our wonderful CS team), can effectively communicate and reinforce frontline associates on current store offers, tips, and promotional messaging that they can use to upsell shoppers at the point of sale. In this case, Nudge’s Spark, Buzz and Knowledge functionality was used that specifically revolved around educating, reminding and encouraging associates to offer customers a checkout offer associated with their purchase (such as batteries, books or affiliated toys) or to sign up for their customer loyalty program. 

—

Over the last year and a half, the patterns uncovered by our in-depth analytics practice infer a strong set of positive correlations between Nudge usage and key performance metrics – indicating that Nudge positively impacts customer experience, reinforces employee execution and retention, and leads to more favorable customer interactions at the point of sale. While more data and further modeling is required before we can infer a clear cause for these positive trends, the numbers indicate we’re heading in the right direction.  

At Nudge, we continually strive to demonstrate the value our platform brings our clients using high quality data and quantitative methods. We recognize it from our qualitative reporting, and we delineated it in our quantitative analysis: Nudge usage and frontline performance have an affinity for one another. Operational efficiencies, point-of-sale behaviors, and task execution are top priorities for our clients. Statistically speaking, it is no coincidence that locations that use Nudge more often also fare better on all three of these fronts. Something to think about 😉.