A TFI research challenge project by Benjamin Timmermans, Monique van Maare and Eva Zinger-Mityok
Posted on April 09, 2018

The research project "The effect of warnings on irresponsible online purchase behaviour" is one of the eight projects selected last year for the TFI Research Challenge.

In this report, the effectiveness of warnings during check-out on online spending is explored. Benjamin Timmermans, Monique van Maare and Eva Zinger-Mityok conducted two online experiments to test their hypotheses on the type of message and the related emotional responses. 


Contrary to the economic assumption that people behave rationally, we rarely weigh the consequences of all of our options - especially the costs and benefits of options that we give up on. Ariely and Kreisler argue that this lack of consideration for so-called opportunity costs "is our biggest money mistake and the reason we make many other mistakes" (Ariely and Kreisler 2017).  

In online buying, this is especially evident. We are subjected daily to advanced, targeted marketing, social media pressure and continually improving online buying journeys. This triggers our impulse to buy, and “disconnects” us from our long-term future needs, like saving for our pension or even paying our mortgage. The user experience for online payments also continues to improve, with features like wallets, one-click ordering and shortened authentication processes, further lowering the “pain” we feel when we buy online.

A low “pain of paying” is associated with higher spending (Van der Horst, and Matthijsen 2013). It has been associated with a larger gap between the time of payment and the time of consumption, and with a payment method in which the payment details, such as the payment amount, are given less attention.

So, as the percentage of low-pain online retail purchases grows to $4 trillion in 20201 and 17.5 percent of all retail commerce in 20212, we can expect problematic spending to increase. In the Netherlands, 22% of the population already has a problematic financial situation (Van der Schors, 2015), credit card debt in the US is at an all-time high3 and the UK is reportedly “in the grip of a personal debt crisis”4.

What is needed to stimulate more financially responsible behaviour? Financial literacy education geared towards improving forward-thinking financial capability, like “how to save for retirement” or “how to select a mortgage” tends to have limited impact on our day-to-day actions. Even experts get it wrong - in the US 46% of financial advisors do not have a retirement plan themselves (Ariely and Kreisler 2017). Thaler and Sunstein have argued that “nudges” offer a more effective way to steer people towards better choices (Thaler and Sunstein, 2008). Nudging is a way to selectively present the options available, so that we do not have to rely on self-control alone to help us act responsibly. So how can we "nudge" people towards financially responsible behaviour? Creating effective "triggers" in their day-to-day digital actions may offer a better chance of success. (Weinmann et. al. 2016).

Can warnings provide “digital nudges” to consider opportunity costs?

In this study, we investigate the effectiveness of warnings presented to the online buyer in triggering more responsible decision-making. For the purpose of this study, we define an irresponsible purchase as one where the buyer goes into debt. The warnings are presented as the last step before committing to a purchase. We designed these warnings to highlight the opportunity cost of the purchase. We expect this to lead to lower spending by raising the “pain of paying”.

Our experiments consist of a questionnaire in which we simulate online purchases of shoes, electronics or concert tickets. These products were chosen as items that are generally desired, but not always essential, and within a similar price range. Once the participants have chosen their preferred object, they are presented with a brief description of the purchase situation, and the payment screen, where the participant can choose to buy the product or cancel the purchase. For every purchase, the bank account balance is lower than the price they are about to pay, therefore completing the payment means going into debt. We vary the warning messages at the time of payment, and record whether the participant decides to cancel rather than buy the product.

The warnings were designed to highlight a negative consequence of the purchase, such as “Warning! You have insufficient balance to complete this purchase” or “Are you sure? You may not reach your savings target this year.”

We distributed our questionnaires via the crowdsourcing platform Crowdflower 5. Crowdsourcing as a data gathering method has demonstrated its value in allowing for quick, cheap, realistic and reliable testing of research scenarios (Timmermans, 2015) as well as providing reliable input for creating predictive models. One of the distinguishing features of this method is the ability to assess participant reliability. The platform itself pre-scores the reliability of participants based on previous tasks. In addition, we included several measures to allow evaluation of the reliability of the crowd worker's answers. Through these measures we filtered out 28% of potentially unreliable data, and we used the remaining 3375 judgements for data analysis.

“The most effective warning reduced spending by 78% compared to no warning”

Warnings are effective in reducing spending

The goal of our study was to understand if warnings can reduce spending, and if so, which type of warnings are most effective. We varied the warnings in severity, concreteness and timing of the consequences. Warning severity refers to the level of danger in the expected negative consequence that is expressed with the warning, as in “Are you sure?” versus “Warning!” Warning timing refers to the timeframe that a described consequence will take effect: this week, this month, or this year. Warning concreteness refers to the level of abstraction in the warning messages. For example, "You may not be able to make further purchases this month" is more abstract than " “Warning! Your account balance will be -$300,-, the interest rate is 14%, or $42 per year". Our expectations were that severe and concrete warnings would lower spending, and that short-term consequences would be more effective than long-term consequences in lowering spending.

Overall, we found that warnings at the time of payment are an effective mechanism to reduce spending. Without a warning, the purchase was completed in 50% of the cases. With a mild warning (“Are you sure? You may not be able to make any further purchases this month”) this was reduced to 34%. The more severe warning (“Warning! You have insufficient balance to complete this purchase”) was the most effective warning with only 11% of purchases completed, reducing spending compared to the no-warning scenario by 78%. We found that warnings describing a concrete financial consequence were more effective than more abstract warnings. For the warnings with a short-term versus a long-term consequence, the results were mixed.

For low-income groups the short-term consequence “Are you sure? You don’t have sufficient balance to complete this purchase right now” was highly effective. For high-income groups, we saw a particularly strong response to the long-term concrete statement of “Warning! Your account balance will be -$300, - the interest rate is 14%, or $42 per year.” We propose that the impact assessment strategies differ per income group and suggest additional study could further clarify what aspects are responsible for this difference.

To understand if there was an emotional impact of the warnings, after each purchase scenario, participants were asked to record their emotional state, which they selected from a list of emotions from the Consumer Emotion Scale (Richins 2007). We found that showing a warning was associated with a more negative emotional response. We interpret this as a confirmation that warnings generate a higher “pain of paying,” especially because the warnings that were most effective in reducing spending also produced the most pronounced negative emotional response.

To assess financial capability for our participants we selected two questions from the OECD Financial Behaviour study (Atkinson 2016): "Before I buy something, I carefully check whether I can afford it" and “I pay my bills on time”. We found that people who score themselves lower on these financial behaviour questions are consistently more likely to complete the purchase. We conclude that this group is less affected by the warning messages. We consider it important to do further research to confirm this with a larger dataset, to identify ways to better engage this group. 

We also asked participants if they appreciated being warned in this way of potentially irresponsible purchases. The response, echoed in the free-format feedback about the task, was overwhelmingly positive, with 80% participants indicating they would “turn on” such warnings.

What’s next?

This study consists of fictitious payment situations. The current results will need to be replicated in actual rather than fictitious payment situations. The appreciation for the functionality expressed by the participants provides a positive reinforcement to take the results forward to real-life payments. One application area we envisage is to build in warning mechanisms in iDeal, a method of payment in the Netherlands that enables people to pay online through their own bank. Since we expect similar effectiveness for credit card and mobile payments, these would also be potential application areas. Clearly, regulatory and other stakeholder concerns, such as retailers’, will need to be considered, so a focus on higher-risk groups, where any improvement in financially responsible behaviour is a common interest, would be a good starting point for implementation.

The current research focused on warnings highlighting a negative consequence of the purchase. In future scenarios we would like to explore the effect of positive consequences, as with “Would you like to save for your vacation instead?” and provide positive reinforcement in case of payment cancellation, as with “Congratulations, you managed to save $300 for your vacation”.

To further improve the effectiveness of the warnings, it would be helpful to improve the predictive model for cancelling the payment. In addition to the severity, timing and concreteness of warning, elements such as buyer demographics including age, their current financial situation across different banks, a more encompassing financial capability score, buyer’s current emotional state and other factors could be tested to improve the current model. The model would also benefit from incorporation of psychological factors. Shephard et. al. (2017) find that psychological variations such as in optimism, impulsiveness, goal orientation and locus of control are predictors of financially capable behaviour.

Further optimalisation of the warning will then be possible, through increased relevance to the personal situation of the buyer. Enriched in this way, any implementations based on this model are likely to require buyer opt-in and stringent privacy safeguards. It remains to be tested if with such increased personal relevance the high level of appreciation for the functionality is retained, or whether this starts to be seen as annoying or intrusive.

This individual line between welcome “nudges” to help improve responsible online buying behaviour, while preserving the buyer’s sense of autonomy and freedom of choice, will require fine-tuning. However, this study shows that the "last chance-moment" between purchase decision and definitive payment clearly offers many opportunities for helping buyers to act more responsibly online.


  1. Online buying statistics from:
  2. Retail statistics from:
  3. U.S. credit card debt figures from:
  4. U.K. household debt quotes from Guardian article:
  5. The crowdsourcing platform used:


  • Ariely, D. (2008) Predictably irrational. The hidden forces that shape our decisions. New York: HarperCollins Publishers. Ariely, D. (2002) Payment method design: psychological and economic aspects of payments. Center for ebusiness@mit. Presented November 13,2002 Weekly Research Lunch Seminar. Ariely, D. and Kreisler, J. (2017) Dollars and Sense: Money mishaps and how to avoid them. Bluebird. Atkinson. 2016. OECD/INFE International Survey of Adult Financial Literacy Competencies. edited by O. I. N. o. F. Education. Paris: OECD. Harari, D. Household debt: statistics and impact on economy. House of Commons Library Briefing paper, Number 7584, 15 December, 2017. Horst, F. van der, and Matthijsen, E. (2013) The irrationality of payment behaviour, DNB Occasional Studies, Vol.11/No.4 (2013). De Nederlandse Bank N.V. Marsha L. Richins. Measuring Emotions in the Consumption Experience Journal of Consumer Research, Vol. 24, No. 2 (September 1997), pp. 127-146. Published by Oxford University Press. Marsha L. Richins (2007), "Consumption Emotions," in Product Experience: Perspectives on Human-Product Interaction, eds. Hendrick N. J. Schifferstein. Schors, A. van der, Werf, M. van der, Schonewille, G. Geldzaken in de praktijjk, 2015, Nibud study. Daniel D Shephard Beyond Financial Literacy: the psychological dimensions of financial literacy. Think Forward Initiative Technical Report, June 2017. Strathman, A., Gleicher, F., Boninger, D. S., & Edwards, C. S. (1994). The consideration of future consequences: Weighing immediate and distant outcomes of behavior. Journal of Personality and Social Psychology, 66, 742–752. Thaler, R.H. and Sunstein, C.R. Nudge: Improving Decisions about Health, Wealth, and Happiness, Yale University Press, New Haven, 2008. Timmermans, B., Aroyo, L., & Welty, C. (2015). Crowdsourcing ground truth for question answering using crowdtruth. In Proceedings of the ACM Web Science Conference (p. 61). ACM. Weinmann, M., Schneider, C. & Brocke, J. Bus Inf Syst Eng (2016) 58: 433. Https:// Zellermayer, O. (1996). The Pain of Paying. Ph.D. Dissertation, Carnegie Mellon University