by Peter Story | September 5, 2022
This blog post summarizes research presented at The Privacy Enhancing Technologies Symposium in July 2022. The research was conducted by a team from Clark University, Carnegie Mellon University, University of Michigan, and the University of Maryland.
As part of our research, we used an experiment to test the effectiveness of different nudging interventions at increasing adoption of Tor Browser. We found that our nudge based on Protection Motivation Theory nearly doubled the odds that participants would use Tor Browser. Our results also show that users commonly encounter usability challenges when using Tor Browser, and that people use Tor Browser for a variety of benign activities. Our study contributes to a greater understanding of factors influencing the adoption of Tor Browser, and how nudges might be used to encourage the adoption of Tor Browser and similar privacy enhancing technologies.
Browsing privacy tools can help people protect their digital privacy. However, our research suggests that the tools which provide the strongest protections (e.g., Tor Browser) are less widely adopted than other tools. This may be due to usability challenges, misconceptions, behavioral biases, or mere lack of awareness. This convinced us to test ways to increase adoption of Tor Browser.
Specifically, we tested the effectiveness of three different nudging interventions, designed to encourage adoption of Tor Browser. First, we tested an informational nudge based on protection motivation theory (PMT), designed to raise awareness of Tor Browser and to help participants form accurate perceptions of it. Next, we added an action planning implementation intention (AP), designed to help participants identify opportunities for using Tor Browser. Finally, we added a coping planning implementation intention (CP), designed to help participants overcome challenges to using Tor Browser, such as extreme website slowness. We tested these nudges in a longitudinal field experiment with 537 participants.
The quote below is an excerpt from our PMT-based nudge. We wrote this text to help participants accurately gauge their susceptibility to browsing privacy threats. Our nudge addressed well-defined threats and common misconceptions about other tools’ protections, as suggested by our prior work.
Many different organizations can gather information about your browsing activity. Here are just a few examples:
And unfortunately, most browsing tools offer only partial protection against these privacy threats. For example:
- Private browsing only partially hides your browsing from advertisers, and does nothing to hide your location from websites or your browsing from your internet service provider or the government
- Most VPNs do nothing to hide your browsing from advertisers, many VPNs keep logs which can be accessed by the government, and some VPNs even spy on their users
- Ad blockers only partially hide your browsing from advertisers, and do nothing to protect against other privacy threats
Other equally important parts of the PMT-based nudge addressed the protections offered by Tor Browser, and gave instructions for using Tor Browser effectively. Our action planning nudge encouraged participants to list privacy-sensitive activities they planned to use Tor Browser for in the coming week. Our coping planning nudge explained how to overcome common challenges to using Tor Browser; for example, using the “New Circuit” button to resolve extreme website slowness. Our paper contains more details about our nudges.
We found that our PMT-based nudge increased use of Tor Browser in both the short- and long-term; participants who saw our PMT-based nudge were nearly twice as likely to report using Tor Browser as those in our control group. Our coping planning nudge also increased use of Tor Browser, but only in the week following our intervention. We did not find statistically significant evidence of our action planning nudge increasing use of Tor Browser.
The tables below summarize these findings. For odds ratios, 1.5, 2, and 3 are the conventional thresholds for small, medium, and large effect sizes, respectively. Results significant at α = 0.05 are bolded. Only participants who encountered challenges could be given the opportunity to form coping plans to overcome those challenges, which is why we tested those comparisons separately.Short-Term Effects of Nudging Interventions Comparison Use of Tor Browser Odds Ratio p-value Control vs PMT Survey 3: 14.9% vs 24.2% 1.83 0.026 PMT vs PMT+AP Survey 3: 24.2% vs 29.8% 1.33 0.125 PMT+AP vs PMT+AP+CP Survey 4: 34.4% vs 40.0% 1.27 0.173 Comparison, for those whoencountered challenges PMT+AP vs PMT+AP+CP Survey 4: 42.3% vs 65.9% 2.64 0.027 Long-Term Effects of Nudging Interventions Comparison Use of Tor Browser Odds Ratio p-value Control vs PMT Survey 5: 15.4% vs 27.3% 2.05 0.011 PMT vs PMT+AP Survey 5: 27.3% vs 32.2% 1.26 0.211 PMT+AP vs PMT+AP+CP Survey 5: 32.2% vs 29.2% 0.87 0.691 Comparison, for those whoencountered challenges PMT+AP vs PMT+AP+CP Survey 5: 47.6% vs 41.5% 0.78 0.678
Our results suggest that there are opportunities to increase adoption of Tor Browser using nudging techniques, particularly those based on protection motivation theory (PMT). Certainly, not everyone is interested in using Tor Browser. However, our nudging techniques show that many people are willing to give it a try, and that our PMT-based nudge can encourage a significant percentage to continue using Tor Browser in the long term. We also tested nudges based on action and coping planning implementation intentions. Although we did not find evidence of these plans further increasing long-term adoption of Tor Browser, those who were given the opportunity to form coping plans were more likely to use Tor Browser in the subsequent week.
Several things should be considered when translating our results to a real-world deployment of nudges. First, our participants knew our nudges were part of a research study. However, how people respond to information depends on which entity delivers that information. Nudges may be more or less effective depending on how people perceive the entity administering the nudges. Tor Browser itself might serve as a trusted messenger for nudges, perhaps incorporating nudges into the browser’s homepage, or displaying them in the UI when various challenges are encountered. For example, if Tor Browser can determine that a website is blocking Tor users, Tor Browser might explain this and recommend using an alternative website.
Second, we only recruited participants who we thought would be highly motivated to use Tor Browser. Specifically, we recruited participants who had prior experience with other privacy tools, and who expressed a high level of interest in preventing at least one privacy threat Tor Browser can protect against. Our intuition was that it would be easier to detect the effects of our nudges among these participants in our experiment; perhaps similar targeting should be employed when deploying nudges in the wild.
Finally, additional research is needed to fully understand the effects of our nudges. We asked people to report whether they used Tor Browser during our study, but we did not capture finer naunces of their behavior. For example, our PMT nudge reminded participants that Tor Browser’s protections are reduced if one logs in to websites. Did participants understand and follow this recommendation? Also, our action planning nudge was designed to help people identify opportunities to use Tor Browser. Although it did not significantly increase the number of participants who reported using Tor Browser in the previous week, did it increase consistency of using Tor Browser? If someone regularly performs a particular privacy-sensitive activity, their identify can be revealed if they forget to use Tor Browser even once. Similarly, our coping plan was designed to help people continue using Tor Browser in spite of challenges. Free text responses from participants suggest their coping plans were helpful to them, though we don’t know this definitively. Future work should explore whether nudges can have these kinds of positive effects.
Did people encounter challenges using Tor Browser?
How commonly did people encounter challenges when using Tor Browser? Did our coping plans address the most common types of challenges? We asked the study participants who reported trying to use Tor Browser whether they had encountered any challenges. The majority of these participants reported encountering some form of challenge, with extreme website slowness being the most common, and websites not working being the second most common.
Our findings suggest that people are likely to encounter challenges when using Tor Browser, so it is important to address these challenges using coping planning or other approaches. Our coping plan templates addressed the two most commonly encountered types of challenges, supporting the validity of our experiment. Please refer to our paper for details about the “Other” challenges.
What activities do people use Tor Browser for?
We encouraged participants in our action planning treatment group to make plans for using Tor Browser. In their plans, we invited participants to list privacy-sensitive activities they might perform using Tor Browser. If they didn’t want to disclose a certain activity, we told them to write “prefer not to disclose.” In total, participants wrote 598 activities, which we coded to identify common themes.
In the table below, we summarize the ten most common types of activities reported by participants. The “Described” column shows the number of times participants described each type of activity. The “Performed” column shows the number of activities participants reported actually performing, while the “Performed Using Tor Browser” column shows the number of activities participants reported using Tor Browser to perform.Activities Code Description Described Performed Performed Using Tor Browser PREFER NOT TO DISCLOSE Either the literal text “prefer not to disclose,” or something close to it. 192 100 33 SHOPPING Looking up information about consumer products, regardless of intention to purchase. 55 39 13 FINANCIAL Looking up information about financial products (e.g., stocks, bitcoin), mortgages, banking, insurance, salaries, applying to jobs, etc. 49 35 8 VAGUE A vaguely defined activity, such as “using a search engine” or “researching things.” 49 41 25 NEWS Looking up information about politics, celebrities, current events, document leaks, etc. 40 29 17 NSFW Pornography or other “Not Safe For Work” content. 36 29 15 MEDICAL Accessing medical information. Includes personal care and cannabis. 33 21 11 VIDEOS Watching videos, movies, or streaming. 26 18 10 YOUTUBE Using YouTube. 25 22 12 SNOOPING Looking up information about non-celebrities (e.g., ex’s, friends, background checks) or similar entities (e.g., employers, competitors). 20 11 4
Our participants shared that they used Tor Browser for many innocuous activities, including shopping, reading the news, and researching medical topics. This stands in contrast to the illicit nature some ascribe to Tor Browser. Our findings suggest that websites may benefit from supporting Tor traffic to cater to privacy-sensitive visitors. For example, the New York Times is available as an Onion Service.
This is a companion discussion topic for the original entry at https://blog.torproject.org/boosting-adoption-tb-behavioral-science/