Bijlage 8_Banning unsolicited store flyers does helping the enviroment hurt retailing

18 Using a time window of ±26 weeks (±13 weeks) expands the set of treated and control households to 1,000 (1,060) and 37,088 (41,149), respectively, for variables unconditional on a trip observed in a week, and to 558 (713) and 21,496 (29,460), respectively, for variables conditional on a trip observed in a week. 19 As we discuss in Web Appendix E, the SC method leads to overfitting in our setting and is not considered as an alternative method here.

26

Total grocery expenditure (in euros)

TABLE 8: ROBUSTNESS OF FINDINGS Time Windows Focal Ln(Y) ±26 ±13 weeks weeks .40 .03 .34 .21 [-.45, [-.07, [-.50, [-.70, 1.24] .13] 1.17] 1.12]

Specific Cohorts Coh. 1 Without only coh. 1 -.43 .60 [-1.77, [-.40, .91] 1.60]

DID

Equal time weights

.70 [-.20, 1.59]

.56 [-.32, 1.45]

Total number of units purchased

.09 [-.62, .80]

.01 [-.02, .04]

.01 [-.69, .71]

.09 [-.64, .82]

-.38 [-1.57, .82]

.20 [-.63, 1.03]

.39 [-.37, 1.15]

.28 [-.47, 1.02]

Share of private labels (%)

-.26 [-1.02, .49]

-

-.36 [-.96, .25]

-.19 [-.80, .42]

-1.14 [-2.59, .31]

-.01 [-.89, .86]

-.28 [-1.05, .48]

-.29 [-1.06, .48]

Share of purchases on promotion (%)

-.41 [-.92, .10]

-

-.33 [-.79, .12]

-.13 [-.64, .39]

.00 [-1.03, 1.04]

-.53 [-1.11, .06]

-.42 [-.94, .09]

-.40 [-.91, .10]

Number of shopping trips

.01 [-.02, .05]

.01 [-.01, .02]

.00 [-.03, .04]

.00 [-.03, .04]

.01 [-.05, .07]

.01 [-.03, .06]

.02 [-.02, .06]

.02 [-.02, .06]

Number of different retail chains visited

.00 [-.03, .02]

.00 [-.01, .01]

-.02 [-.04, .00]

-.03 [-.05, -.01]

.04 [-.01, .09]

-.01 [-.05, .02]

.00 [-.03, .03]

.00 [-.03, .03]

Trip share HDs vs. traditional retailers (%)

-.05 [-1.09, .98]

-

-.05 [-.80, .71]

.16 [-.53, .85]

-1.11 [-3.28, 1.07]

.25 [-.93, 1.43]

-.22 [-1.26, .82]

-.16 [-1.21, .90]

Grocery expenditure at primary chain (in euros)

.19 [-.60, .98]

-.04 [-.18, .11]

.44 [-.37, 1.25]

-.05 [-.89, .78]

.25 [-.91, 1.42]

.17 [-.77, 1.11]

.55 [-.30, 1.41]

.37 [-.48, 1.22]

Number of shopping trips to primary chain

.00 [-.02, .03]

.00 [-.01, .01]

.00 [-.02, .03]

-.01 [-.03, .02]

.00 [-.05, .05]

.01 [-.02, .04]

.01 [-.02, .04]

.01 [-.02, .04]

Notes: 95% confidence interval in brackets; confidence intervals excluding zero in bold. Standard errors/confidence intervals for SynthDID are derived using the Jackknife procedure. For DID, we use two-way clustering by household and week (Berman and Israeli 2022). HD = hard discounter, coh. = cohort.

27 household to visit one retailer less). Notably, this small short-term effect goes to zero in the longer windows. The analysis that only includes Amsterdam (cohort 1) produces equally sized estimates, alsowithout any parameters reaching significance at traditional levels. The results for the analysis without Amsterdam again are small, with no significant parameters. Using the staggered two-way fixed effects DID approach yields comparable parameter estimates. The results of a more restrictive SynthDID model in which we restrict the time weights to be equal but retain the unit weights with regularization shows similar results as our focal model. Overall, the results of our robustness checks provide confidence in the validity of those obtained by our focal model. Alternatively, null results can be driven by a lack of power to detect meaningful effects. To confirm that our sample size offers sufficient power to detect economically meaningful effects, we calculate a minimum detectable effect (MDE). Rather than assess the required sample size (“How many observations are required to show an effect of X?”), we consider the effect as a function of sample size (i.e., “How small is the smallest effect we can identify based on the sample size we have?”). In Web Appendix F, we detail how to calculate Minimum Detectable Effects (MDEs) given our setting.

We find the MDEs, like the treatment effects, to be economically small (e.g., we should be able recover a change in expenditure of as little as 1.09 euro on a weekly basis, where our SynthDID and DID estimates are .40 and .71, respectively), suggesting enough power to find a meaningful effect should it be present. We provide all MDEs in Web Appendix F. Subgroup Analysis Thus far, our ITT analysis focused on the population level, i.e., the average change in behavior of all households in a municipality in which a ban is introduced, regardless of their decision to opt in or not. However, the effects can differ across households that opt in (and thus continue to receive store flyers) or households that already opted out prior to the ban (and thus continue to not receive flyers) on the one hand and households that do not opt in (and stop receiving such mail) on the other hand. To assess the effect of no longer receiving flyers, or LATE, we zoom in households for which implementing the municipality-level ban corresponded to such a change and contrast these to households for which no

28 change in store flyer reception occurs. Formally, there are three subsamples for which we can analyze the impact of a ban: households that (1) previously received flyers and do not opt in (“compliers”), (2) previously received flyers and opt in to continue (“defiers”), and (3) already opted out prior to the ban and continue not to receive flyers (“always-takers”).20,21 The first group experiences a material change after the implementation (i.e., they no longer receive flyers); store flyer reception does not change for other two groups. Web Appendix H provides household demographics for each group; both overall and by cohort. For the subgroup analysis, we separately compare each subgroup with the control group from our main analysis and summarize the results in Table 9 (see Web Appendix I for the full results).22 For defiers and always-takers, we find no statistically significant treatment effect across all dimensions, as expected. Beyond lacking statistical significance, the estimated effects also center around zero and are economically small.23 This is in line with the proposed mechanism of search cost reduction and the heterogeneity across households herein: Defiers and always-takers are more extreme in their evaluation of the benefits they derive from receiving store flyers.

For defiers, store flyers are likely to lower their search costs (to the extent that the benefits exceed the additional costs of opting in), such that they opt in to continue receiving flyers. The always-takers already indicated not wanting to receive store flyers and opted out before, and they continue not receiving them. Compliers, on the other hand, are affected by the ban. They did not opt out before but do not seem to value the benefits to outweigh the hassle or environmental costs,

20

Our data contain information about the decision to (not) opt in for about 60% of the households in treatment municipalities. To include all households in the treatment municipalities, we use a random forest algorithm to impute the sticker usage of other households (see Web Appendix G). We obtain a hit rate of 94.8%; the results are virtually identical when we re-estimate our models using the 60% of households for which we have all information. 21 In our analysis, we consider a households’ opt-in decision in the year the ban was implemented. Our data shows that 95.1% of the households that opt in after implementation are still opted in one year later and 95.8% of those who did not opt in, continue to not opt in one year later. Nonetheless, to the extent that households may change their decision, we also redid the subgroup analysis using a ±13-week window and find similar results. 22 The subgroups are defined only for treated municipalities, so we have two alternative ways to analyze subgroup effects in a SynthDID framework. The first is to re-estimate the weights ω $ !" and λ&#" for each subgroup, effectively altering the control group across the main analysis and subgroups, which would hamper comparability across the analyses. The second option runs a SynthDID for both subgroups using the weights ω $ !" and λ&#" generated in the main analysis.

This option guarantees that the control group is the same, at the risk of having a control group pretrend that may not optimally match the subgroup’s pre-trend. In line with Berman and Israeli (2022), we select the latter option. The results of the former option are virtually identical. 23 Combining defiers and always-takers into a single “non-complier” subgroup (for which there is no change in the reception of the story flyer) also does not change the results (see Web Appendix I).

29 or find the costs to contradict the new social norm too high, as they do not opt in after the legislation change. As such, they no longer learn about stores’ advertised promotions through the store flyer. Interestingly though, also for compliers we find little change in any of the shopping measures. The results show a small decrease in the share of purchases of promotion (τ# = -.97, p = .03), but economically this change of less than one percentage point is small compared to the average share of purchases on promotion in the pre-treatment period (21.6%).24 TABLE 9: RESULTS SUBGROUP ANALYSES Compliers (without Always-takers Defiers (opted in) a sticker) (opted out before) Total grocery expenditure (in euros) .18 .70 .19 [-1.11, 1.46] [-.70, 2.10] [-1.52, 1.90] Total number of units purchased -.08 -.03 .67 [-1.10, .95] [-1.28, 1.21] [-.67, 2.01] Share of private labels (%) -.55 -.03 -.25 [-1.91, .81] [-.94, .88] [-2.45, 1.94] Share of purchases on promotion (%) -.97 .27 -1.07 [-1.82, -.11] [-.45, .99] [-2.33, .18] Number of shopping trips .01 .00 .05 [-.05, .07] [-.06, .05] [-.03, .13] Number of different retail chains visited -.02 .01 .01 [-.07, .02] [-.03, .05] [-.06, .09] Trip share HDs vs.

traditional retailers (%) -1.04 .21 1.53 [-3.06, .97] [-.61, 1.04] [-2.16, 5.21] Grocery expenditure at primary chain -.02 .79 -.66 [-1.28, 1.23] [-.43, 2.01] [-2.40, 1.07] Number of shopping trips to primary chain .00 .01 .00 [-.04, .04] [-.02, .05] [-.06, .06] Notes: 95% confidence interval in brackets; confidence intervals excluding zero in bold. HD = hard discounter. Number of treated (control) households in the complier subgroup analysis: 344 (32,303) and 136 (13,474) for variables unconditional respectively conditional on a trip observed in a week; defier subgroup analysis: 372 (32,303) and 168 (13,474); always-takers subgroup analysis: 176 (32,303) and 57 (13,474).

24

As an alternative to using all households as controls in the subgroup analysis, we can impose certain rules that would qualify the eligibility of each control. While we consider simple rules that would select the appropriate set of controls infeasible, we ran a robustness check in which we consider control households that receive flyers as controls for complier and defier subgroups, and those control households that already opted out as controls for the always-taker subgroup. The results are substantively similar: We again find no statistically significant treatment effect for defiers and always-takers, and a small decrease in the share of purchases on promotion for compliers (τ" = .96, p = .03). In a second robustness check, we use the (to-be-treated) households from later cohorts that later also decide to not opt in as controls for the complier subgroup of the first cohort. For share of purchases on promotion, we find a similar effect size but with a larger p-value (τ" = -1.68, p = .11). The effects for the other eight shopping dimensions are of similar size and again not statically significant. Summarizing, using a subset of controls that are similar to the treated households before treatment or behave similarly when the treatment does happen yields substantively the same insights.

30 Treatment Heterogeneity While we observe null effects across households, it might well be possible that some households were indeed affected by the change in regulation. For policymakers, it is relevant to know whether vulnerable segments (e.g., low-income, or older households) are affected by the ban and adjust their shopping behavior. To shed light on this question, we split treated and control households into different subgroups depending on households’ (1) income (low, middle, high), (2) age in years (< 65, ≥ 65), (3) propensity to visit multiple retailers in pre-treatment period (≤ median, > median), and (4) price sensitivity (low, moderate, high25). For each of these subgroups, we re-estimate Equation 1 for each dependent variable and present the results in Web Appendix J.26 Across the 90 sets of estimates (9 dependent variables x 10 household groups), we find only two significant effects: Older households and households that are moderately price sensitive (neither disagree not agree with the statement) appear to reduce their share of purchases on promotion (τ# = -.90, p = .03 and τ# = -1.28, p = .02 respectively). Yet, the reduction is small compared to the pre-treatment average of 21.6%; importantly, it does not seem to affect the households’ overall expenditures.

Interestingly, we find no significant effects for households that are highly price sensitive but only for those that are moderately price sensitive. One explanation could be that a larger extent of the highly price-sensitive households have opted in to continue receiving store flyers. Indeed, our data shows that about 39% of the highly price-sensitive households have opted in to continue to receive store flyers versus 32% of the moderately price-sensitive households.27 We explore the relationship between household characteristics and opting in more formally in the “Profiling Households that (Do Not) Opt In to Receive Flyers” section. We first consider three possible options that

25

Based on a question from GfK’s annual household survey, “When buying groceries, I don’t really pay attention to the price”. We categorize the answer options “completely agree” and ”agree” as “low”, “neither agree nor disagree” as “moderate”, and “disagree” and “completely disagree” as “high”. In total, we have data for this question for 96% of all households (similar percentages for treated and control households). 26 The results are robust to using all control households and the same weights ω $ !" and λ&#" as in the main analysis and to re-estimating Equation 1 using all control households (cf. same group control households, e.g., low income). 27 At the same time, compared to younger households, a larger share of older households opt in (34% vs. 47%, respectively), while we still find a moderate effect on the share of purchases on promotion. This suggests that older households that do not opt in and stop receiving flyers were more likely to use flyers to guide their shopping behavior than younger households that do not opt in.

31 could explain a lack of meaningful changes in shopping behavior. Explanations for a Lack of Meaningful Changes in Shopping Behavior The preceding analyses show that banning unsolicited store flyers and switching to an opt-in system has no meaningful effect on household grocery shopping behavior, despite fears by retailers and flyer distributors that households may stick to the default option of not receiving store flyers because they find the process of opting in to be too much of a hassle or because the policy change changes what is socially desirable. As we discussed, the information provided in store flyers can guide consumers’ shopping behavior by creating awareness for promoted products and lowering search costs. While households that strongly value store flyers are likely to have opted in to continue to receive store flyers and some households already opted out before the ban, such that little changes for these groups, there is a large group of households which are meaningfully affected by the ban in terms of receiving store flyers. No longer receiving them means they no longer hear about stores’ promotions through the flyer, which may increase their search costs. Yet, our results show that even for those households, there is no substantive change in their shopping behavior.

What could explain this lack of meaningful changes in shopping behavior? Below, we discuss three possible options. First, in response to the ban, retailers may have adjusted their marketing strategy in the treatment municipalities. In the Netherlands, the content for store flyers is determined at the national level by all retailers, and most other marketing tools (e.g., TV and radio advertising) similarly target the entire country. As the implementation of the ban is staggered, adjustments to a retailers’ national media efforts are unlikely. Still, any such national changes would influence households in our treatment and control markets similarly. Retailers may have created regionally customized billboards in treatment municipalities. We interviewed a general manager of the largest Dutch grocery franchise chain and a marketing director of JCDecaux, the largest billboard provider in the Netherlands, to understand retailers’ possible reactions. The general manager signaled “a huge reliance on the store flyer” and also noted that costs saved due to the local bans had been redirected to other national advertising channels. The marketing director of the billboard provider identified “no change in sales” for billboard advertising in the

32 treatment markets, explaining that billboards do not provide a perfect substitute for store flyers, due to their general usage to highlight a few products, typically of only one brand. As such, we consider local adjustments in marketing strategy an unlikely explanation for the lack of a meaningful effect. Second, after households stop receiving store flyers, they may have adopted its digital version. Store flyers are available on retailers’ websites and through apps, and several other websites aggregate flyers from all retailers. While many households consider digital flyers inferior to print flyers (Progressive Grocer 2024; Stantec 2024),28 it is conceivable that compliers (who do not opt in to continue receiving print flyers) turn to digital versions. If so, households in treatment areas should display increased digital readership. To study this further, we use a single-wave survey, conducted in January 2021, just after the latest introduction of the ban, and obtain cross-sectional information from 2,186 households in treatment and control areas regarding their digital flyer readership (see Web Appendix K for the full details). We do not find evidence of higher readership of digital flyers in the treatment areas.

Among households in the treatment area with the most recent ban implementation (i.e., Rotterdam in November 2020, two months prior to the survey), we also do not uncover any indicator that, among digital flyer users, they started doing so at different times than households from control areas. Thus, while some households may use digital flyers, the absence of a difference between treatment and control municipalities suggest that increased adoption in response to the ban is unlikely to explain the lack of any meaningful effect. A final explanation is that the widespread dependence on store flyers may not have been as high as retailers and flyer distributors seem to believe. Previous literature already showed considerable heterogeneity in the usage of the store flyer, indicating that not all consumers who receive them, use them to decide on their shopping (Gázquez-Abad et al. 2014; van Lin and Gijsbrechts 2016), and that feature advertising only drives the store choice decision of certain consumers (Srinivasan and Bodapati 2006) and has only limited effects across the conversion funnel (Seiler and Yao 2017). Households that do not opt in

28

Survey reports suggest that digital flyers are not perfect substitutes, as many consumers consider them inferior to the experience of reading a print flyer, potentially driven by the tactile experience of reading the print flyer (Progressive Grocer 2024; Stantec 2024). Such inferiority may lead to a lower engagement with its content.

33 and stop receiving store flyers after the ban, may not have used flyers in the first place. While they can no longer hear about stores’ advertised promotions through the flyer and their search costs could have gone up, our results suggest that they did not derive many benefits from store flyers and did not rely on them to guide their shopping behavior. Put differently, the store flyer does not seem to have been a search cost shifter for these households. The concerns of retailers and flyer distributors that households stick to the default option of not receiving store flyers because they find the process of opting in to be too much of a hassle or because the policy change changes what is socially desirable, therefore seem ungrounded. While households that find store flyers worth their while likely opted in and continue to receive store flyers, households that derived little benefit from them and did not use them but did not opt out before either, no longer receive them. As such, it seems that the latter households actually stuck to the default of receiving store flyers because they found the process of opting out to be too much of a hassle or out of mere habit before the opt-in policy, such that they were fine with the change. In other words, the default change seems to have improved the sorting of households.

Summarizing, a lack of a meaningful effect could be due to adjustments in marketing strategy in treated municipalities, an increased usage of the digital store flyer, or because complying households do not use store flyers to guide their stopping behavior in any meaningful way. As we explain above, the first two explanations are unlikely such that we conclude that the 50% of households that no longer receive store flyers, were largely unresponsive to these costly marketing efforts, while those that actually find them worth their while opt in to keep receiving flyers. We seek a better profile of the potentially responsive households that opt in next. Profiling Households that (Do Not) Opt In to Receive Flyers Households that are prone to receive and read store flyers differ from other households on various dimensions, such as demographics and shopping-related characteristics (e.g., Gázquez-Abad et al. 2014; van Lin and Gijsbrechts 2016). Furthermore, research into people’s general opt-in behaviors online (i.e., permission marketing) indicates that people who opt in differ from others in their demographics and online behavior (Kumar, Zhang, and Luo 2014). Therefore, to profile households that opt in to continue

34 receiving store flyers after the ban (the “defier” subgroup), we use a binary logit model to regress their opt-in choice (1 = opt in, 0 = otherwise) on (1) household grocery shopping behavior 52 weeks before the ban (averages of total grocery expenditure, number of units purchased, share of private labels, share of purchases on promotion, number of shopping trips, number of different retail chains visited, trip share hard discounters vs. traditional retailers), (2) household demographics (household size, age of head of household, income class, social class), (3) household price sensitivity (based on their response to the statement “When buying groceries, I don’t really pay attention to the price”), (4) their readership of digital flyers (based on their response to the survey we previously reported), and (5) neighborhood characteristics obtained from Statistics Netherlands (percentage of households on social welfare, percentage of Western immigrants, percentage of non-Western immigrants, home ownership rate, average home value, population density, distance to the nearest grocery store, number of grocery stores in a 3-km radius, distance to the nearest department store, number of department stores in a 5-km radius).

We do not include grocery expenditure and number of shopping trips to the primary chain because they cause multicollinearity due to their correlation with total grocery expenditure and number of shopping trips. Table 10, column 2, contains the estimates. Not surprisingly, households that opt in (defiers) purchase a larger share of groceries on promotion (β = 2.60, p < .01), in line with findings on flyer readership by van Lin and Gijsbrechts (2016). This corroborates the face validity of our estimates. In addition, the remainder show a significant estimate for the average number of units purchased (β = .02, p= .05), such that households that purchase more units on a weekly basis also are more likely to opt in. In line with work on flyer readership by Gázquez-Abad et al. (2014) and van Lin and Gijsbrechts (2016), we also find that households that opt in are more likely to be older (β = .24, p < .01) and of larger size (β = .25, p = .03). They also come from lower social classes (β = -.18, p = .03) and reside in neighborhoods with a higher percentage of households on social welfare (β = 5.06, p < .01) and fewer grocery stores nearby (β = -.03, p = .04). Interestingly, readership of digital flyers does not correlate with opting in (p = .59), such that readers of digital flyers seem to be as inclined to continue to receive print flyers as those that do not read digital flyers.

This is in line with survey reports suggesting that households consider

35 TABLE 10: LOGIT MODEL ESTIMATES OF OPT-IN CHOICE (N = 620) Compliers Always-takers Defiers (without a (opted out (opted in) sticker) before) Estimate (SE) Estimate (SE) Estimate (SE) Household grocery shopping behavior in year before introduction of the ban Total grocery expenditure (in euros) -.01 (.01) -.01 (.01) .03 (.01)** Total number of units purchased .02 (.01)* .01 (.01) -.06 (.01)** Share of private labels (%) .19 (.75) -.71 (.68) 1.21 (.85) Share of purchases on promotion (%) 2.60 (.93)** -.70 (.87) -2.82 (1.13)* Number of shopping trips -.04 (.11) -.12 (.11) .23 (.14) Number of different retail chains visited .32 (.23) .04 (.22) -.55 (.28) Trip share HDs vs.

traditional retailers (%) .73 (.56) .13 (.54) -1.19 (.68) Household demographics Household sizea .25 (.11)* .11 (.10) -.39 (.14)** b ** Age of head of household .24 (.06) .01 (.05) -.22 (.06)** c Income class .01 (.02) -.01 (.02) .01 (.03) Social classd -.18 (.08)* .08 (.08) .11 (.10) e Household price sensitivity -.07 (.11) -.08 (.10) .22 (.13) Reader of digital flyers .10 (.19) .00 (.18) -.04 (.22) Neighborhood characteristics Percentage of households on social welfare (%) 5.06 (1.58)** -4.16 (1.57)** -.49 (1.62) Percentage of Western immigrants (%) -.63 (1.00) 1.30 (.91) -.77 (1.12) Percentage of non-Western immigrants (%) -.64 (.54) .38 (.49) .56 (.58) Home-ownership rate (%) .69 (.37) -.19 (.34) -.45 (.40) Average home value (in 100,000 euros) -.11 (.09) -.05 (.08) .21 (.10)* 2 Population density (1,000s of addresses per km ) .09 (.11) .12 (.10) -.24 (.12)* Distance to the nearest grocery store (in kms) .32 (.25) .03 (.25) -.56 (.29) Number of grocery stores in a 3-km radius -.03 (.02)* .00 (.01) .03 (.02)* Distance to the nearest department store (in kms) .01 (.11) -.15 (.11) .23 (.12) Number of department stores in a 5-km radius .08 (.05) -.02 (.04) -.07 (.05) Intercept -4.54 (1.06)** .16 (.90) 1.47 (1.06) Maximum variance inflation factor (VIF) 8.06 8.05 7.93 a

1 = 1 member, 2 = 2 members, 3 = 3 members, 4 = 4 members, 5 = 5 or more members. 1 = 12-19 y.o. (years old), 2 = 20-24 y.o., 3 = 25-29 y.o., 4 = 30-34 y.o., 5 = 35-39 y.o., 6 = 40-44 y.o., 7 = 45-49 y.o., 8 = 50-54 y.o., 9 = 55-64 y.o., 10 = 65-74 y.o., 11 = 75 y.o. or older. c Net monthly income: 1 = <700, 2 = 700-900, 3 = 900-1,100, 4 = 1,100-1,300, 5 = 1,300-1,500, 6 = 1,500-1,700, 7 = 1,700-1,900, 8 = 1,900-2,100, 9 = 2,100-2,300, 10 = 2,300-2,500, 11 = 2,500-2,700, 12 = 2,700-2,900, 13 = 2,9003,100, 14 = 3,100-3,300, 15 = 3,300-3,500, 16 = 3,500-3,700, 17 = 3,700-3,900, 18 = 3,900-4,100, 19 = >4,100. d 1 = D (lower), 2 = C, 3 = B-minus, 4 = B-plus, 5 = A (upper); based on the education level and occupation of the head of the household. e Based on a household’s response to the statement “When buying groceries, I don’t really pay attention to the price”, with 1 = completely agree, 2 = agree, 3 = neither agree nor disagree, 4 = disagree, 5 = completely disagree. * p < .05, ** p < .01. Notes: The dependent variables are defined as 1 for households that opted in/without a sticker/already opted out before (respectively), 0 otherwise.

The model is estimated on households’ opt-in choice in the first year after the introduction of the ban, excluding households for which we impute their opt-in choice, households for which we do not observe a full year of shopping behavior before the introduction of the ban, and households for which we did not obtain their response to the statement on price sensitivity or readership of digital flyers. HD = hard discounter. Variables in percentages are divided by 100 for readability. b

36 digital flyers inferior to print flyers (Progressive Grocer 2024; Stantec 2024), suggesting digital replacements may be more of a complement than a substitute. The above analysis presents retailers with a profile of potentially responsive households. However, to also understand which households do not put a sticker on their mailbox (the compliers) or already opted out (the always-takers), we present results from a similar analysis for these two groups in Table 10 (columns 3 and 4). Compliers seem less likely to reside in neighborhoods with a higher percentage of households on social welfare, but otherwise do not exhibit a discernible profile, possibly because they may be less extreme in their evaluation of the benefits of the store flyers. More interestingly, always-takers exhibit (for the most part) opposite characteristics to households that continue to receive store flyers in that they tend to spend more on a weekly basis but fewer units, purchase less on promotion, are of smaller size and younger age, reside in more affluent neighborhoods, and have access to more grocery stores nearby. All in all, these household profiles provide actionable insights for retailers in that it can help them understand the characteristics of those that prefer to continue to receive store flyer and those that do not want to receive flyers.

DISCUSSION Despite the growth of digital media, retailers seem to believe that store flyers remain a critical medium to inform consumers about advantageous price discounts and to keep their offerings at the top of consumers’ minds. But if the expensive mailers, which require paper, ink, and logistics resources as well, get thrown away unread, they serve little business purpose and are also environmentally problematic. We study a ban on unsolicited store flyers, by moving from an opt-out to an opt-in policy. Specifically, we investigate the extent to which such a change affects households’ grocery shopping behavior. The policy change resulted in a reduction of store flyer distribution by about 50%. Contrary to concerns by retailers, the results of a stacked, synthetic difference-in-differences approach indicate that households’ grocery shopping behavior did not meaningfully change. The findings are robust to different time windows and modeling approaches. Additional purpose-designed survey data corroborate the notion that shifting to digital store flyer usage is an unlikely driver of these null effects. We also do not find evidence that retailers adjusted

37 their marketing communication in treated municipalities in response to a ban. Zooming in on the compliers, who were directly affected by the change in store flyer distribution and may face higher search costs, we find no evidence of a substantial change in grocery shopping behavior either. Are there truly no changes, then? Across all analyses and robustness checks, a very small fraction of estimates are statistically different from zero, pointing to a shift in shopping behavior.29 For instance, our robustness check using a 13-week time window shows a small drop in the number of retail chains visited (i.e., .03 fewer retail chains per week), yet the effect disappears when using longer time windows. This may be suggestive of minor short-term effects in store patronage, yet we note that the effect is economically speaking very small. Further, in some of the subgroup analyses, we observe a small reduction in the share of purchases on promotion. Similarly, the magnitude of change of 1%-point is economically limited. Finally, our results also do not indicate that vulnerable segments or any other household group are differentially affected by the legislative change. We observe a 1%-point reduction in the share of purchases of promotion among older and moderately price-sensitive households, but the effect is again negligible.

Importantly, our results show that their overall expenditures remain unchanged. Overall, our results suggest a win–win–win scenario for retailers, consumers, and the environment. We thus hope our study proves useful for policymakers writing legislation pertaining to physical advertising mail and retailers that rely on them, while also stimulating further research at this important intersection of marketing and public policy. Our findings offer several contributions and insights, particularly for retail managers who, even without a ban or policy change, might wonder whom to target with store flyers. According to our results, around 50% of flyer distribution can be cut without any adverse effects on household expenditures, in line with the old advertising adage: “Half the money I spend on advertising is wasted; the trouble is, I don’t know which.” The profiling analysis suggests that household characteristics that correlate with reading flyers (e.g., promotion sensitivity) also correlate with

29

A total of nine out of 212 estimates (4.2%) reported in Tables 7-9 and in the Web Appendix attain statistical significance at p < .05. Out of the nine significant estimates, five are obtained using the robustness check using a synthetic control method, which shows signs of overfitting (see Web Appendix E). Ignoring the SC method, we obtain four significant estimates out of 203 (equaling 2.0%).

38 opting in to receiving flyers, rendering concerns that households stick to the default option of not receiving flyers unwarranted. Instead, our results seem to suggest that the share of households that lose measurable benefits due to the various costs (opportunity, environmental, and violating social norms) of opting-in is limited. As such, the ban may serve as a self-selection mechanism and improve the sorting of households.30 The results of our analysis further suggest that a purpose-designed survey that measures flyer usage and the likelihood to opt in when considering the various costs involved can be used to gain ex-ante insights on the potential effect of a ban. For retailers who have discontinued flyers unilaterally (and thus no longer distribute them to interested households either), such as OBI, the DIY market leader in Germany, and REWE the secondlargest grocery retailer in Germany (Horizont 2022), our data also indicate that a sizeable segment of customers may still prefer to receive flyers. This is particularly worrisome as competitors can continue to distribute flyers without a ban. Anecdotal evidence supports such concerns. When Walmart temporarily stopped distributing its “Weekly Ad” in a handful of test markets in 2011 and 2012, store traffic fell (Kapner 2015a).

It is worthwhile, however, to explicitly study such a one-sided withdrawal (by one retailer) and its effect on households’ shopping behavior vs. other retailers that continue to send flyers. Our study also contributes to the academic literature on retailing and promotions. More specifically, we contribute to the literature on store flyers and feature advertising by being the first to show how banning unsolicited store flyers and moving from an opt-out to an opt-in policy affects grocery shopping behavior. As we show, even this drastic change does not affect shopping behavior, in line with Srinivasan and Bodapati’s (2006) and Seiler and Yao’s (2017) arguments that store flyers appeal only to a certain segment of consumers and that they have only limited effects across the conversion funnel. Finally, our insights are relevant for policymakers considering legislation to curb the distribution

30

It is likely that either pre- or post-ban, the sorting was worse. Should a change in flyer reception not lead to a change in shopping behavior, one could consider that as evidence that those households were incorrectly sorted prior to the ban (i.e., they received flyers but did not use them), and the change in default improves the sorting. Nevertheless, in the post-ban setting, sorting may still be imperfect, as households that have opted in may still be incorrectly sorted (e.g., they may opt in due to a fear of missing out, while in practice, their shopping behavior is unaffected by the store flyer).

39 of store flyers (or advertising mail more generally). Next to different municipalities of the Netherlands, an opt-in default has been introduced in parts of Australia, Canada, New Zealand. Other governments are considering a similar policy, but face strong opposition from the industry, including retailers. In response to such opposition, policymakers can point to our results, which indicate that such fears are unfounded with respect to households’ shopping behavior and that an opt-in default can reduce waste without hurting their business. In terms of environmental savings, calculations based on inputs provided by the Association of Dutch Municipalities and flyer distributors show that the reduced distribution in the seven treatment municipalities alone (close to 1.5 million households in total) are similar to the annual energy consumption of approximately 1.7% for treated households (equal to 214,268 gigajoules) and .9% of the annual garbage (14,784 tons).

While somewhat more modest, the reduced distribution also could lower emissions by .3% for the treated households (14,481 tons of CO2) and decrease water usage by .4% (278,720 m3) per year.31 These environmental savings would roughly quadruple if the ban were rolled out nationally.32 Although these savings might be smaller compared to some other sustainability initiatives, our results suggest that the savings from a ban on unsolicited store flyers can be obtained without meaningfully affecting consumers or retailers that depend on them. Yet, it is important to note that, next to environmental savings, governments considering similar legislation also need to take into account the downstream consequences on the distribution ecosystem, including job losses, when evaluating the societal benefits.33 Importantly, the dominant revenue source for flyer distributors is tied to the number of prints (Stantec 2024). A cut in distribution by 50% will significantly reduce this number. The industry employs 20,000 delivery drivers alone and was sized close to 700 million euros in 2023 (Verbeek 2023) and the potential halving in the number of prints is likely to have a sizeable impact on jobs and the

31

To facilitate policy evaluation for municipalities, the Association of Dutch Municipalities provided a tool (“Milieu-impacttool Ja/Ja-sticker”) to assess environmental impacts, according to the opt-in rate and estimates of production costs and waste collection costs (VNG 2022). 32 We predict households’ opt-in choice in the other municipalities would they also introduce ban using a similar approach as we used to impute missing values in the treatment municipalities (see Web Appendix G), excluding the opt-in choices of geographically close households and municipality dummies. 33 We thank the Editor for suggesting this broader account of the societal benefits.

40 industry in general. Indeed, the ban has not been rolled out nationwide as several municipalities share concerns about potential side effects (see also the “Setting and Institutional Details” section) (Stantec 2024). We close by noting some limitations of our study, which suggest avenues for further research. Our study is the first to study the implementation of a ban on unsolicited store flyers. Reliance on these store flyers traditionally has been high in the Netherlands (Adformatie 2020), but store flyers are globally widespread. Our insights are promising for the many municipalities in the Netherlands that have not (yet) instituted such bans, but it is useful to determine if the observed null effects generalize across countries, some of which are in the process of implementing such changes (like France and Luxembourg). Second, we focus on grocery shopping contexts; other retailers, such as electronics and furniture retailers, which consumers visit less frequently and might not always be top-of-mind, also rely heavily on store flyers. Their flyers also feature promotions, but creating awareness of these less-frequently visited retailers represents another major purpose. We leave it to future research to test whether our results generalize to other retail industries.

Third, policymakers (and flyer distributors and retailers alike) are keen to understand the effects of the ban, and to consider what implementation details might be revised. A national adoption of the ban may be on the horizon in the Netherlands as the Dutch government commissioned an exploratory report to study the legal and practical feasibility (Stantec 2024). Given the exploratory nature of this report, ample time remains until a potential adoption of a nationwide system. We encourage research into specific implementation details that might increase opt-in or out rates (and who continues to receive flyers) and thereby influence shopping behavior. Reception might differ substantially, depending on how the ban is communicated and implemented, including an outright ban, in which case interested households no longer receive flyers either. Such an outright ban would allow one to study the value of store flyers but requires either experimental geographical variation in applying these bans or sufficient exogenous variation in flyer reception at the individual level. Fourth, our results suggest that increased usage of digital flyers (e.g., apps) is no likely explanation for the lack of meaningful effects. Indeed, survey reports indicate that many consumers consider them inferior to print flyers

41 (Progressive Grocer 2024; Stantec 2024), which could lead to a lower engagement with its content. Still, digital flyers come in many forms and some retailers may be ahead in terms of digitization and provide a better experience. Future research could test how digital flyers should be presented to improve their experience and engagement.34 All in all, we hope that our study proves useful for policymakers concerned with legislation surrounding physical advertising mail and retailers who rely on them and stimulates further research in the increasingly important intersection of marketing and public policy.

34

To measure a retailer’s level of digitization one could look at the richness of retailer apps in terms of the number of features. In a robustness check, we assessed whether shopping behavior at retailers with a lower versus higher digitization effort was differently affected but find no evidence for this. A retailer’s digitization effort may be correlated with other retailer-specific factors however, and we encourage experimental research into this matter.

42 REFERENCES Abadie, Alberto (2021), “Using Synthetic Controls: Feasibility, Data Requirements, and Methodological Aspects,” Journal of Economic Literature, 59 (2), 391-425. Adformatie (2020, November 11), “De folder is onmisbaar: zelfs de Tesla-rijder 42eft geen nee-sticker [The Store Circular Is Indispensable: Even a Tesla Driver Doesn’t Have a “No” Sticker],” https://www.adformatie.nl/gedragsverandering/de-folder-onmisbaar. Ailawadi, Kusum L. and Sunil Gupta (2014), “Sales Promotions,” in The History of Marketing Science, Russell S. Winer and Scott A. Neslin, eds. Singapore/Hanover: World Scientific-Now Publishers, pp. 463-497. Arkhangelsky, Dmitry, Susan Athey, David A. Hirshberg, Guido W. Imbens, and Stefan Wager (2019), “Synthetic Difference-in-Differences,” working paper, https://www.aeaweb.org/conference/2020/preliminary/paper/Yt6FAHSa. Arkhangelsky, Dmitry, Susan Athey, David A. Hirshberg, Guido W. Imbens, and Stefan Wager (2021), “Synthetic Difference-in-Differences,” American Economic Review, 111 (12), 4088-4118. Berman, Ron, and Ayelet Israeli (2022), “The Value of Descriptive Analytics: Evidence from Online Retailers,” Marketing Science, 41 (6), 1074-1096. Clarke, Damian, Daniel Pailañir, Susan Athey, and Guido Imbens (2023), “Synthetic Difference-in-Differences Estimation,” IZA working paper no. 15907, https://docs.iza.org/dp15907.pdf.

De Chaisemartin, Clément, and Xavier d’Haultfoeuille (2020), “Two-Way Fixed Effects Estimators With Heterogeneous Treatment Effects." American Economic Review, 110 (9), 2964-2996. Dinner, Isaac, Eric J. Johnson, Daniel G. Goldstein, and Kaiya Liu (2011), “Partitioning Default Effects: Why People Choose Not to Choose,” Journal of Experimental Psychology: Applied, 17 (4), 332-341. EUWID Pulp and Paper (2022, May 9), “Luxembourg's action on postal ads draws criticism from outside its borders,” https://www.euwid-paper.com/news/markets/luxembourgs-action-on-postal-ads-draws-criticismfrom-outside-its-borders-090522/. Fernandez, Raquel, and Dani Rodrik (1991), “Resistance to Reform: Status Quo Bias in the Presence of Individual-Specific Uncertainty,” American Economic Review, 81 (5), 1146-1155. Fox, Edward J., and Stephen J. Hoch (2005), “Cherry-Picking,” Journal of Marketing, 69 (1), 46-62. Gabel, Sebastian, Dominik Molitor, and Martin Spann (2024), “Frontiers: The Effect of an Ad Ban on Retailer Sales: Insights from a Natural Experiment,” Marketing Science, 43 (4), 723-733. Gauri, Dinesh K., Brian Ratchford, Joseph Pancras, and Debabrata Talukdar (2017), “An Empirical Analysis of the Impact of Promotional Discounts on Store Performance,” Journal of Retailing, 93 (3), 283-303. ----, K.

Sudhir, and Debabrata Talukdar (2008), “The Temporal and Spatial Dimensions of Price Search: Insights from Matching Household Survey and Purchase Data,” Journal of Marketing Research, 45 (2), 226-240. Gázquez-Abad, Juan Carlos, Francisco J. Martínez-López, and Vanesa Barrales-Molina (2014), “Profiling the Flyer-Prone Consumer,” Journal of Retailing and Consumer Services, 21 (6), 966-975. ----, and ---- (2016), “Understanding the Impact of Store Flyers on Purchase Behaviour: An Empirical Analysis in the Context of Spanish Households,” Journal of Retailing and Consumer Services, 28 (1), 263-273. Gedenk, Karen, Scott A. Neslin, and Kusum L. Ailawadi (2010), “Sales Promotion,” in Retailing in the 21st Century, Manfred Krafft and Murali K. Mantrala, eds. Berlin/Heidelberg: Springer, pp. 393-407. GfK/NMO (2024), “Bereiksonderzoek voor folders 2023/2024 [Reach Study for Flyers 2023/2024],” https://spotta.nl/resources/Pdf/Bereiksonderzoek%20Folders%202023%202024.pdf. Gijsbrechts, Els, Katia Campo, and Tom Goossens (2003), “The Impact of Store Flyers on Store Traffic and Store Sales: A Geo-Marketing Approach,” Journal of Retailing, 79 (1), 1-16. Godinho de Matos, Miguel, and Idris Adjerid (2022), “Consumer Consent and Firm Targeting after GDPR: The Case of a Large Telecom Provider,” Management Science, 68 (5), 3330-3378.

Goldfarb, Avi, Catherine Tucker, and Yanwen Wang (2022), “Conducting Research in Marketing with QuasiExperiments,” Journal of Marketing, 86 (3), 1-20. Goodman-Bacon, Andrew (2021), “Difference-in-Differences with Variation in Treatment Timing,” Journal of Econometrics, 225 (2), 254-277. GroceryDrive (2018, January 10), “Study: Consumers prefer print to digital when it comes to ad circulars,” https://www.grocerydive.com/news/grocery—study-consumers-prefer-print-to-digital-when-it-comes-to-ad-

43 circulars/534350/. Gupta, Sandeep K. (2011), “Intention-to-Treat Concept: A Review,” Perspectives in Clinical Research, 2 (3), 109-112. Guyt, Jonne, and Els Gijsbrechts (2018), “On Consumer Choice Patterns and the Net Impact of Feature Promotions,” International Journal of Research in Marketing, 35 (3), 490-508. ----, and ---- (2020), “Evaluating the Effectiveness of Retailer-Themed Super Saver Events,” Journal of Marketing, 84 (2), 92-113. Haas, Michaela (2021, December 6), “Solutions: How Amsterdam Got Rid of Junk Mail,” Atlanta Journal Constitution, https://www.ajc.com/opinion/solutions-how-amsterdam-got-rid-of-junkmail/AAR7VVYP7ZAA7MLTM4VYMPOOFE/. Honka, Elisabeth, Ali Hortaçsu, and Matthijs Wildenbeest (2019), “Empirical Search and Consideration Sets,” in Handbook of the Economics of Marketing, Jean-Pierre Dubé and Peter E. Rossi, Eds., Amsterdam: NorthHolland, pp. 193-257. Horizont (2022, July 27), “Auch Rewe verzichtet künftig auf gedruckte Werbeprospekte [Rewe will also stop using printed advertising brochures], ” https://www.horizont.net/marketing/nachrichten/ab-2023-auch-reweverzichtet-kuenftig-auf-gedruckte-werbeprospekte-201577. Imbens, Guido W. and Donald. B. Rubin (2015), “Causal Inference in Statistics, Social, and Biomedical Sciences,” New York City, NY: Cambridge University Press. Johnson, Eric.

J, and Daniel Goldstein (2003), “Do Defaults Save Lives?” Science, 302 (5649), 1338-1339. Kapner, Suzanne (2015a, March 11), “Retailers Can’t Shake the Circular Habit,” The Wall Street Journal, https://www.wsj.com/articles/retailers-cant-shake-the-circular-habit-1426113760. ---- (2015b, August 18), “Retailers Cut Back on Newspaper Circulars,” The Wall Street Journal, https://www.wsj.com/articles/retailers-cut-back-on-newspaper-circulars-1439945338. Kumar, Viswanathan, Xi Zhang, and Anita Luo (2014), “Modeling Customer Opt-in and Opt-out in a Permission-Based Marketing Context,” Journal of Marketing Research, 51 (4), 403-419. Lemon, Katherine N., and Stephen M. Nowlis (2002), “Developing Synergies between Promotions and Brands in Different Price-Quality Tiers,” Journal of Marketing Research, 39 (2), 171-185. Li, Kathleen T. (2020), “Statistical Inference for Average Treatment Effects Estimated by Synthetic Control Methods,” Journal of the American Statistical Association, 115 (532), 2068-2083. ----, and Venkatesh Shankar (2023), “A Two-Step Synthetic Control Approach for Estimating Causal Effects of Marketing Events,” Management Science, forthcoming. ----, and Garrett P.

Sonnier (2023), “Statistical Inference for the Factor Model Approach to Estimate Causal Effects in Quasi-Experimental Settings,” Journal of Marketing Research, 60 (3), 449-472 ----, and Christophe Van den Bulte (2023), “Augmented Difference-in-Differences,” Marketing Science, 42 (4), 746-767. Li, Xingyi, Yiting Deng, Puneet Manchanda, and Bert De Reyk (2024), “Can Lower Expert Opinions Lead to Better Consumer Ratings?: The Case of Michelin Stars,” Management Science, forthcoming Markteffect (2022), “JA-JA monitor,” report. McShane, Blakeley B., Eric T. Bradlow, John G. Lynch, and Robert J. Meyer (2024), ““Statistical Significance” and Statistical Reporting: Moving Beyond Binary,” Journal of Marketing, 88 (3), 1-19. Mettler, Suzanne, and Joe Soss (2004), “The Consequences of Public Policy for Democratic Citizenship: Bridging Policy Studies and Mass Politics,” Perspectives on Politics, 2 (1), 55-73. Near (2022, June 22), ““Oui Pub” : Mailbox flyers will soon be “opt in” rather than “opt out” in France – how should retailers adapt their media strategy?,” https://near.com/blogs/oui-pub-mailbox-flyers-will-soon-beopt-in-rather-than-opt-out-in-france-how-should-retailers-adapt-their-media-strategy/.

NOS (2021, October 8), “Ja-sticker beleeft opmars: 'Er is 27 miljoen kilo papier bespaard' [YES-Sticker on the Rise: 27 Million Kilo of Paper Has Been Saved],” https://nos.nl/artikel/2400839-ja-sticker-beleeft-opmarser-is-27-miljoen-kilo-papier-bespaard. Pearl, Judea (2000), “Causality – Models, Reasoning, and Inference,” Cambridge, UK: Cambridge University Press. Pieters, Rik, Michel Wedel, and Jie Zhang (2007), “Optimal Feature Advertising Design Under Competitive

44 Clutter,” Marketing Science, 53 (11), 1815-1828. Progressive Grocer (2024, April 4), “Yes, Print Ads Are Still Relevant,” https://progressivegrocer.com/yes-printads-are-still-relevant. Retailtrends (2016, April 21), “Amsterdam stemt in met ja/ja-sticker [Amsterdam Approves Yes/Yes Sticker],” https://retailtrends.nl/news/44319/-amsterdam-stemt-in-met-ja-ja-sticker. Roth, Jonathan, Pedro Sant’Anna, Alyssa Bilinski, and John Poe (2023), “What’s Trending in Difference-inDifferences? A Synthesis of the Recent Econometrics Literature,” Journal of Econometrics, 235 (2), 22182244. ----, and ---- (2023), “When Is Parallel Trends Sensitive to Functional Form?” Econometrica, 91 (2), 737-747. Samuelson, William, and Richard Zeckhauser (1988), “Status Quo Bias in Decision Making,” Journal of Risk and Uncertainty, 1, 7-59. Schultz, P. Wesley (1999), “Changing Behavior With Normative Feedback Interventions: A Field Experiment on Curbside Recycling,” Basic and Applied Social Psychology, 21 (1): 25-36. Seiler, Stephan, and Song Yao (2017), “The Impact of Advertising Along the Conversion Funnel,” Quantitative Marketing and Economics, 15 (3), 241-278. Srinivasan, V. Seenu, and Anand V. Bodapati (2006), “The Impact of Feature Advertising on Customer Store Choice,” working paper, https://papers.ssrn.com/sol3/papers.cfm?abstract_id=901458.

Stantec (2024), “Verkenning Landelijke Invoering Opt-in Systeem voor Ongeadresseerd Reclamedruk [Exploration of Nationwide Implementation of an Opt-in System for Unsolicited Advertising Mail],” https://www.rijksoverheid.nl/documenten/rapporten/2024/06/05/bijlage-3-onderzoek-landelijke-invoeringopt-insysteem. Tankard, Margaret E., and Elizabeth Levy Paluck (2016), “Norm Perception As a Vehicle for Social Change,” Social Issues and Policy Review, 10 (1), 181-211. Van Lin, Arjen and Els Gijsbrechts (2016), “The Battle for Health and Beauty: What Drives Supermarket and Drugstore Category-Promotion Lifts?” International Journal of Research in Marketing, 33 (3), 557-577. Verbeek, Jan (2023, April 29), “Folderbezorger Spotta wil Nee-stickers omzeilen [Flyer distributor Spotta wants to bypass No-sticker],” Financieel Dagblad, https://fd.nl/bedrijfsleven/1474782/folderbezorger-spotta-wilnee-stickers-omzeilen. Vericast (2020, June 11), “Unprecedented Times Prove Retailer Circulars Remain a Powerful Engagement Tool,” https://www.vericast.com/insights/blog/unprecedented-times-prove-retailer-circulars-remain-apowerful-engagement-tool/. VNG (2022), “Tool helpt bij besluit over stickers ongeadresseerd drukwerk [Tool Helps with Decision over Stickers for Unsolicited Advertising Mail],” https://vng.nl/nieuws/tool-helpt-bij-besluit-over-stickersongeadresseerd-drukwerk.

Widdecke, Kai A., Wiebke I.Y. Keller, Karen Gedenk, and Barbara Deleersnyder (2023), “Drivers of the Synergy between Price Cuts and Store Flyer Advertising at Supermarkets and Discounters,” International Journal of Research in Marketing, 40 (2), 455-474. Woolf, Amber-Leigh (2019, June 21), “Junk Mail and Windscreen Wiper Fliers Could Soon be Regulated in Wellington,” https://www.stuff.co.nz/environment/113629701/junk-mail-and-windscreen-wiper-fliers-couldsoon-be-regulated-in-wellington.

1 WEB APPENDICES BANNING UNSOLICITED STORE FLYERS: DOES HELPING THE ENVIRONMENT HURT RETAILING?

Table of Contents WEB APPENDIX A: EXAMPLES OF STICKERS USED TO OPT OUT/IN

p. 2

WEB APPENDIX B: COMPOSITION OF TREATED AND CONTROL SAMPLES

p. 3

WEB APPENDIX C: PRE-TREATMENT TRENDS

p. 5

WEB APPENDIX D: INTERVAL ESTIMATE CURVES

p. 11

WEB APPENDIX E: SYNTHETIC CONTROL METHOD

p. 13

WEB APPENDIX F: POWER TO DETECT MEANINGFUL EFFECTS

p. 17

WEB APPENDIX G: IMPUTATION OF OPT-IN CHOICE

p. 20

WEB APPENDIX H: COMPOSITION OF SUBGROUPS

p. 21

WEB APPENDIX I: RESULTS OF SUBGROUP ANALYSES

p. 27

WEB APPENDIX J: TREATMENT HETEROGENEITY

p. 29

WEB APPENDIX K: READERSHIP OF DIGITAL STORE FLYERS

p. 35

2 WEB APPENDIX A: EXAMPLES OF STICKERS USED TO OPT OUT/IN FIGURE WA1: EXAMPLES OF STICKERS USED TO OPT OUT/IN A: OPT-OUT APPROACH B: OPT-IN APPROACH

Translation: No (“NEE”) to unsolicited advertising mail | Yes (“JA”) to community newspapers

Translation: Yes (“JA”) to unsolicited advertising mail | Yes (“JA”) to community newspapers

Note: In the opt-out approach, households receive store flyers and other unsolicited advertising mail unless they use the sticker on the left. In the opt-in approach, households receive no store flyers unless they use the sticker on the right.

3 WEB APPENDIX B: COMPOSITION OF TREATED AND CONTROL SAMPLES Table WA1 show descriptive statistics of the household demographics for treated and control households across all cohorts. Table WA2 show similar statistics per cohort. TABLE WA1: DEMOGRAPHICS TREATED AND CONTROL HOUSEHOLDS, ACROSS COHORTS Treated (N = 892) Control (N = 32,303) Mean SD Min Max Mean SD Min Max Household sizea 2.09 1.16 1 5 2.42 1.22 1 5 Age of head of householdb 7.88 2.19 1 11 7.80 2.17 1 11 Income classc 9.70 5.07 1 19 10.25 4.92 1 19 Social classd 3.17 1.36 1 5 3.07 1.36 1 5 a

1 = 1 household member, 2 = 2 household members, 3 = 3 household members, 4 = 4 household members, 5 = 5 or more household members. b 1 = 12-19 y.o. (years old), 2 = 20-24 y.o., 3 = 25-29 y.o., 4 = 30-34 y.o., 5 = 35-39 y.o., 6 = 40-44 y.o., 7 = 45-49 y.o., 8 = 50-54 y.o., 9 = 55-64 y.o., 10 = 65-74 y.o., 11 = 75 y.o. or older. c Net income per month: 1 = below 700, 2 = 700-900, 3 = 900-1,100, 4 = 1,100-1,300, 5 = 1,300-1,500, 6 = 1,5001,700, 7 = 1,700-1,900, 8 = 1,900-2,100, 9 = 2,100-2,300, 10 = 2,300-2,500, 11 = 2,500-2,700, 12 = 2,700-2,900, 13 = 2,900-3,100, 14 = 3,100-3,300, 15 = 3,300-3,500, 16 = 3,500-3,700, 17 = 3,700-3,900, 18 = 3,900-4,100, 19 = 4,100 or more. d 1 = D (lower), 2 = C, 3 = B-minus, 4 = B-plus, 5 = A (upper); based on the education level and occupation of the head of the household.

TABLE WA2: DEMOGRAPHICS TREATED AND CONTROL HOUSEHOLDS, ACROSS COHORTS A: COHORT 1 Treated (N = 174) Control (N = 7,336) Mean SD Min Max Mean SD Min Max a Household size 1.84 1.04 1 5 2.38 1.20 1 5 Age of head of householdb 8.61 1.87 2 11 7.97 2.13 2 11 Income classc 8.89 5.12 1 19 9.84 4.74 1 19 Social classd 2.96 1.44 1 5 2.94 1.38 1 5

Household sizea Age of head of householdb Income classc Social classd

B: COHORT 2 Treated (N = 293) Control (N = 7,001) Mean SD Min Max Mean SD Min Max 2.23 1.21 1 5 2.42 1.22 1 5 7.62 2.31 1 11 7.79 2.20 1 11 10.17 5.11 1 19 10.18 4.87 1 19 3.28 1.32 1 5 3.05 1.36 1 5

Household sizea Age of head of householdb Income classc Social classd

C: COHORT 3 Treated (N = 183) Control (N = 6,027) Mean SD Min Max Mean SD Min Max 2.13 1.13 1 5 2.42 1.23 1 5 7.78 2.10 3 11 7.75 2.17 2 11 9.95 5.30 1 19 10.43 5.01 1 19 3.24 1.33 5 3.12 1.35 1 5

4

Household sizea Age of head of householdb Income classc Social classd

TABLE WA2 (CONTINUED) D: COHORT 4 Treated (N = 29) Control (N = 5,971) Mean SD Min Max Mean SD Min Max 2.14 1.25 1 5 2.43 1.23 1 5 7.41 2.28 3 11 7.74 2.17 2 11 8.59 4.57 2 19 10.46 5.00 1 19 3.21 1.30 1 5 3.13 1.35 1 5

Household sizea Age of head of householdb Income classc Social classd

E: COHORT 5 Treated (N = 213) Control (N = 5,968) Mean SD Min Max Mean SD Min Max 2.08 1.16 1 5 2.43 1.23 1 5 7.81 2.19 2 11 7.73 2.17 2 11 9.67 4.78 1 19 10.46 5.00 1 19 3.10 1.38 1 5 3.13 1.35 1 5

a

1 = 1 household member, 2 = 2 household members, 3 = 3 household members, 4 = 4 household members, 5 = 5 or more household members. b 1 = 12-19 y.o. (years old), 2 = 20-24 y.o., 3 = 25-29 y.o., 4 = 30-34 y.o., 5 = 35-39 y.o., 6 = 40-44 y.o., 7 = 45-49 y.o., 8 = 50-54 y.o., 9 = 55-64 y.o., 10 = 65-74 y.o., 11 = 75 y.o. or older. c Net income per month: 1 = below 700, 2 = 700-900, 3 = 900-1,100, 4 = 1,100-1,300, 5 = 1,300-1,500, 6 = 1,5001,700, 7 = 1,700-1,900, 8 = 1,900-2,100, 9 = 2,100-2,300, 10 = 2,300-2,500, 11 = 2,500-2,700, 12 = 2,700-2,900, 13 = 2,900-3,100, 14 = 3,100-3,300, 15 = 3,300-3,500, 16 = 3,500-3,700, 17 = 3,700-3,900, 18 = 3,900-4,100, 19 = 4,100 or more. d 1 = D (lower), 2 = C, 3 = B-minus, 4 = B-plus, 5 = A (upper); based on the education level and occupation of the head of the household.