Peak-Coincident Demand Savings from Residential Behavior-Based Programs: Evidence from a Large-Scale Field Experiment Abstract: This study estimates the impacts of a utility’s behavior-based program on residential electricity use during the utility’s system peak hours. Between June 2010 and May 2013, PPL Electric sent home energy reports to approximately 100,000 residential customers as part of a large field experiment. The reports encouraged customers to adopt electricity savings measures that would reduce electricity use during the utility’s system peak. We collected hourly energy-use data between June 2012 and September 2012 for 20,000 homes in the behavior program’s treatment and control groups. Panel regression analysis of 44 million hourly energy use records indicated that the average peak demand savings were about 0.07 average kW per home, which translated to a system peak reduction of about 6.5 MW. PPL’s cost of demand savings was about $185/average kW. This compares to a U.S. utility average cost of savings of $164/average kW for residential load management programs (EIA, 2012). The study shows that residential behavior programs can help utilities to manage their system peak loads. While PPL Electric’s behavior program was more expensive than traditional demand response programs, more focused messaging about reducing peak energy use or targeting of high electricity users may help it achieve cost parity.
James I. Stewart* June 15, 2014
______________________________________
*Senior Economist, The Cadmus Group Inc. Contact information: 720 SW Washington St., Portland, Oregon, 97214. Tel: 503-799-4562. Email:
[email protected] or
[email protected]. I wish to thank Pete Cleff at PPL Electric, Kevin Cooney, and Hossein Haeri for helpful comments as well as participants at the 2013 Behavior, Energy, and Climate Change Conference and Portland State University Economics Department workshop. The views in this paper are my own and do not necessarily reflect PPL Electric’s or those of my employer, The Cadmus Group. All errors are my own.
1
1. Introduction Increasingly, U.S. utilities are employing load-management programs to reduce residential
electricity use during system peak hours when the marginal cost of supplying electricity is high. Examples of such programs include utility direct load control and dynamic pricing.
Direct-load-control programs enable a utility to reduce customer loads during peak hours through remote communication and control technologies installed on residential customer
air conditioners and water heaters. Dynamic-pricing programs charge customers higher prices for electricity during peak hours. Both types of programs are effective at reducing
residential loads during system peak hours (Allcott 2011b; Faruqui and Sergici 2009;
Newsham and Bowker 2010; Wolak 2006).
This paper evaluates the effectiveness of another type of intervention to help
electric utilities to manage system-peak loads: residential behavior-based programs.
Applying insights from the behavioral social sciences, these programs use non-price
interventions to encourage utility customers to reduce electricity consumption across a variety of end-uses. 1 Examples of behavior-based interventions include providing customers with feedback about their energy use, educating them about energy savings
opportunities, comparing their energy use to that of their peers, providing them with
rewards for conserving energy, or encouraging them to participate in tournaments,
1
For an overview of behavior-based program strategies, see Ignelzi, P., J. Peters, L. Dethman, K. Randazzo, and
L. Lutzenhiser, 2013. For studies on how the programs impact energy use, see Ayres, Raseman, and Shih
2009; Costa and Kahn 2010; Allcott 2011; Allcott and Rodgers forthcoming; Agnew, Gaffney, and Rosenberg 2013, Kahn and Wolak 2013.
2
competitions, or games. Behavior-based programs have the potential to help utilities to
manage their system peaks because the programs can target electricity end-uses such as air
conditioning that contribute significantly to system peaks. In addition, the programs can
reach thousands of customers, so that even a small reduction in peak energy use per home
can translate into a large overall reduction in system demand. Finally, behavior-based programs may help utilities reduce peak energy consumption of residential customers who may not be interested in participating in dynamic-pricing or direct-load-control programs.
We estimate the effect of a home energy reports program on the system peak loads
of PPL Electric, a large electric distribution company in central and eastern Pennsylvania.
The behavior-based program, implemented by Opower, sent six energy reports per year to approximately 50,000 homes between June 2010 and May 2013 and to about 55,000
additional homes between June 2011 and May 2013. In the rest of this paper, we refer to
the first group of homes as the Legacy Group (LG) and the second group as the Expansion Group (EG). The reports provided a summary of each home’s historical energy use, a
comparison of the home’s energy use to that of similar homes in its neighborhood, and tips about how to make the home more energy efficient. Many of the tips such as increasing
home insulation levels, replacing inefficient windows, buying an energy efficient air conditioner, and adjusting thermostat settings focused on electricity savings from air conditioning, which contributed significantly to system peaks.
An important feature of PPL Electric’s behavior program was that it was
implemented as a field experiment using a randomized control trial (RCT) design. Opower
randomly assigned homes eligible for the program to a treatment or control group. Homes 3
in the treatment group received the energy reports while those in the control group did not receive reports and were not informed about the study. The program’s experimental design
ensured that receiving the energy reports was orthogonal to energy use characteristics, and that any difference in post-treatment peak energy use between treated and control homes reflected the program’s effect.
Panel regression analysis of 44 million energy use records between June 2012 and
September 2012 of 20,000 randomly-sampled PPL Electric homes in the behavior program
treatment and control groups showed that the average demand savings during system peak hours were about 0.08 kWh/hour per LG home and 0.06 kWh/hour per EG home. Savings
per LG and EG home were about 2.2% and 1.7%, respectively, of peak energy use and about 150% of average demand savings during non-peak hours. The peak savings were
approximately equivalent to turning off one 60 watt or 80 watt light bulb or reducing the running time of a typical residential central air conditioner by six to eight minutes per
hour. For the behavior-based program overall, PPL Electric achieved peak demand savings
of about 6.5 MWh/hour.
An interesting finding was that LG and EG homes had different demand savings
patterns. In LG homes, demand savings were higher during system peak hours and
increased during the afternoon as outside temperatures and energy use for air conditioning climbed. The positive correlation between demand savings and system peak hours likely reflected the adoption of measures to reduce electricity use for air conditioning in LG
homes. In contrast, in EG homes, demand savings during system peak and non-peak hours
were approximately equal, and demand savings did not increase significantly during the 4
afternoon. Electricity demand savings in EG homes likely derived from sources other than
air conditioning measures. This difference in savings patterns between LG and EG homes is likely due to inclusion of a large number of homes in the EG that had participated in the
PPL Electric energy efficiency programs before receiving the first reports. It is likely that
previous efficiency program participants had already adopted measures to reduce
electricity use for air conditioning.
The cost of peak demand savings from the energy reports program was about
$169/average kW in the LG and $221/average kW in the EG. PPL’s average cost of demand savings for the whole program was about $185/average kW. This compares to an average annual cost of savings for the typical utility residential demand response program of
$164/average kW (EIA 2011). Thus, the behavior-based program’s cost of peak savings
was only about 12% higher than that of a utility’s traditional load management program.
This paper makes several contributions. First, it is one of the first to use high
frequency data to estimate electricity savings from a residential behavior-based program
(Allcott and Rodgers forthcoming; Gilbert and Graff Zivin 2013; Kahn and Wolak 2013;
Todd 2014). 2 High frequency electricity-use data has not been widely available because of
privacy concerns. Previously, to estimate demand savings from residential behavior
2
Allcott and Rodgers (forthcoming) use daily electricity use data to track the durability of savings from a
behavior-based program during the program and persistence of savings after the program. Gilbert and Graff
Zivin (2013) find that homes reduce their energy use during afternoon and early evening peak energy-use
hours in the week after receiving their utility bill. Kahn and Wolak (2013) estimated the effect on electricity
use of educating utility customers about the marginal rate of electricity they face. Customers who found out they faced a high rate reduced their electricity use between 3 and 5%.
5
programs, evaluators have used surveys of program participants about the timing of
specific actions taken to save energy and assumptions about the distribution of energy savings over hours of the month or year. The approach in this paper does not rely on self-
reported actions or such assumptions.
Third, this study shows that the energy efficiency of a home before receiving energy
reports affected reductions in peak energy use. Homes that had not participated in a utility energy efficiency program before treatment had the greatest savings during weekday
afternoon and evening hours. In contrast, homes that had participated previously in an
efficiency program had the largest savings during morning hours. These savings patterns are consistent with previous efficiency program participants having already adopted measures to save electricity used in air-conditioning. 3
Finally, this study shows that utilities can use home energy reports programs to
help manage their system peaks. Previous studies show that such reports can reduce annual electricity use between 1% and 3% (Allcott 2011a; Allcott and Rodgers
forthcoming; Ayres, Raseman, and Shih 2009; Costa and Kahn 2010). Though the main
objective of PPL Electric’s behavior-based program was energy savings, the program
yielded significant demand savings during the utility’s system peak. The utility’s cost per
kW of savings for the behavior program was more than that for a traditional utility demand response program, but the difference was not large. With more focused messaging on
saving electricity during peak hours or targeted marketing of potential high savers, it might 3
Gilbert and Graff Zivin (2013) also find transitory impacts of monthly utility bills on electricity use that is
consistent with households reducing electricity used in air conditioning.
6
be possible for the behavior-based program to achieve cost parity with some direct load-
control and dynamic-pricing programs. Finally, the behavior program enabled the utility to reduce the peak loads of residential customers who might not have been interested in
participating in a traditional utility demand response program.
2. PPL Electric’s Residential Behavior-Based Program PPL Electric serves residential and non-residential customers in several large
metropolitan areas as well as in surrounding ex-urban and rural areas in Pennsylvania.
Figure A1 in the Appendix shows a map of the utility service area. The utility experiences
peak loads in the summer, typically during warm afternoons and evenings when residential air-conditioning loads are high. All of the utility’s residential customers have smart meters installed that record energy use (kWh) at hour intervals.
This utility’s behavior-based program began in April 2010 when the program
implementer, Opower, sent energy reports to 50,000 of the utility’s residential customers. All eligible customers had: (1) detached, single-family dwellings; (2) above-average energy
use (at least 16,000 kWh/year), which indicated the home likely had electric heat and air conditioning; (3) a complete monthly billing history at the same address in the previous 12 months; (4) did not share a meter with another home; and (5) a sufficient number of
neighbors with similar characteristics. Each home received between six and eight reports in the first year and six reports in each of the next two years.
In May 2011, PPL Electric expanded the program to include an additional 55,000
residential customers. Customers in this group either had above-average electricity use or 7
had previously participated in one of PPL Electric’s energy-efficiency programs. 4 Twenty-
five percent of EG homes had participated in an efficiency program before receiving energy
reports. In contrast, less than four percent (3.8%) of LG homes had participated in a PPL
Electric efficiency program before receiving reports. As noted above, we refer to the
original group of homes that received reports as the legacy group (LG), and the second group of homes to receive reports as the expansion group (EG).
Each energy report was customized and included three information modules.
Appendix Figure 2 shows an example of a report. The first module provided an analysis of
the home’s energy-use history, indicating how electricity use in the current year compared to that in the previous one. The second module was a social comparison of the home’s electricity use to approximately 100 similar neighbors. Several previous studies have suggested normative comparisons of energy use can motivate customers to reduce their
consumption (Allcott 2011a; Allcott and Rodgers, forthcoming; Ayres, Raseman, and Shih,
2009; Costa and Kahn 2010; Nolan et al. 2008). The third module provided action steps and custom tips for reducing electricity use at home. The action steps promoted PPL Electric programs, which included rebates on energy-efficient appliances such as clothes washers,
refrigerators, and air conditioners. The tips also targeted many end-uses contributing to PPL Electric’s system peak loads.
PPL Electric and Opower implemented the behavior-based program as a field
experiment, employing a randomized control trial design. Opower identified homes eligible 4
PPL Electric’s efficiency programs mostly provide rebates for the purchase of high efficiency appliances and other
energy using products such as air conditioners, refrigerators, and CFLs.
8
for the program and randomly assigned them to either a treatment or a control group.
Homes in the treatment group received energy reports while those in the control group did
not receive reports and were not informed about the study. The LG control group included 50,000 homes, and the EG control group included about 25,000 homes. Because homes were assigned to receive home energy reports at random, receiving a report was
uncorrelated with energy use, and any significant difference between treatment and
control group homes in post-treatment electricity use could be attributed to the program. As part of the analysis, we verified that the annual and monthly consumption of homes in
the treatment and control groups during the 12 months before the program started was statistically equal, consistent with the random assignment of homes to the treatment.
While this study focuses on estimating demand savings during system peak hours
between June and September of 2012, we also estimated annual energy savings using monthly customer bills. The monthly energy savings can be converted to average hourly
savings and serve as a point of reference for the analysis of hourly savings. Figure 1 and
Figures 2 show monthly energy savings, respectively, per LG home between June 2010 and May 2013 and EG home between June 2011 and May 2013. During the twelve months
preceding the summer of 2012 (June 2011-May 2012), LG homes saved an average of 306
kWh/year or 0.035 kWh/hour. EG homes saved 317 kWh/year or 0.036 kWh/hour. 5 The
5
The savings estimates were based on a panel regression analysis of average daily consumption on home
fixed effects, month-by-year fixed effects, and interactions between the month-by-year fixed effects and an
indicator for membership in the treatment group. Confidence intervals were estimated using Huber-White standard errors clustered on homes. See PPL Electric (2012).
9
savings are in the range of those reported in evaluations of other Opower behavior-based
programs (Allcott, 2011a; Allcott and Rodgers, forthcoming; Rosenberg, Agnew, and Gaffney, 2013).
[Insert Figure 1 here]
[Insert Figure 2 here]
3. Hourly Energy Use Data Data for this study came from PPL Electric’s customer information and billing
system as well as from Opower. PPL Electric provided electricity-use data from AMI meters installed at residential customers’ premises. PPL Electric also identified the 100 hours of
highest system electric demand during the summer of 2012 and supplied data on customer demographics,
socio-economics,
housing
characteristics,
and
efficiency
program
participation. Opower provided data about which homes received home energy reports and report frequency.
We randomly sampled 5,000 treatment group homes and 5,000 control group
homes from each of the LG and EG populations for the demand savings analysis. The energy-use data covered 2,232 summer hours between June 15, 2012 and September 15,
10
2012. 6 Altogether, the data include over 44 million hourly energy-use records for 20,000
homes.
Random sampling of homes from the LG and EG populations had the potential to
introduce sampling error in the savings estimates. 7 We used monthly electricity use data
to verify that the electricity use of sampled and non-sampled homes (as well as of sampled homes in the treatment and control groups) in the year before the program was balanced. Table 1 shows results of t-tests of balance between sampled treatment and control homes
in pre-treatment annual average daily consumption, summer average daily consumption, and energy-efficiency program participation.
[Insert Table 1 here]
The differences between sampled treatment and control homes in pre-treatment
electricity use were small and statistically insignificant, less than or equal to 0.2 kWh/day
or 0.0083 kWh/hour. There were also not any significant differences between sampled treatment and control homes in previous efficiency program participation.
Figure 3 and Figure 4 provide additional evidence about the equivalence of sampled
treatment and control homes before treatment, showing the distributions of pre-treatment
6
We are not aware of other papers that have estimated behavior program peak demand savings using hourly
energy-use data. One study used daily energy-use data to estimate energy savings and savings persistence
and durability from Opower’s home energy reports program (Allcott & Rodgers 2012). 7
Instead of analyzing energy use of all program homes before and after the start of the program, we sampled
from the program population and collected data for summer months during the program. It was felt this would minimize the burden on the utility of pulling a large amount of energy use data.
11
electricity use between June 2009 and September 2009 for treated and control LG homes and between June 2010 and September 2010 for treated and control EG homes. [Insert Figure 3 here] [Insert Figure 4 here] In both of the LG and EG samples, the distributions of pre-treatment energy use of
treated and control homes were almost identical. Balance between sampled homes was important because hourly electricity-use data for the pre-treatment period was not available, and the estimated program treatment effects were therefore based on a simple
difference rather than a difference-in-differences of hourly electricity use. Any significant
time-invariant differences in average electricity use between homes in the treatment and control groups could bias the savings estimates.
Figure 3 and Figure 4 also illustrate a key difference between the LG and EG
populations. As noted above, the EG included homes with above-average electricity use as well as a significant percentage (about 25%) of homes that had participated in a PPL Electric energy-efficiency program before treatment. Energy-efficiency program
participation reduced electricity use, so the EG would have included some homes with relatively low energy use. In fact, the left tail of the EG electricity-use distribution had relatively more homes than the corresponding tail of the LG distribution. The EG
distribution had a fatter right tail as well, suggesting that the EG also included a relatively larger number of high consumption homes than the LG. 12
3.1 PPL Electric System Peak Hours and Residential Electricity Use Pennsylvania’s Act 129 defines system peak hours as the top 100 hours of a utility’s
system demand. In 2012, PPL Electric’s top 100 hours of demand occurred on 16 summer
weekdays (three in June, nine in July, and four in August) when outdoor temperatures and
air-conditioning loads were very high. Figure 5 shows the incidence of the utility’s system
peak hours across weekday hours and the average temperature during each hour. About 90% of system peak hours occurred between 12:00 p.m. and 7:00 p.m. temperature during system peak hours was about 90 degrees.
The average
[Insert Figure 5 here]
Figure 6 and Figure 7 show average electricity use per LG and EG control-group
home during weekdays with one or more system peak hours and weekdays without any system peak hours.
[Insert Figure 6 here] [Insert Figure 7 here]
In legacy and expansion group homes, electricity use followed similar trends on peak and non-peak weekdays. Energy use increased over the day, reaching a peak between 4:00 p.m.
and 6:00 p.m. This increase likely reflected the effect of higher outdoor temperatures on
demand for electricity used in air-conditioning. As expected, demand was higher on days with system peaks, and the difference was greatest during the late afternoon and early evening when air conditioning demand was greatest. Demand was approximately 1 kW or 33% higher between 4:00 p.m. and 6:00 pm on system peak days. 13
4. Peak-Coincident Demand Savings Estimates In this section, we report estimates of the behavior program’s average treatment effect
during the utility’s system peak hours. Residential loads, particularly for air conditioning,
are a significant contributor to the utility’s system peak. As such, it is reasonable to expect that the home energy reports program, which encouraged, among other measures, more
efficient use of air conditioning, may have resulted in significant demand savings.
Let electricity use, kWhit, of home i in hour t, t=1, 2, …, 2,320, between June 15, 2012
and September 15, 2012 be given by the following regression equation: kWhit =β1Top100(1)t*Treat(1)i + β2(1-Top100(1)t)*Treat(1)i + τt + εit
(Equation 1)
where Top100(1)t=1 if hour t was a system peak hour, and Top100(1)t=0, otherwise;
Treat(1)i=1 if home i received a home energy report and Treat(1)i=0, otherwise; τt is an
effect specific to hour t representing average electricity use in sampled homes; and εit is the random error term for home i in hour t. The parameter β1 represents the behavior-based
program conditional average treatment effect during the utility’s system peak hours in
kWh per hour. The parameter β2 represents the conditional average treatment effect during non-peak hours, also in kWh per hour. It is expected that β1 < 0 and β2 < 0.
Equation 1 includes hour fixed effects (as shown) or Top100t(1) to control for
positive correlation between residential electricity use and system peak hours caused by
residential demand for air conditioning. With these controls, savings are measured as the 14
difference in energy use between treatment and control homes net of any difference in
average energy use between peak and non-peak hours. Without them, the model would ignore the fact that residential electricity use is highest during system peak hours and
estimates of the behavior program’s savings would be biased downwards. 8 This approach will control for all variables, not just those related to weather, that might have been correlated with both residential electricity use and system peak hours.
Note also because of the experimental program design, the estimated treatment
effects are net of other activities that may have reduced peak demand such as participation in the utility’s demand response programs. In addition, the estimated treatment effect is
really an “intent to treat” because some homes were “non-compliers”, that is, they opted out of the program, or the householders who manage the use of electricity and pay the
energy bills disregarded or failed to notice the reports. Therefore, the estimated treatment effect reflects the average effect of complying and non-complying homes on peak energy use. 9 8
Another approach to account for this correlation is to include weather variables such as cooling degree
hours as explanatory variables on the right side of Equation 1. For a given base temperature, say, 65 degrees,
a cooling degree hour (CDH) equals max(0, outside temperature-base temperature). A CDH is supposed to capture the effect of outside temperature on electricity demand for space cooling. A drawback of this
approach is, however, that it is necessary to make functional form assumptions about the relationship
between weather and energy use. Another drawback is that the approach would not control for correlation between εit and Top100t caused by other non-weather-related unobservable variables.
9At
the end of 2012, 1.5% of treated homes in the LG had opted out of the program. At the same time, about
1.3% of treated homes in the EG had opted out. Phone surveys in 2012 of PPL Electric behavior program
15
We estimated Equation 1 separately for homes in the LG and EG populations. The
models were estimated by ordinary least squares (OLS) with Huber-White standard errors clustered on homes to account for correlation in each home’s energy use across hours (Bertrand, Duflo, and Mullainathan, 2004).
4.1 Peak-Coincident Behavior Program Savings Estimates We first estimated Equation 1 with a general intercept without controls for any
positive correlation between residential electricity use and system peak hours. Estimates
of savings during system peak hours for LG and EG homes were negative and statistically significant, suggesting that the program caused electricity-use to increase (results not
reported but available from the author). Rather than indicating that the program caused energy use to increase, however, the results indicate that the model was specified
incorrectly. Energy use of treated homes during peak hours was measured relative to
control group average energy use across all hours rather than average energy use during peak hours.
Next, we estimated Equation 1 with the necessary controls for the positive
correlation between home energy use and system peak hours. Table 2 shows the results
from the estimation of Equation 1 for homes in the LG and EG populations. As the savings
estimates for models with Top100t(1) (Models 1 and 3) and those for models with hour treated homes did indicate most householders recalled receiving the reports. Over 96% of survey
respondents said they remembered receiving and at least glancing at one or more reports (95% confidence with ±3% precision).
16
fixed effects (Models 2 and 4) were very similar, we focus the following discussion on the latter set of models.
[Insert Table 2 here] The behavior-based program saved electricity in treated homes during both system
peak and non-peak hours. The program average treatment effect during system peak hours was to reduce electricity use by about 0.08 kWh/hour per LG home (Model 2) and 0.06 kWh/hour per EG home (Model 4). These effects were statistically significant and
equivalent to turning off one 60-watt or 80-watt light bulb or reducing the run-time of a
typical residential central air conditioner by about 6-8 minutes per hour. Peak savings per LG and EG home were about 2.2% and 1.7%, respectively, of average electricity use during
system peak hours. 10 Savings per home were smaller in the EG than the LG likely because of
differences between the groups in either the duration of treatment or pre-program energy use. At the beginning of the analysis period, June 2012, LG homes had received energy
reports for about two years while EG homes had received them for only about one year. As noted, the EG also included a larger percentage of more efficient homes.
During non-peak hours, average savings were smaller, about 0.05 kWh/hour per LG
home and about 0.04 kWh/hour per EG home. The average electricity savings per home was about 33% smaller during non-peak than peak hours. 10
Average electricity use during the top 100 hours was 3.60 kWh/hour per LG control home and 3.65
kWh/hour per EG control home.
17
Figure 8 and Figure 9 provide additional perspective about the program treatment
effects during system peak hours, showing average demand savings per treated home
during system peak hours between 10:00 a.m. and 10:00 p.m. and non-peak weekday hours.
[Insert Figure 8] [Insert Figure 9] In LG homes (Figure 8), average demand savings during system-peak and non-
system peak hours increased over the day, reaching a maximum in the late afternoons and
early evenings. Increasing use and savings of electricity for air conditioning likely explains this pattern, though higher rates of home occupancy in the late afternoon might also have
contributed. In addition, demand savings were greater during system-peak than nonsystem peak hours except between 8:00 p.m. and 10:00 p.m. when savings were approximately equal. This pattern suggests that demand savings derived from reductions
in electricity use in air conditioning because it was electricity use for air conditioning that
significantly increased during system peak hours.
In contrast, in EG homes (Figure 9), average demand savings did not increase
between 12:00 p.m. and 7:00 p.m, instead remaining approximately constant (during non-
system peak hours) or slightly decreasing (system peak hours). In addition, savings during system peak and non-system peak hours were approximately equal. These patterns suggest
that, in contrast to savings in LG homes, there was little correlation between savings and PPL Electric system peaks and that savings were not strongly correlated with either 18
weather or hour of the day and probably unrelated to air conditioning. The participation of
many EG homes in PPL Electric efficiency programs before the start of the behavior program could explain this pattern. If some EG homes had previously installed measures that increased air-conditioning efficiency, their electricity use and savings would have shown less sensitivity to outside temperature.
To understand better the effect of previous efficiency-program participation on
program treatment effects, we estimated the average savings per EG home that had
participated in a PPL Electric efficiency program before treatment and the average savings
per EG home that had not participated before treatment. Savings were estimated across
peak and non-peak weekday hours. Figure 10 shows the results. Though some of the
savings are estimated imprecisely, it is evident that demand savings patterns of previous efficiency program participants and non-participants differed.
Previous participants
achieved the highest demand savings between 5:00 a.m. and 10:00 a.m. Their demand
savings then decreased during the afternoon and evening. In contrast, homes that had not participated previously achieved demand savings similar to those of LG homes. Savings
increased over the late morning and afternoon hours before decreasing in the early
evening. This pattern is consistent with energy reports savings having originated from measures to reduce energy use for air-conditioning. The pattern of savings for previous
efficiency program participants is inconsistent with demand savings from air conditioning
measures because savings decreased during hours when air conditioning was greatest. As
previous participants had taken measures to reduce their energy use before receiving the 19
reports, it is possible that they had already taken steps to increase air-conditioning efficiency.
[Insert Figure 10 here]
4.2 Behavior Program Peak-Coincident Demand Savings Table 3 shows the average demand savings for the energy reports program during
system peak hours. We estimated the program system peak savings by multiplying the
peak demand savings per home by the average number of homes in the population that
received the treatment and whose billing account was active during system peak hours. 11 [Insert Table 3 here]
Aggregating the savings per home over the populations of treated homes, we find
that the average demand savings over the top 100 hours was about 3.5 average MW for LG homes and 3.0 average MW for EG homes. 12 The average peak demand savings for the
whole program was 6.5 average MW, which was enough electricity to meet the peak
electricity demand of approximately 1,800 LG or EG control group homes. The 90% confidence interval for the savings was fairly wide, [3.9 MW, 9.1 MW].
11
Homes that opted out of the program (less than 2%) were considered as having received the treatment and
included in the energy use analysis if their accounts were still active. 12
The number of treatment group homes with an active billing account during the top 100 hours averaged
43,208 for the legacy group and 49,510 for the expansion group.
20
4.3 Cost of Peak-Coincident Demand Savings from PPL Electric’s Behavior-Based Program How much did system peak hour savings from the behavior-based program cost PPL
Electric? Table 4 shows the average cost per kW of peak-coincident savings for homes in
the LG and EG. For these calculations, we assumed the utility’s cost of the behavior-based
program was $13.50 per treated home per year. 13 The readers should keep in mind that the main objective of the behavior-based program was energy savings, which would reduce the electricity bills of PPL Electric customers and count toward the utility’s electricity savings goals mandated by Pennsylvania’s Act 129. Demand savings were a secondary objective.
As such, a high cost per kW does not imply that the program was not cost-effective. In fact,
the behavior-based program proved to be highly cost-effective from the perspective of total resource cost in each year that it was evaluated. 14
[Insert Table 4 here]
In 2012, the cost of demand savings from PPL Electric’s behavior program was
estimated to be about $169/average kW in the LG ((1/0.061)*$13.50) and $221/average kW in the EG ((1/0.80)*$13.50). Overall, the weighted average cost of peak demand savings
was about $185/average kW. The higher cost of savings for EG homes reflects the fact that
they had smaller peak-coincident savings per home. The 90% confidence intervals are
wide, reflecting uncertainty about the true demand savings (see Table 3). For example, the
13
This calculation assumes the program implementer sent reports to 50,000 homes in the LG for two years
and 55,000 homes in the EG for one year. (PPL Electric, 2012, p. 51) 14
PPL Electric, 2012, p. 51.
21
confidence interval for the savings cost per LG home has a lower bound of $106/average kW and an upper bound of $419/average kW.
5. Comparison of Savings and Costs with Utility Demand Response Programs How do the peak-coincident savings of PPL Electric’s behavior-based program
compare to those of other residential demand response programs? 15 Critical peak pricing (CPP) or time-of-use pricing can save between 30 and 40% of peak loads (Wolak 2007;
Faruqui & Sergici 2009; Bowker & Newsham 2010; Allcott 2011a). Utility direct load-
control of residential air-conditioning can save between 0.5 kW and 1.2 kW per home
depending on: (1) the cycling or thermostat control strategy, (2) the time of day when events are called, (3) outside temperature, and (4) underlying air-conditioning use
patterns (Newsham and Bowker 2010). Direct load control of water heating can save between 0.3 kW per home and 0.7 kW per home and depend on a similar set of factors.
The peak demand savings from PPL Electric’s energy reports program were about 5
to 6.5% of the expected demand savings per home for a typical CPP program, 5 to 10% of
savings per home for a typical AC-DLC program, and 10 to 25% of savings per home for a
typical WH-DLC program. In other words, the utility would need to deliver home energy reports to 15 to 20 homes for every home on a CPP rate, 10 to 20 homes for every home in 15
An important difference between behavior-based programs and many demand response programs is the
ability of the utility to call on many demand-response resources as needed. In contrast, most behavior-based
programs, like time-of-use pricing, cannot be called upon to produce peak demand savings.
22
an AC-DLC program, and four to 10 homes for every home in a typical WH-DLC program. In
2012, savings from sending energy reports to nine PPL Electric homes were approximately
equal to the average savings per home achieved by the utility’s residential air-conditioning direct load control program (0.62 kW/home).
How does PPL Electric’s average cost of demand savings of $185/kW from the
behavior-based program compare to the average cost of savings for utility residential demand response programs in the United States? Figure 11 shows the comparison.
According to U.S. Department of Energy’s Energy Information Administration, in 2012,
utilities in the United States paid an average of $164/kW in direct costs and customer
incentives for savings from residential load management programs. 16 Thus, the cost of
acquiring demand savings through PPL Electric’s behavior-based program was about 112% of utilities’ average cost of demand savings from residential load management programs. PPL Electric’s behavior-based program would have needed to generate peak-coincident
demand savings of approximately 0.08 kW per home to achieve cost parity with the
average demand response program. If customer incentives, transfers from utilities to
participants, are excluded, PPL Electric paid 280% of the utility average cost of demand savings from residential load management programs. Thus, from a societal resource
16
We obtained 2012 cost and kW savings from EIA Demand Side Management Program data available at
http://www.eia.gov/electricity/data/eia861/. We collected data on residential load management program actual peak reductions (MW) and residential load management annual direct costs and customer incentives for U.S. utilities in 2012.
23
perspective, kW savings from the behavior-based program were more expensive than those from load management programs.
6. Conclusions
[Insert Figure 11 here]
Utilities with high marginal costs of supply during system peak hours have relied on
direct load control and dynamic pricing to reduce residential electricity demand during
peak hours. These programs are generally cost-effective, but have a number of drawbacks,
including small enrollments, complicated tariff schedules, and limited appeal to many utility customers.
This paper suggests that behavior-based programs can help utilities to manage
their system peaks. Behavior-based programs can be implemented on a large scale with the ability to reach thousands of residential customers. In addition, they may enable utilities to
reach residential customers who might not be interested in participating in demand
response programs. Behavior program can also produce energy savings that other types of
demand response programs do not.
PPL Electric’s energy reports program resulted in system peak-coincident demand
savings of approximately 0.07 average kW per home or about 6.5 average MW in total. The utility’s average cost of savings was approximately $185/average kW. This was slightly
greater than the average cost per kW for U.S. utility load management programs. An
interesting finding was that a home’s efficiency before treatment affected the amount of 24
peak demand savings achieved. Homes that had participated previously in a PPL efficiency program had lower demand savings during system peak hours, presumably because they had already taken steps to reduce electricity use for air conditioning.
Though PPL Electric’s behavior program was more costly compared to the average
demand response program, the difference in cost per kW of savings was not that large.
With messaging that is more focused on demand savings or timed to coincide with system peak hours, it might be possible for the behavior program to achieve greater demand savings and reach cost parity with direct load control and dynamic pricing programs.
25
References Allcott, H. 2011. Social Norms and Energy Conservation. Journal of Public Economics, 95(2), 1082-1095.
Allcott, H. 2011b. Rethinking Real-Time Pricing. Resource and Energy Economics, 33(2) 820-842.
Allcott, H., T. Rodgers (forthcoming). The Short-Run and Long-Run Effects of Behavioral Interventions: Experimental Evidence from Energy Conservation. American Economic Review.
Ayres, Ian, Sophie Rasman, and Alice Shih, 2009. Evidence from Two Large Field
Experiments that Peer Comparison Feedback Can Reduce Residential Energy Use. NBER Working Paper 15386 (September).
Bertrand, Marianne, Esther Duflo, and Sendhil Mullainathan, 2004. How Much Should We
Trust Difference-in-Differences Estimates? Quarterly Journal of Economics, 119 (1),
249-275.
Costa, Dora and Matthew Kahn, 2010. Energy Conservation Nudges and Environmental Ideology: Evidence from a Randomized Residential Electricity Field Experiment. NBER Working Paper No 15939 (April).
Enernoc Utility Solutions, 2013. Paving the Way for a Richer Mix of Residential Behavior Programs. Prepared by P. Ignelzi, J. Peters, L. Dethman, K. Randazzo, and L. Lutzenhiser. CALMAC Study SCE0334.01
26
Faruqui, Ahmad and Sanem Sergici, 2010. Household Response to Dynamic Pricing of
Electricity: A Survey of 15 Experiments. Journal of Regulatory Economics 38, 193225.
Gilbert, B. and J.S. Graff Zivin, 2013. Dynamic Salience with Intermittent Billing: Evidence from Smart Electricity Meters. National Bureau of Economic Research Working Paper 19510.
Kahn, M. and F. Wolak, 2013. A Field Experiment to Assess the Impact of Information
Provision on Household Electricity Consumption. Prepared for the California Air
Resources Board and the California Environmental Protection Agency. Available at http://arbis.arb.ca.gov/research/rsc/3-8-13/item6dfr08-325.pdf.
Newsham, Guy R. and Brent G. Bowker, 2010. The Effect of Utility Time-Varying Pricing
and Load Control Strategies on Residential Summer Peak Electricity Use: A Review. Energy Policy 38, 3289-3296.
Nolan, J., W Schultz, R. Cialdini, N. Goldstein, V. Griskevicius, 2008. Normative Influence Is Underdetected. Personality and Social Pyschology Bulletin 34, 913-923.
PPL Electric, 2012. First Annual report to the Pennsylvania Public Utility Commission for
the Period June 2011 through May 2012, Program Year 3. Prepared by The Cadmus Group, Inc. November 15, 2012.
Rosenberg, Mitchell, G. Kennedy Agnew, and Kathleen Gaffney, 2013. Causality,
Sustainability, and Scalability – What We Still Do and Do Not Know about the
Impacts of Comparative Feedback Programs. Paper prepared for 2013 International Energy Program Evaluation Conference, Chicago. 27
Todd, Annika et al., 2014. Insights from Smart Meters: The Potential for Peak Hour Savings from Behavior-Based Programs. Lawrence Berkeley National Laboratory Report LBNL-6535E.
Wolak, F., 2006. Residential Customer Response to Real-Time Pricing: The Anaheim Critical-Peak Pricing Experiment. Stanford University working paper.
28
Figure 1. Legacy Group Monthly Average Energy Savings 3
Average kWh per home per day
2 1
Pretreatment
0
-1 -2
Jul-09 Sep-09 Nov-09 Jan-10 Mar-10 May-10 Jul-10 Sep-10 Nov-10 Jan-11 Mar-11 May-11 Jul-11 Sep-11 Nov-11 Jan-12 Mar-12 May-12 Jul-12 Sep-12 Nov-12 Jan-13 Mar-13 May-13
-3
Source: Figure shows average daily electricity savings for LG homes. Savings were estimated using difference-in-difference regression of average daily electricity use. The regression included monthby-year fixed effects and customer fixed effects. 95% confidence intervals for savings estimated using Huber-White standard errors clustered on homes.
Figure 2. Expansion Group Monthly Average Energy Savings
3
Pre-
Average kWh per home per day
2 1 0
-1
May-13
Mar-13
Jan-13
Nov-12
Sep-12
Jul-12
May-12
Mar-12
Jan-12
Nov-11
Sep-11
Jul-11
May-11
Mar-11
Jan-11
Nov-10
Sep-10
-3
Jul-10
-2
Source: Figure shows average daily electricity savings for EG homes. See Figure 1.
29
Figure 3. Legacy Group Distribution of Pre-Treatment Average Daily Consumption (kWh), June 2009September 2009
Figure 4. Expansion Group Distribution of Pre-Treatment Average Daily Consumption (kWh), June 2010-September 2010
30
18 16 14 12 10 8 6 4 2 0
91
91
91 91 91
92
91 90
90
90
91 90
90
89
89 87
88 87 86
11 12 13 14 15 16 17 18 19 20 21 22 Hour Ending Number of Peak Hours
31
Temperature
Outside temperature
Number of system peak hours
Figure 5. Incidence of Top 100 Hours of PPL Electric System Demand
Figure 6. Legacy: Weekday Hourly Electricity Use of Control Group Homes
Average kW per home
4.0 3.5 3.0 2.5 2.0 1.5 1.0 0.5 0.0
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 Hour ending Days without system peak hours
Days with system peak hours
Source: Analysis of electricity use of LG control homes.
Figure 7. Expansion: Weekday Hourly Electricity Use of Control Group Homes
Average kW per home
4.0 3.5 3.0 2.5 2.0 1.5 1.0 0.5 0.0
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 Hour ending Days without system peak hours
Days with system peak hours
Source: Analysis of electricity use of EG control homes. 32
Figure 8. Average Demand Savings During System Peak and Weekday Non-peak Hours – LG Homes 0.16 Avg. kWh/hour per home
0.14 0.12 0.1 0.08 0.06 0.04 0.02 0 1
-0.02
3
5
7
9
Non-peak hour savings Peak hour savings
11
13
15
Hour ending
17
19
90% CI LB 90% CI LB
21
23
90% CI UB 90% CI UB
Notes: Savings estimates for system peak hours based on panel regression of hourly energy use on indicators for hours between 10:00 a.m and 10:00 p.m. and hour indicators and Treat(1). Savings estimates for non-peak hours based on panel regression of hourly energy use on 24 indicators for hours between 1:00 a.m. and 12:00 p.m., 24 interaction variables between the hour of the day variables and an indicator for the weekend, and 48 interaction variables between Treat(1), and the previous two sets of variable. Standard errors were clustered on homes.
Average kWh/hour per home
Figure 9. Average Demand Savings During System Peak and Weekday Non-peak Hours – EG Homes 0.2 0.15 0.1
0.05 0
-0.05
1
3
5
Non-peak hour savings Peak hour savings
7
9
11
13
15
Hour ending
90% CI LB 90% CI LB
Notes: See Figure 8.
33
17
19
21
90% CI UB 90% CI UB
23
Avg kWh / hour per home
Figure 10. Average Demand Savings of EG Homes During Weekday Hours by Efficiency Program Participation Before Treatment
0.2 0.15 0.1 0.05 0 -0.05 -0.1
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24
Hour ending Efficiency program non-participants
90% CI LB
90% CI UB
Efficiency program participants
90% CI LB
90% CI UB
Cost per kW of savings
Figure 11. Comparison between Behavior-Based and Utility Demand Response Programs Cost of Demand Savings $500 $450 $400 $350 $300 $250 $200 $150 $100 $50 $0
$185
$164 $66
PPL BB Program (2012)
EIA (2012) Residential DR Direct costs + incentives
34
EIA (2012) Residential DR Direct costs only
Table 1. Tests of Balance between Sampled Treatment and Control Homes Control Group Pre-treatment annual adc (kWh) Pre-treatment summer adc (kWh) Energy-efficiency program participation before treatment (%)
51.7 50.3 3.7
Treatment Group Legacy Group 51.9 50.4
N
p value
9,994 9,994
0.610 0.761
3.8 9,994 Expansion Group 62.0 9,996 74.6 9,996
0.881
Pre-treatment annual adc (kWh) 61.9 Pre-treatment summer adc (kWh) 74.5 Energy-efficiency program participation before treatment (%) 25.4 28.4 9,996 Note: adc is average daily consumption in kWh. p value is based on t-test of equality of mean consumption between treatment and control groups.
35
0.869 0.917 0.446
Table 2. Panel Regression Analysis of Hourly Energy Use Legacy Constant Top100(1)
Model 1 2.203*** (0.0123) 1.458**
Model 2
(0.0120) Top100(1)*Treat(1)
(1-Top100(1))*Treat(1)
Hour fixed effects R
2
N homes
Expansion Model 3 Model 4 2.411*** (0.0160) 1.377** (0.0131)
-0.0798**
-0.0798**
-0.0610*
-0.0609*
(0.0289)
(0.0289)
(0.0366)
(0.0366)
-0.0511**
-0.0512**
-0.0396*
-0.0397*
(0.0169)
(0.0169)
(0.0226)
(0.0226)
No
Yes
No
Yes
0.033
0.208
0.022
0.145
9,994
9,994
9,996
9,996
N observations
22,255,467 22,255,467 22,239,390 22,239,390 Notes: ** indicates estimate is statistically significant at the 1% level, * indicates significance at the 10% level. Dependent variable was hourly energy use. Models estimated by OLS. Standard errors are Huber-White clustered on homes.
36
Table 3. Estimated Peak-Coincident Behavior Program Demand Savings Peak demand Program peak savings per 90% CI Lower 90% CI Upper demand 90% CI Lower 90% CI Upper home (avg. Bound Bound savings (avg. Bound Bound kW) MW) Legacy 0.080 0.032 0.127 3.45 1.39 5.50 Expansion 0.061 0.001 0.121 3.02 0.04 6.00 Total 0.070 0.025 0.114 6.46 3.87 9.06 Notes: Demand savings were the average kWh/hour per home in top 100 hours of utility system demand in 2012. Estimates based on results of Model 2 and Model 4 in Table 2. Estimates are for savings at meter and do not account for line losses. Savings estimates were based on panel regression of hourly energy use with standard errors clustered on homes. Program average demand savings equals the per home average demand savings times the average number of program homes during peak hours.
37
Table 4. PPL Electric Behavior Program Demand Savings Costs
Legacy Expansion Average
90% Confidence Interval Lower Bound $106 $111 $125
Cost per kW of savings $169 $221 $185
90% Confidence Interval Upper Bound $419 $18,443 $429
Notes: Cost per kW based on regression estimate of average demand reduction during peak hours. Estimates assume program cost per home per year of $13.50.
38
Figure A1. PPL Electric Service Territory
Source: PPL Electric website.
39
Figure A2. Example Home Energy Report
40
41
