OPENing Our Home

Our apartment in Cambridge Massachusetts is similar to many in the region—just under 1,000 ft2 of space in a three family home built in the early 20th century. It’s a wonderful building with charm and character, but literally no insulation. It’s been a cold and generally uncomfortable space during much of the winter. Fortunately for us, that changed this past January when we had cellulose insulation blown into the exterior walls through the Mass Save program. We had over $3,300 worth of insulation work done and paid just a fraction of the cost; if your state offers free or discounted energy audits and efficiency work, you should seriously consider it.

I’ve been tracking our energy consumption since we moved in just under 2 years ago, and calculating OPEN (operational energy) efficiency scores each month since our first year. Our average score has been a 63 (out of 100), which is above average but not great considering the steps we have taken to reduce our electricity use—killing vampire loads, using LED bulbs, shutting down our wireless router when not in use, etc. These steps certainly improved our score (and saved us money), but it’s just not possible to have an efficient home—even an apartment—in the northeast without insulation. Energy required for space heating dominates here; it accounts for approximately 68% of the energy used in our apartment, with 22% used for hot water and 12% for electricity (nationally these averages are 41.5%, 17.7% and 40.8% respectively).

apartment_energy_allocation
My wife and I agree that the apartment feels more comfortable since the insulation was blown in, but I’ve been looking forward to seeing the impact the insulation work would have on our OPEN score. I want to be able to quantify the improvement rather than relying solely on our (likely biased) qualitative assessment. After receiving our first utility bills since the work was done and calculating a score, this is what we found:

OPEN_scores_Jan_2017_v2

After just a single winter month, our OPEN score jumped from an average of 63 to a 67! As you can see from the graph, our score over the last year (dark blue line) has been fairly consistent, bouncing between a low of 61 and a high of 64 prior to the insulation. It will be interesting to see how the score progresses over the next few months, but it’s already clear that there is a quantifiable improvement in our home’s energy performance. It’s gratifying to see that the money and time spent on the insulation work is having a definitive impact.

Calculate OPEN scores for your home here: www.resynergysystems.com

Author’s note: The OPEN score and energy consumption graph in this post was updated on March 29th 2017 to correct for a data entry error. Text referring to incorrect scores has been updated as well.

Is Above Average Good Enough For You?

Did you ever receive a grade of “B” in a high school math class? How would have felt if you had? Would you have changed your behavior, maybe studying more (or less)? The answer is likely different for each of us; some might have felt happiness or relief, others disappointment and a determination to work harder. The grade would have affected each of us differently.

Might this be true for home energy scores as well?

Here at Resynergy Systems, we’re not only interested in sharing OPEN operational efficiency ratings with homeowners to help them understand how their homes are performing, we’re also interested in understanding the effect of sharing a rating with a homeowner—does the score itself impact behavior?

We had a theory—based on research from MIT—that sharing an operational rating during a home energy assessment would influence a homeowner’s decision to invest in efficiency upgrades. In order to test this theory, we decided to conduct an online experiment based on self-reported intention.

There were some surprises in our findings, but the results strongly supported our hypothesis that homeowners who received an operational rating would be more likely to invest in energy efficiency.

What did we do?
Our main hypothesis was that homeowners who receive a less than stellar operational score for their home would be more likely to invest in efficiency upgrades (e.g. air sealing or insulation work) than those who do not receive a score. We also thought this effect might be stronger for homeowners who were concerned about their energy consumption for environmental or cost reasons. We wanted to conduct a randomized control trial to confirm or reject our theory.

We discussed our ideas with Trisha Shrum, an environmental economist from Harvard (and co-founder of DearTomorrow) who provided feedback and guidance on structuring the research. She also pointed us to Amazon’s Mechanical Turk, an online marketplace used by academics and businesses for research and virtual work services. “MTurk” is a global network of web-connected workers, who review and can decide to accept Human Intelligence Tasks (HITs)—virtual work requests—for pay. In our case, we paid Turk workers a small amount of money to participate in a survey we constructed for this research.

The survey walked each participant through a hypothetical home energy assessment (i.e. audit), with a real-time randomization process that allowed a subset of participants to receive an OPEN score for their home as part of the assessment. We settled on sharing a conservative, above average score of 64 (out of 100, where 100 indicates most efficient) for the study, which we presented visually:

OPEN_score_homes_sample_v1OPEN rating comparison graphic

Only U.S. residents were allowed to participate, and our goal was 400 responses. We had 401 responses in total, with 16 that were incomplete or failed to meet our data quality standards. This left us with unique and clean responses from 385 participants in 47 different states (if you are curious, we missed Idaho, Nebraska and South Dakota). Just under half (48%) of our participants received the OPEN score during their virtual assessment (all receiving the same score of 64, as shown above). These respondents were also asked a series of questions to help us understand how they felt about the score. The remaining participants (52%) did not receive a score and were not aware that the other participants had received one. All respondents were asked the same set of questions about their willingness to invest in efficiency after the audit, and all were presented with the same efficiency upgrade options with deeply subsidized costs: leave the home as is ($0), air seal the home ($100) or insulate and air seal the home ($500). We then asked a series of questions about energy use attention and concern, along with a handful of demographic questions.

research_flow
OPEN rating research structure

What did we find?
We first looked at the entire population of respondents. As expected, we found that homeowners receiving an OPEN score were more likely to say they would invest in air sealing or insulation work, with a moderate increase of 4.2%. Although we could not confirm our hypothesis as this result was not statistically significant (p=0.433), it was promising.

We then analyzed three subgroups within the overall population based on their responses to our survey questions. These groups represent respondents who indicated they:

  • Pay close attention to utility bills (the “attention” group)
  • Are very concerned about reducing energy use (the “concern” group)
  • Feel utility bills are expensive (the “expensive” group)

We believe these subgroups represent homeowners that are concerned about their energy consumption for economic or environmental reasons. When we investigated the effect of a score on these groups, we found that participants in each were over 9% more likely to say they would invest in efficiency when they received an OPEN score. However, the results were only statistically significant (α level .05) in one group: those that identified as paying close attention to utility bills.

A little analysis revealed that the “attention” group encompasses most of the other two subgroups. Of the 250 respondents falling into the “attention” group, 83% also identified as being very concerned about reducing energy use at home, felt their utility bills were expensive, or both. (This is slightly higher than the 75% of respondents in the “expensive” group that belong to either or both of the other two groups.) We believe it is these homeowner concerns that are driving the attention paid to utility bills.

Turk_Respondents_v5_whole_groups Breakdown of respondents concerned about their energy use

The effect of an OPEN score on the “attention” group was statistically significant and compelling. This group represents 65% of our total sample population, and these homeowners were 11.8% more likely to say they would invest in efficiency upgrades when receiving a score (p=0.046).

group % more likely to invest when OPEN score received
 p-value   size (n)
% in Attention group
Attention 11.8% 0.046 250 100.0%
Expensive 11.0% 0.085 250 72.4%
Concern 9.4% 0.169 126 91.2%

These results provide preliminary evidence that receiving an OPEN score may motivate homeowners to invest in efficiency—especially if they pay close attention to their utility bills (i.e. are concerned about energy consumption) to begin with.

The survey provided other important insights:

  • 72% of respondents wanted to know how their energy use compared to other homes. This suggests that people are both interested, and do not currently know, how their energy use compares to others.
  • 82% of respondents indicated they were completely comfortable (57%) or somewhat comfortable (25%) sharing utility information with an energy auditor. This contradicts the belief that homeowners are concerned about privacy and reluctant to share energy data.
  • 90% of respondents who received a score indicated they would take action to improve their OPEN rating if they received a score of 25 or less. Lower scores likely provide a stronger incentive for homeowners to reduce consumption and invest in energy efficiency.

Does it matter?
These findings demonstrate that simply sharing an efficiency score with a homeowner may encourage investment in energy efficiency. Although our experiment was based on an operational score, it is not unreasonable to think we would see similar results from an asset score if shared in a context meaningful to homeowners. One benefit of an operational score, however, is that it is a reflection of a homeowner’s actual energy use and not simply the result of energy modelling of the physical structure.

These results may be especially significant for residential efficiency programs, as the people most affected by the OPEN score indicated they pay close attention to their bills; these are likely the homeowners requesting energy audits and taking advantage of efficiency program offerings. Efficiency programs should consider targeting this population, as sharing an operational score with this group may have substantial benefits with relatively little cost.

Our findings also point to the idea that an OPEN score does not have the same meaning for all people. Although this needs further study (and data), our research suggests that receiving an OPEN score may not have the same effect on homeowners who are unconcerned about energy consumption. In fact, people who indicated they do not pay close attention to their utility bills and received the OPEN score of 64 were 10.9% less likely to say they would invest in efficiency in this study. These results were not statistically significant (p=0.3), but in the same way that a “B” grade has different significance for individual students, it is possible that less-concerned homeowners view a score of 64 as a positive rather than negative. Again, this needs further study, but these results are interesting and unexpected.

Summary
Our initial research results are promising, and we plan to continue our study by providing a randomized set of homeowners their actual OPEN ratings, based on their own energy use, during home energy assessments. We expect to find that homeowners receiving an unsatisfactory energy rating will be more likely to spend real money on efficiency improvements to their home, and that the determination of unsatisfactory is subjective. The actual score received will strongly influence the likelihood of taking action, but the magnitude of the effect will depend on the mindset of the person receiving it.

The idea that sharing an operational efficiency rating may influence homeowner behavior has implications beyond home energy assessments. Other related research questions we want to answer:

  • Will sharing OPEN ratings as part of outreach efforts motivate homeowners to request home energy assessments?
  • What is the effect of sharing an OPEN rating with a homebuyer or renter as part of a real estate transaction?

More to come.

 

Author’s note: If you have questions, ideas for further research or would like to collaborate with us on research in the field, please contact us.