Review of Energy Performance Contracts

in Small and Very Small Public Housing Authorities

U.S. Department of Housing and Urban Development | Office of Policy Development and Research

The contents of this report are the views of the contractor and do not necessarily reflect the views or policies of the U.S. Department of Housing and Urban Development or the U.S. Government.

Review of Energy

Performance Contracts

in Small and Very Small

Public Housing Authorities

Michael E. Canes

Stuart D. Funk

Mai Anh H. Nguyen

Jyothsna Prabhakaran

Rob Hazelton

September 2020

Foreword

We are pleased to present this report on a national evaluation conducted by the U.S.

Department of Housing and Urban Development (HUD) Office of Policy Development and Research. The study assesses the effectiveness and value of HUD’s Energy Performance Contracting (EPC) program administered by the Office of Public and Indian Housing (PIH).

This report provides the most comprehensive evaluation to date about the value and effectiveness of HUD’s EPC program. The research relies on survey response data and administrative data drawn from Utilities Expense Level (UEL) quantities that public housing authorities (PHAs) annually report to HUD as well as the financial and physical indicators used to gauge PHA conditions. The information collected during the research wil assist HUD in more informed decision making on future energy efficiency policies related to the nation’s public and assisted housing stock.

For this reason, the EPC program remains a valuable financing mechanism in HUD’s toolbox. Overall, this evaluation il ustrates that energy conservation efforts within HUD’s public housing stock wil require policies that encourage and motivate PHAs to sustain partnerships with private capital financiers and companies that deliver energy services, whether through EPCs, the Rental Assistance Demonstration, or other pathways.

Through continued and new efforts, more PHAs wil be able to participate in these energy savings activities, resulting in better housing for the communities they serve.

Todd Richardson

General Deputy Assistant Secretary for Policy Development and Research U.S. Department of Housing and Urban Development

Contents

Executive Summary ....................................................................................................... iv

Chapter 1 Introduction and Background ...................................................................... 1

Chapter 2 Method ............................................................................................................ 3

Phase 1 Survey ........................................................................................................... 3

Corrections to the Phase 1 Data ........................................................................... 3

Implications for Survey Response Analysis ........................................................... 3

Phase 2 Survey ........................................................................................................... 3

Followup Interviews ............................................................................................... 6

Updated Analysis of Operating Data ..................................................................... 6

Chapter 3 Survey Analysis and Results ....................................................................... 7

Statistical Weighting .................................................................................................... 7

Statistical Hypothesis Testing—Comparing Group 1 with Group 3 ............................. 8

Development of Ratings and Averages ................................................................. 8

Calculation of Weighted Average Scores .............................................................. 8

Hypothesis Testing .............................................................................................. 10

Within Group Differences—Units Subject to an Energy Performance

Contract versus Units Not Subject to an Energy Performance

Contract ......................................................................................................... 11

Sources of Physical Condition Improvements ........................................................... 13

Why Not Undertake an Energy Performance Contract? ........................................... 13

How Can HUD Improve Its Energy Performance Contract Program? ....................... 15

Chapter 4 Followup Interviews .................................................................................... 16

Introduction ................................................................................................................ 16

Representation by Group .......................................................................................... 16

Results ...................................................................................................................... 16

Interviewee Recommendations ................................................................................. 17

Chapter 5 Updated Analysis of Utility Expense Level Data ...................................... 19

Previous Analysis—Procedures ................................................................................ 19

Original Data Source ................................................................................................. 19

Previous Results ....................................................................................................... 20

Updated Analysis ...................................................................................................... 20

Other Tests ................................................................................................................ 21

i

Chapter 6 Conclusions and Recommendations ......................................................... 22

Conclusions ............................................................................................................... 22

Recommendations .................................................................................................... 24

Appendix A Responses to Survey Questions ............................................................ 25

Appendix B Summary Notes from Survey Followup Telephone Interviews ............ 54

References ..................................................................................................................... 63

Exhibits

Exhibit 2-1. Survey Invitations for Phase 2 ...................................................................... 4

Exhibit 2-2. Responses to Phase 2 Survey ...................................................................... 4

Exhibit 2-3. Survey Response Rates—Phase 1 and Phase 2 Combined ........................ 4

Exhibit 2-4. Total Responses—Phase 1 and Phase 2 ..................................................... 5

Exhibit 2-5. Total Responses—Phase 1 and Phase 2 ..................................................... 5

Exhibit 3-1. Survey Questions Relating to Utility, Financial, and Physical

Condition ............................................................................................................... 8

Exhibit 3-2. Ratings Attached to Answers to Utility-Related Questions ........................... 8

Exhibit 3-3. Weighted Averages, Standard Deviations, and Standard Errors .................. 9

Exhibit 3-4. Group 1 and Group 3 Differences in Average Utility Scores ....................... 10

Exhibit 3-5. Differences between Average Financial and Physical Ratings ................... 11

Exhibit 3-6. Energy Performance Contracts Covering Al Units versus

Some Units in a Public Housing Agency ............................................................. 11

Exhibit 3-7. Public Housing Agency Perceptions of Utility Savings at

Energy Performance Contract versus Non-Energy Performance

Contract Units ...................................................................................................... 12

Exhibit 3-8. Financial Results for Units Covered by Energy Performance

Contracts versus Those Not Covered ................................................................. 12

Exhibit 3-9. Physical Condition at Units Covered by an Energy

Performance Contract versus Those Not Covered .............................................. 13

Exhibit 3-10. How Physical Condition Improvements Were Obtained at

Energy Performance Contract-using Public Housing Agencies .......................... 13

Exhibit 3-11. Reasons Why Smaller Public Housing Agencies Haven’t

Implemented Energy Performance Contracts ..................................................... 14

Exhibit 3-12. Numbers of Public Housing Agencies Using Other Means

to Invest in Energy and Water Efficiency ............................................................. 14

Exhibit 3-13. Suggestions for Improving Energy Performance Contract

Program Adoption ............................................................................................... 15

Exhibit 4-1. Followup Interviews of Smaller Public Housing Agencies

by Group .............................................................................................................. 16

i

Exhibit 5-1. Previous and Updated Rates of Electricity Consumption

Decline for Group 1 and Group 3 Public Housing Agencies ................................ 20

iii

Executive Summary

In a previous study for the U.S. Department of Housing and Urban Development (HUD), the LMI team1 found that HUD’s Energy Performance Contract (EPC) program effectively improves utility performance and financial and physical condition in the nation’s publicly subsidized housing. That project included all sizes of public housing authorities (PHAs). HUD subsequently asked the team to investigate EPC program effectiveness when considering only small (under 500 units) and very smal (under 250 units) PHAs. While such PHAs dominate the total of those in the United States, to date, only a small number have undertaken an EPC.

This study presents our findings, conclusions, and recommendations regarding HUD’s question. We followed three lines of investigation.

1. A sample of 185 small and very smal (henceforth “smaller”) PHAs received an online survey. Using survey data, the team tested whether the PHAs that used EPCs outperformed those who did not with respect to reducing utilities and improving financial and physical condition. The survey matched the one used in the earlier phase but was limited to a sample of smaller PHAs.

2. A subsample of the survey respondents participated in followup telephone interviews. The team interviewed seven of the PHAs in more depth concerning their experiences with EPCs or improving their utility consumption without this program. These seven responses were combined with eight interviews with smaller PHAs from the earlier phase for 15 total interviews.

3. The team used information from HUD’s Utility Expense Level (UEL) database2 to statistically test the efficacy of EPCs in improving smaller PHA utility consumption.

After analyzing the data, the LMI team reached the following conclusions concerning the efficacy of EPCs for smal er PHAs.

1. Survey results offer strong evidence that smaller PHAs that used EPCs achieved greater reductions in energy and water consumption than did smaller PHAs that did not. The PHAs that conducted EPCs achieved statistically significantly greater improvements for three of eight utility types and insignificant but positive improvements for the other utility types. Such differences were accompanied by greater improvements in financial and physical conditions among those PHAs that used EPCs.

2. In cases where EPCs were applied to some units in a PHA but not others, the units affected by the EPCs were more energy- and water-efficient and in better financial and physical shape. These results, however, were not statistically significant.

3. Updated statistical analyses of UEL data, comparing energy consumption for smaller PHAs before and after an EPC, showed a statistically significant reduction. A second test showed that energy consumption decreased by more for 1 The LMI team consists of LMI and Dominion Due Diligence Group.

2 This database includes annual PHA water, electricity, fuel oil, natural gas, and coal consumption and expense data, beginning in July of each year and ending in June of the following year.

iv

PHAs who performed EPCs than for PHAs who did not. Both conclusions hold at a 95 percent level of statistical confidence.

4. Followup interviews indicated that PHAs who undertook one or more EPCs were satisfied with savings in energy and water costs.

5. No PHA indicated that tenant payment of utilities was a barrier to undertaking an EPC. Most said they were aware of HUD subsidy programs, such as the Resident Paid Utility incentive program, and had taken advantage of these.

The EPC program has been effective at reducing energy and water consumption and has resulted in better financial and physical conditions for smaller PHAs. However, the majority of smaller PHAs have not undertaken EPCs. The survey and our followup interviews sought the reasons for this. The following findings relate to this question.

1. Among smaller PHAs who had not undertaken an EPC, the principal reason was that it would not be cost effective. Other reasons included that HUD’s EPC

process is too complicated and that a lack of upfront cash makes it difficult to begin the EPC process.

2. Most smaller PHAs who had not performed an EPC indicated that they had invested in energy and water use efficiency in other ways, principally through the use of capital and operating funds and, in some instances, via grants or subsidies.

3. Some smaller PHAs found that HUD’s EPC review process took longer than necessary. They argued that the review should take no longer than 60–90 days.

4. Some smaller PHAs indicated that they had too few units to make such a program worth the effort or that their staffs were so small that it would be impossible for them to manage one.

5. Some interviewees suggested that HUD could induce more smaller PHAs to undertake an EPC by publicizing successful ventures by other such PHAs. They felt that HUD should identify and communicate best practices for carrying out an EPC to smal er PHAs.

6. Some interviewees asserted that many smaller PHAs have little experience in a venture like an EPC and see the chal enges as daunting. They felt that these PHAs are subject to one or more of the following shortcomings:

a. They lack familiarity with large projects involving a substantial share of their units.

b. They are unable to judge the accuracy of what an energy performance contractor tel s them.

c. They lack familiarity with local permitting requirements for large projects.

d. They have no previous experience with raising capital from outside sources.

e. They lack familiarity with HUD’s EPC review process.

Based on these findings, we recommend the following:

1. Because the EPC program has succeeded for many smaller PHAs, HUD should maintain the program, if not strengthen it.

v

2. HUD can encourage more smaller PHAs to engage in the program in several ways:

a. Providing smaller PHAs an opportunity to apply for an upfront cash stipend to enable them to undertake an EPC.

b. Bringing successful EPCs done by smaller PHAs to the attention of this class of PHA.

c. Detailing best practices in carrying out an EPC.

d. Limiting the EPC application review time to no more than 60–90 days, perhaps with more assistance to smaller PHAs regarding how to complete their part of the process.

e. Instructing smaller PHAs regarding things they should understand to implement a successful EPC:

i. What to look for in contractor estimates of potential savings (potential downsides as well as upsides).

ii. How to manage tenant expectations and disruptions to tenant operations.

iii. How to take advantage of HUD subsidies with which tenants pay part or all of utilities.

3. If HUD follows up with a targeted program for smaller PHAs, it should first try such a program in a limited geographic area, measuring results for effectiveness.

In the earlier phase, we offered one other recommendation concerning information that would be useful to HUD. We repeat that recommendation here:

4. HUD should investigate whether the Rental Assistance Demonstration (RAD)3 or the EPC program produces more energy and water efficiency in publicly subsidized housing. Such information would assist HUD in evaluating where to best direct its efforts to encourage the efficient use of these resources.

3 RAD is a federal housing program administered by HUD that enables assisted rental housing providers to access private capital markets for purposes of development and rehabilitation. The program was initially authorized by Congress in 2012 as a demonstration program and since has expanded from a ceiling of 60,000 units to 455,000 units.

vi

Chapter 1

Introduction and Background

The U.S. Department of Housing and Urban Development (HUD) sought to assess the effectiveness and value of its Energy Performance Contract (EPC) program and encourage more public housing authorities (PHAs) to participate. HUD engaged LMI, an independent contractor, to survey a sample of PHAs and statistically analyze administrative data covering energy usage and other relevant metrics.

LMI’s assessment of the EPC program proceeded in two phases. The first phase compared trends in the utility consumption and the financial and physical condition of PHAs that had implemented EPCs to those that had not. In that phase, LMI examined PHAs of all sizes and geographic areas. That study also sought information on the extent to which EPC

processes and other factors affected the program’s adoption rate (Bower et al., 2020). The second phase of the research focused on smaller PHAs only. 1 This document reports the finding and methods for Phase 2, but for context, it first briefly summarizes the approach taken in the earlier phase.

Phase 1’s principal method divided PHAs into three groups: those who implemented an EPC (Group 1), those who began the EPC process but did not complete it (Group 2), and those who never undertook an EPC (Group 3). The analysis statistically compared Groups 1 and 3.

The comparison found that PHAs who implemented an EPC decreased their utility (electricity, natural gas, and water) consumption and expenditures more rapidly than PHAs who did not. This finding was based on a statistical analysis of HUD administrative data, such as utility expense level (UEL) data, coupled with evidence collected via a survey of several hundred PHAs. In some (but not all) cases, the differences in energy performance were statistically significant. Survey evidence also indicated that PHAs who had implemented an EPC

improved their units’ financial and physical condition more than PHAs who had not.

In Phase 1, some PHAs said they could not use the EPC program because they had insufficient staff or other resources. Other PHAs indicated that HUD’s Rental Assistance Demonstration (RAD) program was a better option for them. This latter view was reflected in survey answers, opinions expressed in a series of 20 telephone interviews of PHAs by members of the LMI team, and an interview series conducted by one team member at a 2018

RAD conference in Washington, D.C.

In Phase 2, the statistical analysis of UEL data has been updated, and the same survey from Phase 1 was sent to a new sample limited to smal er PHAs. 2 LMI also sought to better understand why smaller PHA participation in the EPC program is limited. HUD’s research indicates that only 1.5 percent of very small PHAs and 15.3 percent of small PHAs participate (HUD, 2016).

1 The term “smaller” PHAs refers to a combination of very small (under 250 units) and small (between 250 and 499 units) PHAs.

2 The same survey was used because a new or modified survey would have necessitated a second Paperwork Reduction Act review by the Office of Management and Budget. Such reviews can be time consuming; whatever might have been gained by making small changes to the survey was not worth this delay.

1

HUD has encouraged smaller PHAs to increase their use of EPCs. For example, in 2018, HUD’s Public and Indian Housing (PIH) unit issued a notice aimed at small and medium PHAs about partnering with public utilities on EPCs. 3

Congress also has shown an interest in encouraging smaller PHAs to increase the energy efficiency of their units. The Economic Growth, Regulatory Relief, and Consumer Protection Act of 20184 incentivized small PHAs to conduct EPCs by enabling them to retain gains from reductions in utility costs over a period of up to 20 years.

The rest of this report is organized as follows:

• Chapter 2 explains the survey method and reports Phase 2 response rates by group.

• Chapter 3 reports statistical results obtained from our analysis of survey responses.

• Chapter 4 summarizes findings from our followup interviews, which elaborate on survey findings.

• Chapter 5 reports the results of our statistical investigation of operating data, essentially an update of previous analyses but limited to smaller PHAs only.

• Chapter 6 offers conclusions and recommendations.

• Appendix A provides question-by-question response data from the survey.

• Appendix B provides more extensive information concerning the followup interviews.

3 PIH Notice 18-20, “Partnering with Utilities on Energy Performance Contracts,” November 6, 2018.

4 Public Law 115-174, 115th Congress, May 24, 2018.

2

Chapter 2

Method

Phase 1 Survey

In Phase 1, LMI created a survey to elicit perspectives from PHAs regarding how much, if at all, the implementation of an EPC or some other program had improved their energy and water use efficiency, their financial condition, and the physical condition of their units. Following Office of Management and Budget review under the Paperwork Reduction Act, the survey was distributed to three groups: those who had performed an EPC (Group 1), those who had started, but not finished, an EPC (Group 2), and those who had never implemented an EPC (Group 3).

Groups 1, 2, and 3 were asked questions to compare their perspectives on various metrics, such as the amounts of electricity or water saved. LMI then tested whether statistically significant differences existed between the answers given by Groups 1 and 3.

The smaller PHA responses we received in the Phase 1 survey were separated from the rest and later combined with the responses received in Phase 2 for statistical analysis. This combining is further explained below.

Corrections to the Phase 1 Data

After survey administration, we analyzed the self-reported groups and compared them with the groups identified by HUD in the original population frame. Some respondents, including smaller PHAs, classified themselves differently than the original population frame from HUD. Project timing could explain some of this discrepancy. For example, if a PHA had started but not finished an EPC at the beginning of the project but later completed it, HUD information might have classified the PHA in Group 2 while the PHA would classify itself as Group 1. It is also possible that the PHA undertook some other non-EPC energy efficiency project which it then mistakenly classified.

Implications for Survey Response Analysis

Because we wanted to see whether an EPC made a meaningful difference in energy performance, we needed to make appropriate corrections to the survey data. Concerning the discrepancies between how PHAs classified themselves and how the LMI team did based on HUD data, we decided that the PHA itself would have the best knowledge of whether it considered or completed an EPC. Therefore, we accepted the PHA’s classification and assumed it had answered the appropriate set of survey questions. However, we checked the discrepancies with HUD’s Energy Center and where the center had definite proof that a PHA had completed an EPC even though it reported that it hadn’t we stayed with HUD’s original classification.

Phase 2 Survey

In Phase 2, the same survey was sent to a fresh sample, limited to smaller PHAs, to gain this group’s perspectives on the same questions. For Group 1, there were 107 smaller PHAs in all, 54 of which had responded to the Phase 1 survey. Some of the 53 remaining PHAs had 3

received the earlier survey as well but had not responded. Al 53 were sent the survey, with those who had not responded earlier encouraged to do so this time.

For Group 2, there were only 46 PHAs in the entire population, 18 of which had responded earlier and 3 whose classification changed (for example, they hadn’t completed an EPC when categorized but later had done so), leaving 25 to be surveyed this time. Al were sent the Phase 2 survey.

For Group 3, 39 had responded to the earlier survey. For this phase, another 107, stratified by region, were sent the survey. None of these 107 had been sampled previously.

Exhibit 2-1 summarizes the Phase 2 survey sample. The 53, 25, and 107 from groups 1, 2, and 3 noted in the preceding paragraphs are shown as totals in the exhibit.

Exhibit 2-1. Survey Invitations for Phase 2

Region

Group 1 Group 2 Group 3

Total

Northeast

22

7

39

68

South

13

5

25

43

West/Midwest

18

13

43

74

Total

53

25

107

185

The response rate to the survey in Phase 2 was lower than in Phase 1. Exhibit 2-2

shows the number of respondents by group. Group 1 responded at a high rate, a little over 75

percent. This was consistent with the rate of response obtained in the earlier phase. However, Groups 2 and 3 responded at much lower rates. We anticipated a lower rate of response from Group 2 because those receiving it had not responded before, indicating they were not inclined to do so. However, we expected a higher response rate from Group 3. We sent several reminder emails to the smaller PHAs while the survey was open, but we received few additional responses. Overall, the response rate to our Phase 2 survey was around 38 percent.

Exhibit 2-2. Responses to Phase 2 Survey

Group 1

Group 2

Group 3

Total

Sent survey

53

25

107

185

Responses

40

5

25

70

Response rate

75.5%

20.0%

23.4%

37.8%

Although we would have liked a higher survey response rate in Phase 2, we had always intended to combine the responses from Phase 1 with those of the second phase. The combined response rate over Phases 1 and 2 was around 55 percent. Exhibit 2-3 shows the breakdown by area and group over both phases.

Exhibit 2-3. Survey Response Rates—Phase 1 and Phase 2 Combined Region

Group 1

Group 2

Group 3

Total

South

78%

50%

49%

59%

West/Midwest

82%

43%

47%

60%

4

Region

Group 1

Group 2

Group 3

Total

Northeast

69%

92%

24%

47%

Total

76%

61%

38%

55%

Exhibits 2-4 and 2-5 show the combined total number of respondents in Groups 1, 2, and 3. The combined responses of Groups 1 and 3, 94 and 64 respectively, were used to test whether statistical differences exist in utility and other performance factors between EPC users and non-users. Of the 94, 54 (57 percent) were from Phase 1 and 40 from Phase 2, while of the 64, 39 (61 percent) were from Phase 1 and 25 from Phase 2. The next chapter reports the results of these tests.

Exhibit 2-4. Total Responses—Phase 1 and Phase 2

Group 3

Region

Group 1

Group 2

(Phase 1 + Phase 2)

Total

South

21

6

20

47

West/Midwest

40

6

28

74

Northeast

33

11

16

60

Total

94

23

64

181

Exhibit 2-5. Total Responses—Phase 1 and Phase 2

5

Followup Interviews

In Phase 1, we conducted 20 followup telephone interviews with PHAs of all sizes from around the country to gain deeper insights into their experiences with EPCs or with other methods of improving the energy and water efficiency of their units. For Group 1, the interviews consisted of a series of questions concerning the PHA’s experience with the HUD EPC process, the energy or water-saving projects undertaken, whether the PHA also had completed a RAD project, and what suggestions for improvement to the EPC process they might offer. For Groups 2 and 3, questions were related to why they either had not started an EPC or not completed one they had started, what other energy or water savings projects they had undertaken, and suggestions for improvements to the HUD EPC process.

Of the 20 PHAs in the followup interviews, eight were smaller. The results of those interviews are included in appendix B of this report. In Phase 2, 19 smal er PHAs were contacted for followup interviews. Of these, seven were interviewed. Thus, our description of lessons learned from the followup telephone interviews includes the responses of 15 smaller PHAs.

Updated Analysis of Operating Data

In Phase 1, PHA operating data of various sorts were examined statistically to analyze whether EPCs significantly affected energy and water efficiency and financial and physical condition. Of various data sets examined, only UEL data covering electricity consumption proved useful.

Tests using these data established a statistically significant effect on consumption among smaller PHAs and all PHAs taken together.

Since then, another year’s worth of UEL data has become available, and we conducted a test of EPC effectiveness again with 2018 data added to our original sample. Because our sample was limited to smaller PHAs in the updated analysis, the total number of PHAs in the population was less than before. However, the additional data supplied a larger number of years for each of the smaller PHAs in the sample pool. Chapter 5 describes the procedures and the results.

6

Chapter 3

Survey Analysis and Results

Statistical Weighting

Our approach to testing the efficacy of HUD’s Energy Performance Contract (EPC) program compared performance measures between the EPC and non-EPC groups (Group 1 and Group 3) for summary statistics, such as means, weighted averages, and proportions. In probability-based sample designs, however, the collected data must account for selection probabilities and be adjusted for differences in proportions among the different strata or subgroups. We employed the following steps to weight the survey data collected and calibrate the data to population totals.

1. Base weights: Our sample design was stratified so that the selection probabilities were not equal among the different strata. For example, small southern PHAs in Group 1 had a sampling probability of 1 (since Group 1 PHAs were selected with certainty), whereas small southern PHAs in Group 3 had a selection probability of 0.031. To account for unequal selection probabilities, we calculate base weights by taking the inverse of selection probabilities.

Base weight h = 𝑁𝑁h,

𝑛𝑛h

where Nh is the population count of stratum h, and nh is the sample size in stratum h.

2. Adjust for non-responses: As noted in chapter 2, not all PHAs responded to the survey.

We calculated response rate (RR) adjustment factors by taking the inverse of the response rate for each stratum as shown in the following formula: RR weight h = 𝑛𝑛h,

𝑟𝑟h

where nh is the sample size of stratum h and rh is the number of responses collected in stratum h.

3. Applying both weights together, we arrived at a combined weight for each stratum h using the formula:

Final weight h = base weight h x RR weight h.

This final weight calibrates all responses to known population counts and keeps the response proportions of each stratum the same as in the population.

The weighted data are used for calculating weighted means, weighted standard deviations, and weighted standard errors only when comparing Groups 1 and 3 and performing statistical hypotheses testing. Other reports, frequencies, cross-tabulations, charts, and tables use the raw, unweighted data.

7

Statistical Hypothesis Testing—Comparing Group 1 with

Group 3

Exhibit 3-1 shows the survey questions in the statistical analyses for group comparisons. These inquiries include questions on utility, financial, and physical condition performance.

Exhibit 3-1. Survey Questions Relating to Utility, Financial, and Physical Condition Group 1: EPC

Group 3: Alternative investments

Group 3: No energy investments

Q9–Q11. On average, by how

Q40–Q42. On average, by how much

much did utility consumption

did utility consumption change due to

change due to the EPCs you

the utility conservation investments you

implemented?

made?

Q12–Q14. On average, by how

Q43–Q45. On average, by how much

much did the following utility

did utility expenses change due to the

expenses change due to the

utility conservation investments you

EPCs you implemented?

made?

Q21. Overall, how did the EPCs Q47. How did the utility conservation Q49. Over the past 5 years, would you

you executed affect the financial investments affect the overall financial say that the financial condition of your condition of your PHA?

condition of your PHA?

PHA has shown: (measure of extent of

improvement)?

Q22. How did the EPCs affect the Q48. How did the utility conservation Q50. Over the past 5 years, would you

overal physical condition of the

investments affect the overall physical say that the physical condition of the included properties?

condition of the properties in your PHA? properties within your PHA has shown: (measure of extent of improvement)?

EPC = Energy Performance Contract. PHA = public housing agency.

Development of Ratings and Averages

For each of the questions in exhibit 3-1, we needed to standardize the ratings across both groups for meaningful comparisons. Exhibit 3-2 shows the numeric ratings created for comparisons. If a respondent indicated it experienced both an increase and a decrease of consumption or expenses for any of the utility sources, however, that response was removed from the data set and the test.

Exhibit 3-2. Ratings Attached to Answers to Utility-Related Questions Calculation of Weighted Average Scores

After standardizing the rating scores, we compared the ratings of Group 1 PHAs with Group 3

PHAs using weighted averages for consumption and expense for each utility (electricity, natural gas, fuel oil, and water) and financial and physical conditions. The weighted average ratings were calculated using the following generic formula:

𝑛𝑛

Weighted average = ∑𝑖𝑖=1𝑤𝑤(𝑖𝑖)∗𝑟𝑟𝑟𝑟𝑟𝑟𝑖𝑖𝑛𝑛𝑟𝑟(𝑖𝑖),

∑𝑛𝑛 𝑤𝑤(𝑖𝑖)

𝑖𝑖=1

8

where w(i) = the final weight obtained after the weighting process, rating(i) = the corresponding rating score calculated for the measure, and n = the number of responses.

We il ustrate with an example calculation for the weighted average of electricity consumption for Group 1. The related question for this example is Q9. On average, by how much did electricity consumption change due to the EPC(s) you implemented?

n = number of EPC PHAs that responded to Q9 = 54.

Numerator = sum (weights ÷ electricity consumption rating) for 54 respondents = 156.33.

Denominator = sum of all weights of the 54 respondents = 71.04.

Weighted average = numerator ÷ denominator = 167.05 ÷ 73.72 = 2.20.

Similarly, weighted averages were calculated for all the measures compared across Groups 1 and 3. Exhibit 3-3 shows the detailed averages, standard deviations, and standard errors. Among the weighted averages for utilities, a lower number means a better score; that is, a bigger reduction in utility use or payments. For financial and physical conditions, a higher number indicates a better score—a more significant improvement over time.

Exhibit 3-3. Weighted Averages, Standard Deviations, and Standard Errors Weighted

Std. error

Number of

Raw

Weighted

std.

mean

Utility

Group responses (n)

means

average/mean

deviation

s/sqrt(n)

Electricity

1

54

2.2037

2.2005

1.49152

0.20297

consumption

3

26

2.7308

2.8585

5.46642

1.07205

Electricity expense

1

52

2.2885

2.2694

1.54389

0.21410

3

25

2.6000

2.7112

5.3531

1.07062

Natural gas

1

51

2.1961

2.2057

1.54047

0.21571

consumption*

3

16

2.9375

2.9276

4.97485

1.24371

Natural gas

1

49

2.4286

2.4318

1.6443

0.23490

expense**

3

15

2.9333

2.9167

2.80473

0.72418

Fuel oil

1

9

2.2222

2.1511

1.51064

0.50355

consumption

3

5

2.8000

3.0225

3.99463

1.78645

Fuel oil expense

1

7

2.0000

1.9539

1.65908

0.62707

3

5

3.2000

3.1587

2.66576

1.19216

Water

1

53

1.7547

1.7556

1.26168

0.17331

consumption*

3

19

2.6316

2.5663

5.30301

1.21659

Water expense

1

50

2.0600

2.0403

1.51115

0.21371

3

18

2.6111

2.442

5.74622

1.35440

Financial condition

1

77

3.3766

3.3621

1.15498

0.13162

3

54

3.3519

3.2792

5.42743

0.73858

Physical

1

78

4.0385

4.0317

0.72920

0.08257

condition***

3

54

3.7407

3.7302

4.58577

0.62404

*Statistical significance at the 95-percent confidence level.

**Statistical significance at the 90-percent confidence level.

***Statistical significance at the 95-percent level using raw means.

9

Hypothesis Testing

We tested whether statistically significant differences existed in performance between Groups 1

and 3 for each of the consumption, expense, physical, and financial measures. The null hypothesis in each case was “there is no difference between the averages and means of a given measure between the EPC group and the non-EPC group.” The test, therefore, was whether the null hypothesis was rejected, that is, whether a statistical y significant difference existed between the group averages.

Exhibits 3-4 and 3-5 show the results of the hypothesis testing. The measures with an asterisk (*) indicate that the null hypothesis was rejected at a 95-percent confidence level and **

indicates a significant difference at a 90-percent confidence level, meaning that there was a difference and that the chances that this difference resulted from random factors are no more than 5 and 10 percent, respectively. 1

Exhibit 3-4. Group 1 and Group 3 Differences in Average Utility Scores 2.20

Electricity Consumption

2.86

2.27

Electricity Expense

2.71

2.21

Natural Gas Consumption*

2.93

2.43

Natural Gas Expense**

2.92

Group 1: EPC

Group 3: Non-EPC

2.15

Fuel Oil Consumption

3.02

1.95

Fuel Oil Consumption Expense

3.16

1.76

Water Consumption*

2.57

2.04

Water Expense

2.44

0

1

2

3

4

5

EPC = Energy Performance Contract.

* = statistical significance at the 95-percent confidence level.

** = significance at the 90-percent confidence level.

Exhibit 3-4 indicates that among the eight measures, only two showed statistical y significant differences at the 95-percent level (natural gas consumption and water consumption) and one measure at the 90-percent level (natural gas expense). The other differences between weighted averages did not achieve that level of statistical confidence. The Group 1 respondents, however, scored lower in all other measures, indicating that, on average, smal er PHAs that 1 Results indicating statistical significance at a 95-percent (or 90-percent) confidence level means that we are 95 percent (or 90

percent) confident in a true difference in performance between EPC PHAs and non-EPC PHAs. If a result is not statistically significant at a 95-percent (or 90-percent) level, then the probability that the difference resulted from random factors is greater than 5 percent (or 10 percent).

10

executed an EPC consistently reported lower consumption and expenditure on utilities. This result means that the implementation of an EPC made a positive difference in these utilities.

Exhibit 3-5. Differences between Average Financial and Physical Ratings 4.03

Physical condition***

3.73

Group 1: EPC

3.36

Financial condition

Group 3: Non-EPC

3.28

0

1

2

3

4

5

6

EPC = Energy Performance Contract.

*** = statistical significance at the 95 percent level using raw means.

Exhibit 3-5 shows average scores for financial and physical conditions. For those categories, a higher number represents a better outcome. Although the numbers indicate that the financial condition of Group 1 improved by more than Group 3, the difference is not statistically significant. The difference between weighted averages in physical conditions did not show statistical significance; however, when looking at the raw means from survey data, there is a statistically significant difference between the EPC group and non-EPC group at a 95 percent confidence level. This indicates that although we cannot claim a difference for all the smaller non-EPC PHAs in the population, among the PHAs that responded to this question, we did see a difference that cannot be attributed to random chance. This difference may have arisen from the investments in energy-efficient equipment or using financial returns from the energy investments on other upgrades, or both.

By the metrics of whether PHAs implementing EPCs experienced greater improvements in their energy consumption, payments, or related financial and physical conditions, in every single case, the EPC-users fared better. This assumes that both the EPC-using and non-EPC-using PHAs gave unbiased responses to questions concerning the extent of changes in their energy consumption and expenditures, financial, and physical conditions. We have no reason to question that assumption. Though few measures of differences were statistical y significant, the fact that all measures showed positive differences is a strong indication that EPCs helped PHAs.

Within Group Differences—Units Subject to an Energy

Performance Contract versus Units Not Subject to an Energy

Performance Contract

Some of the PHAs who had completed an EPC did not apply it to all housing units, as shown in

exhibit 3-6.

Exhibit 3-6. Energy Performance Contracts Covering All Units versus Some Units in a Public Housing Agency

EPC covered al housing units in the PHA

EPC covered only some units in the PHA

59

21

EPC = Energy Performance Contract. PHA = public housing agency.

11

The 21 PHAs whose EPCs covered only some of the units were asked whether they discerned a difference in the energy and financial performance of units where the EPC had been applied versus units where it had not. Concerning utility savings, of the 19 who answered the question, 9 thought the utility savings were greater at the EPC units, 7 saw little difference, and 3 saw less (see exhibit 3-7).

Exhibit 3-7. Public Housing Agency Perceptions of Utility Savings at Energy Performance Contract versus Non-Energy Performance Contract Units

EPC = Energy Performance Contract.

Similarly, respondents compared financial performance at the units covered by the EPC

versus those that were not (see exhibit 3-8). Eleven thought the results were better at those units, six about the same, and one not as good.

Exhibit 3-8. Financial Results for Units Covered by Energy Performance Contracts versus Those Not Covered

EPC = Energy Performance Contract.

Respondents were asked about the physical condition of the units covered by the EPC

and those that were not (see exhibit 3-9). Of the 19 who replied to the question, 7 thought the physical condition of the EPC units was better than at the non-EPC, 11 about the same, and only 1 thought it was worse.

12

Exhibit 3-9. Physical Condition at Units Covered by an Energy Performance Contract versus Those Not Covered

EPC = Energy Performance Contract.

PHAs believe that EPCs improved the energy, financial, and physical conditions at those units where EPCs were completed. However, the sample numbers are small, and we do not ascribe statistical significance to these replies.

Sources of Physical Condition Improvements

Group 1 PHAs also were asked why, if physical conditions improved at the units covered by their EPCs (whether all units or just some), this occurred. In almost 30 percent of the cases, units were upgraded not just through the physical changes from the EPC itself, such as newer windows, HVACs, etc., but through money savings that then could be reinvested in upgrading the facilities (see exhibit 3-10).

Exhibit 3-10. How Physical Condition Improvements Were Obtained at Energy Performance Contract-using Public Housing Agencies

The EPC itself improved the units The EPC work with utility savings reinvested in unit upgrades 49

20

EPC = Energy Performance Contract.

Why Not Undertake an Energy Performance Contract?

The evidence cited above indicates that many, if not most, PHAs who undertook EPCs report positive effects, including energy, water, and money savings, and improvements in financial health and physical condition. If that is so, why aren’t smaller PHAs undertaking them? The survey sought answers to that question, asking Group 3 PHAs why they didn’t undertake an EPC. Of this group, 66 responded with one or more answers (see exhibit 3-11).

13

Exhibit 3-11. Reasons Why Smaller Public Housing Agencies Haven’t Implemented Energy Performance Contracts

EPC = Energy Performance Contract.

About one-third of the respondents indicated that an EPC would not be cost effective for their PHA. In our telephone followup interviews, several PHAs also made this point. Other important reasons were that HUD’s EPC process is difficult to negotiate and that PHAs didn’t have the upfront money to begin the process. Only 6 of the 66 (about 9 percent) said that RAD

was a better alternative. This contrasted with our telephone interviews in Phase 1 of the project when several smaller PHAs said they’d switched to RAD. However, that was a small sample; in this phase, 66 respondents had the chance to say RAD is a superior alternative, yet only 6 did.

The survey also asked PHAs whether, instead of an EPC, they had made investments in energy and water efficiency using other funding sources, such as capital and operating funds, utility financing, and state or local grants. Exhibit 3-12 relates PHA responses to this question.

Forty-eight of 60 who answered, or 80 percent, made other efficiency investments. This is consistent with what we were told by the Group 3 PHAs during telephone interviews. Most of them said that, while an EPC did not look attractive, they had made strong efforts to increase the energy and water efficiency of their housing units, usual y over many years. In most cases, the monies came from the capital or operating funds (37 of 64 cited these as their sources of funds), but grants, HUD subsidy programs, and other sources also were cited.

Exhibit 3-12. Numbers of Public Housing Agencies Using Other Means to Invest in Energy and Water Efficiency

EPC = Energy Performance Contract. PHA = public housing agency.

14

How Can HUD Improve Its Energy Performance Contract

Program?

Survey respondents were asked what improvements to the EPC program might spur more PHAs to undertake one. Possible improvements were listed, and respondents could choose more than one answer, so the numbers add to more than the number of PHAs answering the question. However, respondents were encouraged to give other answers if they wished. Exhibit

3-13 shows the relative frequency of the answers received.

The main answer involved money. HUD should offer greater operating subsidies or direct payments to PHAs managing their own EPCs or both. The next most important improvement would be simplifying HUD’s EPC process, fol owed closely by furnishing more technical expertise. Though these seem like pertinent suggestions, some of the PHAs said that the single most important factor in a PHA undertaking an EPC is whether the director is motivated to do so. They felt challenges could be overcome through diligent attention to the process. However, other PHAs said the lack of staff in smaller PHAs makes it difficult to focus on something like an EPC. Both perspectives have merit.

Exhibit 3-13. Suggestions for Improving Energy Performance Contract Program Adoption EPC = Energy Performance Contract. PHA = public housing agency.

15

Chapter 4

Fol owup Interviews

Introduction

To learn more about smaller public housing agencies’ (PHAs’) experiences with the Energy Performance Contract (EPC) program, after survey completion, we contacted 19 smaller PHAs, asking for fol owup interviews. Seven agreed. In addition, we interviewed eight smal er PHAs in Phase 1 and so drew inferences from 15 interviews.

We made several efforts to contact smaller PHAs for interviews. After completing the written survey, we sent a note asking if they would consent to do so. Of this group, five agreed.

Later, we asked HUD to send a note to the 14 who had not responded. This action secured an additional interview. Further, one team member with personal contacts among smaller PHAs asked one or two more to respond, which added another. Though we found it difficult to get this class of PHAs to agree to interviews, we obtained a reasonable sample between Phases 1 and 2. The interviews are summarized in appendix B.

Representation by Group

The eight smaller PHAs from the earlier phase of the project consisted of three from Group 1, two from Group 2, and three from Group 3. In Phase 2, we interviewed four more from Group 1

and three more from Group 3. In all, our interview sample consists of seven from Group 1, three from Group 2, and five from Group 3 (see exhibit 4-1).

Exhibit 4-1. Followup Interviews of Smaller Public Housing Agencies by Group Phase

Group 1

Group 2

Group 3

Phase 1

3

3

2

Phase 2

4

0

3

Total

7

3

5

Results

Each of the 15 PHAs interviewed had a unique story, whether they did or did not implement an EPC, but certain features were common in their accounts.

1. All 15 are investing in energy and water efficiency. In many cases, this occurred through an EPC, but even when an EPC wasn’t used, the PHA or local utility had invested in reducing energy and water use. Some of these efforts were inexpensive, such as installing low-flow showerheads or wrapping insulation around water heaters, but others involved replacing water heating units or windows throughout the PHA.

2. Phase 2 interviewees reported less use of the Rental Assistance Demonstration (RAD) than in Phase 1. In our Phase 1 interviews, many of the smaller PHAs had undertaken RAD projects, often folding their EPCs into that program. These PHAs were more enthusiastic about RAD than the EPC program because it enabled a greater investment 16

scope. The Phase 2 interviews, however, did not indicate the same level of interest in RAD. One of the latter phase interviewees had continued its EPC while adding a RAD

project, but this PHA had no plans to fold in or otherwise end its EPC.

3. EPCs pay off. Most of the smaller PHAs that had undertaken an EPC reported satisfaction with the net financial benefits received through energy and water savings. In some instances, these savings were substantial; for others, the savings at least paid back the borrowed capital. In one case, however, the PHA felt that the EPC program had been oversold and that actual results were disappointing relative to expectations.

4. Payment of utilities by tenants was not a barrier to EPCs. We asked PHAs in Phase 2

about tenant payments. However, Group 1 PHAs said they felt the subsidy programs, such as the Resident Paid Utility incentive, coupled with the project savings, were adequate to make their investments financially attractive.

5. HUD’s EPC review process takes too long. A few PHAs indicated that the EPC review process in HUD took an excessive amount of time. The Energy Center or others in HUD

could have been more constructive in correcting their applications to move them along at a faster rate. Delaying the EPC sometimes leads to changed conditions on the ground, such that the investment no longer looks as attractive as it did when first applying.

6. Scale matters. In undertaking an EPC, very small PHAs sometimes have too few units to make such a project worthwhile and too little staff to manage one. A few said that if they could combine their units with those of other smal PHAs, an EPC might make sense, but they were unsure how best to do this.

Interviewee Recommendations

1. No more than 60–90 days to complete the EPC application process. A few PHAs opined that HUD’s internal decision-making processes regarding an EPC application take too long. They would like to see expedited procedures to shorten the review process and get the EPC started in a timelier manner. Further, some PHAs said that county permitting processes could be time-consuming and wanted HUD to offer more guidance on how to deal with these processes.

2. Publicize the EPC program more and supply smaller PHAs with case studies of successes. Some PHAs felt that HUD’s focus on EPCs has waned over the years, and smaller PHAs might undertake the program with stronger encouragement. Specific examples of financially successful EPCs would increase interest in the program.

3. Help smaller PHAs better understand contractors and financing sources. Several PHAs felt that smaller PHAs aren’t well-versed in dealing with energy service contractors and can benefit from more help from HUD in scrutinizing their assumptions and whether they are using best practices for the EPC. Another point was that smaller PHAs do not normally engage in obtaining outside financing and, hence, need assistance to do this properly.

4. Help with tenant education because tenant behavior is an important component of energy and water savings. Several PHAs pointed out that tenants aren’t always motivated to use energy and water efficiently and that their behavior can offset efforts made at the PHA level to achieve such efficiencies. They felt that HUD could encourage tenants to take energy and water efficiency seriously and educate them on how they can help.

17

5. Recognize that smaller PHAs have limited staff and find it difficult to divert resources to managing an EPC. PHAs who had not done an EPC often cited limited staff as a major reason. They felt that HUD needs to offer more management assistance to enable smal er PHAs to undertake an EPC. Some suggested that if an EPC could be regionalized among multiple smaller PHAs, a person could be employed by HUD to manage the project on behalf of several housing authorities.

18

Chapter 5

Updated Analysis of Utility Expense Level Data

Previous Analysis—Procedures

In the earlier phase of this project, the LMI team analyzed utility expense level (UEL) data for a broad sample of public housing agencies (PHAs) of all sizes to evaluate whether these data supported survey results concerning energy consumption that we were analyzing separately.

We compared average electricity consumption in the years before Energy Performance Contract (EPC) completion with average consumption afterward for Group 1. We compared electricity consumption in the years before 2010 to that in the years afterward for Group 3.

These results gave us two averaged rates of change in electricity consumption, one for Group 1

and another for Group 3. We then compared these averaged rates of change and calculated whether the difference was statistical y significant at a 95-percent level.

In making these calculations and comparisons, we followed a 3-step process. First, we cleansed the data, removing inconsistencies and obvious errors and fil ing in missing data where we could. Next, for each PHA, we calculated the average rates of electricity consumption according to the following formulas.

EPC_pre (all years) = ∑ 𝑈𝑈𝑈𝑈 (𝑜𝑜𝑜𝑜 𝑟𝑟𝑎𝑎𝑎𝑎 𝐸𝐸𝐸𝐸𝑈𝑈 𝐸𝐸𝑃𝑃𝑃𝑃𝑃𝑃 𝑏𝑏𝑏𝑏𝑜𝑜𝑜𝑜𝑟𝑟𝑏𝑏 𝐸𝐸𝐸𝐸𝑈𝑈 𝑦𝑦𝑏𝑏𝑟𝑟𝑟𝑟) 𝑁𝑁𝐸𝐸𝐸𝐸𝑈𝑈−𝑝𝑝𝑟𝑟𝑏𝑏

Non-EPC_pre (all years) = ∑ 𝑈𝑈𝑈𝑈 (𝑜𝑜𝑜𝑜 𝑟𝑟𝑎𝑎𝑎𝑎 𝑛𝑛𝑜𝑜𝑛𝑛−𝐸𝐸𝐸𝐸𝑈𝑈 𝐸𝐸𝑃𝑃𝑃𝑃𝑃𝑃 𝑏𝑏𝑏𝑏𝑜𝑜𝑜𝑜𝑟𝑟𝑏𝑏 2010) 𝑁𝑁𝑁𝑁𝑜𝑜𝑛𝑛−𝐸𝐸𝐸𝐸𝑈𝑈𝑝𝑝𝑟𝑟𝑏𝑏

EPC_post (all years) = ∑ 𝑈𝑈𝑈𝑈 (𝑜𝑜𝑜𝑜 𝑟𝑟𝑎𝑎𝑎𝑎 𝐸𝐸𝐸𝐸𝑈𝑈 𝐸𝐸𝑃𝑃𝑃𝑃𝑃𝑃 𝑟𝑟𝑜𝑜𝑟𝑟𝑏𝑏𝑟𝑟 𝐸𝐸𝐸𝐸𝑈𝑈 𝑦𝑦𝑏𝑏𝑟𝑟𝑟𝑟) 𝑁𝑁𝐸𝐸𝐸𝐸𝑈𝑈−𝑝𝑝𝑜𝑜𝑃𝑃𝑟𝑟

Non-EPC_post (al years) = ∑ 𝑈𝑈𝑈𝑈 (𝑜𝑜𝑜𝑜 𝑟𝑟𝑎𝑎𝑎𝑎 𝑛𝑛𝑜𝑜𝑛𝑛−𝐸𝐸𝐸𝐸𝑈𝑈 𝐸𝐸𝑃𝑃𝑃𝑃𝑃𝑃 𝑟𝑟𝑜𝑜𝑟𝑟𝑏𝑏𝑟𝑟 2010) 𝑁𝑁𝑁𝑁𝑜𝑜𝑛𝑛−𝐸𝐸𝐸𝐸𝑈𝑈𝑝𝑝𝑜𝑜𝑃𝑃𝑟𝑟

Where,

NEPC-pre = number of years in the pre-EPC period;

NNon-EPCpre = number of years before 2010;

NEPC-post = number of years in the post-EPC period;

NNon-EPC-post = number of years after 2010 for non-EPC PHAs; and UC = utility consumption of a PHA.

Third, we removed outliers from the calculations because we believed that they were mostly due to measurement errors, recording errors, etc. Outliers were calculated as average rates of change in electricity consumption for a PHA that were at least 50 percent greater or less than group averages.

Original Data Source

Our tests’ UEL data came from HUD form 52722 and was furnished to us by HUD. UEL data run from July 1 in a year to June 30 of the next year. We received data through June of 2017

with which to make our comparisons.

19

Previous Results

For smal er PHAs and for all sizes of PHAs taken together, a statistically significant difference exists between the rate at which electricity consumption decreased for Group 1 PHAs relative to that of Group 3. For the smaller EPC-using PHAs, the decline in electricity consumption averaged -11.3 percent. In contrast, for the non-EPC PHAs, it averaged -2.8 percent, with the difference between the two rates statistical y significant at the 95-percent level. While electricity consumption dropped for both groups, it dropped significantly more for PHAs who had undertaken an EPC.

Updated Analysis

Because more UEL data became available since the earlier analysis, we tested EPC

effectiveness again with 2018 data added to our original sample. We followed the same three steps outlined above. Since our sample was limited to smaller PHAs in the updated analysis, the total number of PHAs in the population was less than before. However, the additional data furnished a larger number of years for each of the smaller PHAs in the sample pool.

The two sample populations consisted of 19 smaller PHAs who had completed an EPC

and 63 who had not. The results indicate little change from the previous test; that is, adding another year of data had little influence on the overall results. EPC-using smaller PHAs continue to show a significantly greater rate of decline of electricity consumption in the years after an EPC was performed than in the years before. The difference in the averaged rates of consumption was -12.9 percent and is statistically significant at the 95-percent level.

A greater decline rate occurred for non-EPC using smaller PHAs comparing pre-2010 to post-2010 years as well, but the difference in decline rates was significant only at a 90 percent level of statistical confidence.

A second test was whether the rate of decline for EPC users (Group 1) was statistical y significantly greater than that for the non-EPC using PHAs (Group 3). Relative to the earlier phase of the project, the rate of decline for Group 1 PHAs increased somewhat, from -11.5

percent to -12.9 percent, while that for Group 3 PHAs also increased, from -2.8 percent to -3.7

percent. As before, this difference between Group 1 and 3 rates of decline in electricity consumption was statistically significant at a 95 percent level. Exhibit 5-1 summarizes the previous and updated analyses.

Exhibit 5-1. Previous and Updated Rates of Electricity Consumption Decline for Group 1

and Group 3 Public Housing Agencies

Group Group

1

3

(pre- (pre-

and

and

post- post-

EPC) 2010)

(%)

(%)

Previous −11.5 −2.8

Updated −12.9 −3.7

EPC = Energy Performance Contract.

20

Other Tests

In the earlier phase of the project, we tried several approaches using operating data to test whether the implementation of an EPC made a difference to PHA financial or physical condition, including examinations of various metrics that rated the financial or physical condition of PHA housing units and PHA financial data. None of these measures proved useful; many other factors bear on publicly subsidized housing finances and conditions. For that reason, we did not repeat these examinations in Phase 2.

21

Chapter 6

Conclusions and Recommendations

Conclusions

This second phase of our work for HUD focused on the following research question, taken from HUD’s performance work statement (PWS) for the option phase of this project:1

Is the Energy Performance Contract (EPC) program an effective mechanism for very smal - to small-sized public housing agencies (PHAs) (fewer than 250 and 500 units, respectively) to make utility improvements and improve the conditions of their housing authority? 2

To address this question, the LMI team used three methods to acquire data: an online survey sent to smaller PHAs only, fol owup telephone interviews with a selected sample, and examination of administrative data. From our research, the LMI team offers the following findings and conclusions:

1. Survey results offer strong evidence that smaller PHAs who used EPCs saved energy and water at a greater rate than smaller PHAs without EPCs. The results also indicate strengthened relative financial and physical conditions. A few of the differences between these groups of PHAs were statistically significant, and all showed positive effects.

2. In cases where EPCs were applied to some units in a PHA but not others, the units affected by the EPCs were more energy and water efficient and in better financial and physical shape.

3. Updated statistical analysis of utility expense level (UEL) data, comparing consumption for smaller PHAs before and after an EPC, showed a statistically significant reduction. A second test showed that energy consumption decreased by more for PHAs who performed EPCs than for PHAs who did not. Both conclusions hold at a 95 percent level of statistical confidence.

4. Followup interviews indicated that PHAs who undertook one or more EPCs were satisfied with savings in energy and water use and costs. 3

5. No PHA indicated tenant payment of utilities was a barrier to undertaking an EPC. Most said they were aware of HUD subsidy programs, such as the Resident Paid Utility (RPU) incentive program, 4 and had taken advantage of these.

These findings address the research question. For smaller PHAs who have undertaken an EPC, it has been an effective mechanism to make utility improvements and improve their 1 PWS: 17-233-SOL-00048.

2 The work statement does not ask whether there was a distinction between very small and small PHAs, and we did not organize the data to examine this question. However, a brief review of what data we had revealed that very smal PHAs who did an EPC

improved their energy and water efficiency more than did small PHAs who did one. We also found, though, that very small PHAs who did not do an EPC improved their energy and water efficiency more than small PHAs who did not do one. Because the issue of very small vs. small PHAs was beyond scope, we did not investigate why this may have been the case. Also, we caution that the sample sizes were small, and we do not ascribe statistical significance to either of these results.

3 According to data shown in answer to Q. 2 in appendix A, 79 had done only one EPC and 14 had done two or more.

4 The RPU allows a PHA to exclude from its operating fund rental income calculations, for the duration of the EPC, any increased rental income that results from decreased utility allowances achieved through reduced utility consumption.

22

housing authority conditions. However, the great majority of smaller PHAs have not undertaken an EPC. Both the survey and our followup telephone interviews sought reasons for this. The following findings relate to this question.

1. Among smaller PHAs who had not undertaken an EPC, the principal reason was that it would not be cost effective. Other reasons included that HUD’s EPC process is too complicated, and a lack of upfront cash makes it difficult to begin the EPC process.

2. Most smaller PHAs who had not performed an EPC indicated that they had invested in energy and water use efficiency in other ways, principally through capital and operating funds and, in some instances, via grants or subsidies.

3. Some smal er PHAs found that the HUD review process took longer than necessary.

They argued that the review should take no longer than 60–90 days.

4. Some smaller PHAs indicated that they had too few units to make an EPC worth the effort or that their staffs were so small that it would be impossible to manage one.

5. Some interviewees suggested that HUD could induce more smaller PHAs to undertake an EPC by publicizing successful ventures by other such PHAs. They also felt that HUD

should publicize targeted best practices for carrying out an EPC for smal er PHAs.

6. Some interviewees asserted that many smaller PHAs have little experience in a venture like an EPC and see the challenges as daunting. They felt that these PHAs: a. lack familiarity with large projects involving a substantial share of their units at one time,

b. cannot judge the accuracy of what an energy performance contractor tells them, c. lack familiarity with local permitting requirements for large projects, d. have no previous experience with raising capital from outside sources, and e. lack familiarity with HUD’s EPC review process.

By inference, while many smaller PHAs might have a genuine interest in reducing energy and water use and cost, they are unfamiliar with the steps needed to conduct an EPC and find it easier to finance efficiency projects from operating and capital funds.

7. Several PHAs indicated that tenant behavior is an important factor in energy and water use in publicly subsidized housing. They felt that HUD should strongly encourage tenants to use these resources efficiently.

These various findings suggest that while the EPC program delivers positive results for smaller PHAs, many smaller PHAs cannot undertake these efforts due to lack of resources or size. A number of them suggest that a cash advance of some sort would help overcome these barriers. Further, some PHAs that have successfully undertaken an EPC feel that, with further publicity from HUD coupled with positive examples, lists of best practices, etc., additional smaller PHAs could be induced to try the program. These methods may help, but since HUD

already tries to encourage smaller PHAs to undertake EPCs, it is unclear how successful they will be. Stil , we list these suggestions in recommendation 2 with a followup recommendation if HUD considers acting on them. We also include a recommendation from the earlier phase of the project, which remains uninvestigated.

23

Recommendations

1. Because the EPC program succeeds for many smaller PHAs, HUD should maintain the program, if not strengthen it.

2. HUD can encourage more smaller PHAs to engage in the program in several ways: a. Providing smaller PHAs an opportunity to apply for an upfront cash stipend to enable them to undertake an EPC.

b. Bringing successful EPCs done by smaller PHAs to the attention of this class of PHA.

c. Detailing best practices in carrying out an EPC.

d. Limiting the EPC application review time to no more than 60–90 days, perhaps with more assistance to smaller PHAs regarding how to complete their part of the process.

e. Instructing smal er PHAs regarding things they should understand to implement a successful EPC:

i. What to look for in contractor estimates of potential savings (possible downsides as well as upsides).

ii. How to manage tenant expectations and disruptions to tenant operations.

iii. How to take advantage of HUD subsidies where tenants pay part or all of utilities.

3. If HUD follows up with a targeted program for smaller PHAs, it should first try such a program in a limited geographic area, measuring results for effectiveness.

4. HUD should investigate whether RAD or the EPC program produces more energy and water efficiency in publicly subsidized housing. Such information would assist HUD in evaluating where to best direct its efforts to encourage the efficient use of these resources.

24

Appendix A

Responses to Survey Questions

Population and Responses by Region

We categorized regions according to U.S. Census regions: South, West, Midwest, and Northeast. Since the West region had smaller public housing agency (PHA) sub-group counts, we combined the West and Midwest regions into a single category to secure a nationwide representation for this survey. We also compared the three regions to the 10 HUD regions to ensure consistent mapping. Exhibits A-1, A-2, and A-3 show the mapping of the 10 HUD

regions to the U.S. Census regions for each state.

Exhibit A-1. Mapping of HUD Regions to U.S. Census Regions

State name

HUD region

Census region

Connecticut

New England

Northeast

Maine

New England

Northeast

Massachusetts

New England

Northeast

New Hampshire

New England

Northeast

Rhode Island

New England

Northeast

Vermont

New England

Northeast

New Jersey

New York/New Jersey

Northeast

New York

New York/New Jersey

Northeast

Washington, DC

Mid Atlantic

South

Delaware

Mid Atlantic

South

Maryland

Mid Atlantic

South

Pennsylvania

Mid Atlantic

Northeast

Virginia

Mid Atlantic

South

West Virginia

Mid Atlantic

South

Puerto Rico

Southeast Caribbean

South

Virgin Islands

Southeast Caribbean

South

Alabama

Southeast Caribbean

South

Florida

Southeast Caribbean

South

Georgia

Southeast Caribbean

South

Kentucky

Southeast Caribbean

South

Mississippi

Southeast Caribbean

South

North Carolina

Southeast Caribbean

South

South Carolina

Southeast Caribbean

South

Tennessee

Southeast Caribbean

South

Illinois

Midwest

West/Midwest

Indiana

Midwest

West/Midwest

Michigan

Midwest

West/Midwest

25

State name

HUD region

Census region

Minnesota

Midwest

West/Midwest

Ohio

Midwest

West/Midwest

Wisconsin

Midwest

West/Midwest

Arkansas

Southwest

South

Louisiana

Southwest

South

New Mexico

Southwest

West/Midwest

Oklahoma

Southwest

South

Texas

Southwest

South

Iowa

Great Plains

West/Midwest

Kansas

Great Plains

West/Midwest

Missouri

Great Plains

West/Midwest

Nebraska

Great Plains

West/Midwest

Colorado

Rocky Mountain

West/Midwest

Montana

Rocky Mountain

West/Midwest

North Dakota

Rocky Mountain

West/Midwest

South Dakota

Rocky Mountain

West/Midwest

Utah

Rocky Mountain

West/Midwest

Wyoming

Rocky Mountain

West/Midwest

Arizona

Pacific/Hawai

West/Midwest

California

Pacific/Hawai

West/Midwest

Hawaii

Pacific/Hawai

West/Midwest

Nevada

Pacific/Hawai

West/Midwest

Alaska

Northwest/Alaska

West/Midwest

Idaho

Northwest/Alaska

West/Midwest

Oregon

Northwest/Alaska

West/Midwest

Washington

Northwest/Alaska

West/Midwest

26

Exhibit A-2. U.S. Census Regions

Exhibit A-3. HUD Regions

27

Exhibits A-4 and A-5 show the distribution of all small and very small PHAs, and the number of survey responses from each state, respectively.

Exhibit A-4. Population of Public Housing Agencies by State

*The darker shades in the map above represent larger populations of smaller public housing agencies in states.

28

Exhibit A-5. Survey Responses by State

Detailed Survey Questions and Responses

Exhibit A-6’s many charts show the questions from the survey and the responses collected. In all the charts except question 3, value labels reflect the numbers of responses. Percentages of responses are indicated along the axes.

The color codes for the three groups in the bar and pie graphs are as follows:

• Al Group 1 responses = blue

• Al Group 2 responses = orange

• Al Group 3 responses = red

• Questions that were asked of all groups combined = purple.

29

Exhibit A-6. Detailed Survey Questions and Responses

30

31

32

33

EPC = Energy Performance Contract. PHA = public housing agency.

The regional bar graphs represent average consumption change by region and the number of PHAs contributing to that regional average. For example, in the next bar chart, 25

West/Midwest PHAs responded from Group 1 with an average rating of 2.3, while 12 Group 3

PHAs from the same region had an average rating of 3.1. The rating scale is as shown below.

Q9-Q11/Q40-Q42. On average, by how much did utility consumption change due to the EPC(s) you implemented/utility conservation investments you made?

Electricity Consumption

Group 1: EPC

Group 3: No EPC

11 % or more decrease

22

5

6%-1 0% decrease

11

3

0%-5% decrease

13

13

0%-5% increase

6

4

6%-10% increase

0

1

11 % or more increase

2

0

Total

54

26

34

EPC = Energy Performance Contract.

35

EPC = Energy Performance Contract.

36

EPC = Energy Performance Contract.

37

EPC = Energy Performance Contract.

38

EPC = Energy Performance Contract.

39

EPC = Energy Performance Contract.

40

EPC = Energy Performance Contract.

41

EPC = Energy Performance Contract.

42

43

44

The rating scale for the financial and physical condition questions are as shown below: EPC = Energy Performance Contract.

45

EPC = Energy Performance Contract.

46

47

48

49

50

51

52

53

Appendix B

Summary Notes from Survey Fol owup

Telephone Interviews

Group 1: Have Done an Energy Performance Contract

Public Housing Agency 1

• This public housing agency (PHA) conducted an Energy Performance Contract (EPC) which worked out well.

- Savings more than paid for the initial outlays.

- Al of the PHA’s units were included in the project.

- The PHA pays utilities for common areas while tenants pay utilities for their own units.



However, the PHA utilized HUD’s Resident Paid Utility incentive program, and therefore tenant payment of utilities did not impose a barrier.

• The PHA looked at RAD but didn’t think the economics would work out.

• The PHA began the process of doing a second EPC, but the HUD review process was very slow (8 months), and by the time it was completed, conditions on the ground had changed, and the EPC was not undertaken.

• This PHA made the following suggestions:

- Quicker decision making within HUD’s review process, no more than 60–90 days to work through an EPC application and move it along.

- One-on-one training for smaller PHAs, especially on how things work during HUD’s review.

- Incentivize contractors to come in and take a look at the project economics of doing an EPC for a small or very small PHA. Contractors often won’t bother because they are looking for larger-scale projects, but perhaps if HUD offered to cover their costs of responding to a request for proposal or paid a fee upfront to secure a bid, more contractors might be interested.

PHA 2

• This PHA’s EPC only recently got underway, so it is too early to tell what the economic results wil be.

- It is being managed by a non-profit on behalf of the PHA.

- The EPC is very ambitious, covering various energy-saving projects plus some solar energy production.



The EPC wil take many years, in part because it wil take that long to pay back the investment in solar installation.

54

• It took 3 years to get this EPC off the ground. Part of the delay occurred because of local county review procedures, but HUD’s review process also significantly slowed the EPC.

• Tenants pay utilities at the units covered by the EPC, but the PHA was able to take advantage of HUD incentive programs, and therefore, this was not a barrier.

• Part of the EPC involves installing water meters in tenant units. Once those are installed, tenants wil be responsible for paying water utilities, and the PHA expects that water usage wil decline as a result.

• The PHA Director offered the following suggestions to improve HUD’s EPC process:

- Make HUD’s review process more flexible. Too much time was taken, and conditions on the ground can change over a several-year review period.

- Provide small PHAs more guidance on how to negotiate the county permitting process. What needs to be explained to county officials so that an EPC can get off the ground relatively quickly?

- Provide more guidance on where to obtain financing and how to do so. Many small PHAs have never sought outside financing and don’t know how to go about it.

Consider regionalization of very smal PHAs, such as putting them into a single larger unit. A PHA of, say, 25–30 units isn’t likely to find anyone to do an EPC and probably isn’t even considering one, but if several such very small PHAs were combined, the economics of an EPC

might look better to a prospective contractor.

PHA 3

• This PHA was not satisfied with the EPC it had undertaken.

- The director felt that HUD had oversold the program and not made PHAs sufficiently aware of possible downsides.



He thought that HUD’s lengthy review process should have included warnings that contractor assumptions might not work out as projected.



He also felt that the Energy Center was not of much help to him in negotiating the EPC process.

• This PHA is doing a RAD project as well as the EPC.

- Both wil continue because HUD’s subsidies make the EPC program worthwhile on its own, and therefore it wil not be folded into RAD.

- The director had several suggestions for improvements to HUD’s EPC program.



Inform PHAs that contractor assumptions may be optimistic and provide scrutiny of such assumptions during the review process.



Take a holistic approach to how energy performance can be improved in buildings, not just through straightforward energy conservation measures.



Recognize that tenant behavior is a factor in energy use, and encourage, if not require, tenants not to engage in wasteful energy practices.

55

PHA 4 1

• The PHA is doing both an EPC and a RAD project. The EPC project covers 15 years and has some years yet to run.

- A wide variety of energy and water savings projects have been undertaken over the years. At this point, anything that can be done to save energy or water cost effectively has been or is being done.

- In some of the units, tenants were paying part of the utilities, but the PHA was able to take advantage of HUD subsidy programs to gain the full economic benefits of the energy investments.

• The RAD investments are barely breaking even. But physical upgrades to properties are done for other, non-economic reasons such as tenant comfort and safety and to keep the units in good condition.

• Though this PHA was able to successfully execute an EPC, its director was generally pessimistic that EPCs work well for small or very small PHAs. His experience has been that there is a general lack of interest among smaller PHAs to undertake EPCs. He offered the following suggestions:

- Split the initial development fees with smaller PHAs, say 60–40, where HUD

pays 60 percent of the upfront fees of getting started.

- Conduct demonstration projects for smaller PHAs to see for themselves that these projects can pay off.

- Al ow several smaller PHAs in a given region to combine for a single EPC.

PHA 5

• PHA conducted two EPCs that resulted in enough savings to cover their costs.

- The first EPC was self-managed, and the second was done in conjunction with several other PHAs through an energy service contractor.

- The EPCs have resulted in an improved financial condition of the PHA.

- PHA does not need more EPCs. The only remaining improvements would have very low payback.

• PHA has also done other energy conservation work using programs through its utility company.

- Completed weatherization of units at no direct cost to the PHA and removed that from the EPC project.

• PHA has converted to RAD, which is viewed as a financial and administrative change that does not affect the day-to-day operations.

- The first EPC was paid off before converting to RAD, while the second was bought out through the RAD process. Buying out the EPC went smoothly and did not present any issues.

1 At the time the interview was conducted, PHA 3 indicated it had become a medium-sized PHA.

56

• Recommendations for improving the EPC program:

- Provide training to the tenants. Tenant behavior has a big impact on how successful the EPC can be.

- Education or assistance for the PHA with the procurement process.



When procuring an energy service contractor (ESCo), PHAs need to know how to get the best value for the work being conducted by the contractor.



ESCos may have preferred bidders, and there could be opportunities to save cost by requiring the ESCo to expand the bidder pool.



Opportunities can be found through thorough questioning of the ESCo bidding process and the PHA being well informed. The EPC wil be paid off faster if expenditures are limited from the beginning.



Those administering the EPC at the local level need to question and be aware of exactly what the ESCo is offering and realize they are also trying to make a profit.

- Account for distribution cost in savings calculations since utility distribution costs can increase faster than commodity costs.

- PHAs need to be very dedicated to make sure everything in the EPC process is done as efficiently as possible.

PHA 6

• The EPC process was started in 2005 and was completed through an ESCo.

- Savings were enough to cover the cost of debt with some extra left over.

- Had some issues after the EPC installation with the expected level of continued tenant education.



Education of tenants is important because they drive the level of consumption.



Continuous tenant education was included in the EPC contract but did not meet expectations.

• Believe that the EPC program has been more or less taken over by the RAD

program.

- Unsure if the two could be used together. The financing and details would have to be discussed very early in the project to figure out if they would work together.

• Recommendations for improving the EPC program:

- Tenant education, because tenants need to be aware of what the EPC is doing for them, especially if they pay utilities.



Education on how to conserve energy and be more efficient.



PHA used an education awareness team that employed young people from the community to help educate tenants.

57



PHA included an Energy Fair in the EPC contract to put on an

educational event for the tenants after the EPC installation to help familiarize tenants with new products.

PHA 7

• The EPC was conducted in 2000 as a 12-year contract.

- Believe EPC went smoothly and didn’t feel that being a small PHA presented any additional challenges with completing the EPC.

- Energy and water savings exceeded expectations.

- Had issues with the Energy Information and Performance Center system (online platform) when that was implemented.

- View having an EPC as a potential hindrance when updating and transferring the portfolio to RAD.

• PHA did not extend the existing EPC and would not consider another EPC

because they have converted to RAD.

- Able to make more wholesale improvements under the RAD program.

- Have done modernization under RAD, which has included energy savings.

- Believe that PHAs can choose RAD or an EPC but not both.

• Recommendations for improving the EPC program:

- The interviewee didn’t directly work on obtaining an EPC but believes it could be more streamlined.

- More technical expertise could be provided.



Field office could not provide the assistance needed when the EPC was executed.

- The EPC and RAD programs need to be more coordinated. The programs come out of different offices in HUD, but they are not communicating among themselves.

Group 2: Started but Did Not Complete an Energy

Performance Contract

PHA 1

• Did not finish the EPC; decided to go with RAD. Al changes that would have been included in the EPC were incorporated into the RAD project.

- The EPC would not have been able to address al the changes needed, only small changes.

- RAD provided more economic benefits.

- Utilities are tenant paid, and tenants love the renovations and utility savings.

• Could not provide any recommendations for improving the EPC program.

58

PHA 2

• Did not finish the EPC because it could not address the full needs of the project and was able to convert to RAD to accomplish the tasks planned for in the EPC

along with other infrastructure needs that would not have been covered by the EPC.

- The EPC process was a lot of paperwork and more of a repair than addressing the full needs of the buildings.

- Didn’t see the value of completing an EPC for a small PHA.

• Strong preference for the RAD program.

- Much broader and able to bring housing to a higher standard immediately (eliminated $80 mil ion worth of deferred maintenance).

- Addressed all needs, not just energy.

• Recommendations for improving the EPC program:

- HUD would need to provide a large subsidy for completing an EPC. The RAD

program is a much better option.

PHA 3

• The PHA started the EPC process in 2009/2010, working with an ESCo, and does not know why the EPC was not completed.

• The PHA was slightly likely to try an EPC again, but the RAD program is definitely something it will pursue (starting to look into it).

- EPCs are a challenge for a small PHA with little staff and funding.

• Recommendations for improving the EPC program:

- Need more direct support from HUD, to the point of HUD coming in and executing the EPC. As a new executive director, little was known about the EPC program, and the program was not understood.

Group 3: Never Did an Energy Performance Contract

PHA 1

• This PHA has never done an EPC.

- Contractor energy audit indicated that an EPC would not pay off for the PHA.

• The PHA has, however, invested in several energy and water efficiency projects.

- Low flow showerheads and toilets.

- Replacement of a hot water boiling unit.

- New hot water tank.



Water efficiency investments funded by the local utility company.

• PHA has looked at RAD but has no plans to participate at this time.

59

• Tenant behavior is a factor in energy and water use.

- Tenants conserve normally while staff is on hand during the week.



But on weekends, they run hoses into their units and draw water to fil backyard swimming pools, wash cars, and the like.

• For this PHA to engage in an EPC, it would have to combine with others to achieve scale, and HUD would have to provide a manager or pay for one to undertake such a project.

PHA 2

• The PHA has not utilized the EPC program but has made many investments in energy and water savings over time.

- Partially financed by American Recovery and Reinvestment Act monies.

- Otherwise financed by the PHA’s capital and operating funds.



Includes roof replacement, new windows, new HVACs, smaller water heaters, aerated showerheads and sink faucets, LED lighting in common areas, etc.

• A recent energy audit indicated there are no more short-term savings to be had from further energy or water-related investments.

• Tenants pay part of utilities, but some tenants engage in energy wasteful behavior, keeping temperatures very high in winter and very low in summer.

• The director is aware of the RAD program but has not applied for it.

- Because capital funding is consistently low, RAD may be the only way to maintain the housing units at an acceptable level in the future.

• No recommendation for the EPC program per se, but if HUD could help regulate tenant behavior regarding energy and water usage, that would be helpful.

PHA 3

• This PHA has not done an EPC because the numbers don’t work out, and it would not be cost effective.

- Director was at another PHA where one was looked into, so is familiar with the program.

- RAD also is not economically feasible at this PHA as the anticipated rental payments wouldn’t be sufficient to cover the investment costs.

• However, the PHA has engaged in energy efficiency programs.

- Programmable thermostats.

- Low-flow showerheads.

- Insulation wrapping around water heaters.



Program paid for by the local utility.

- Residents pay for utilities and hence receive direct benefits from energy efficiency investments.

60

• Suggestions to HUD for how to encourage more smaller PHAs to undertake EPCs:

- Provide more publicity to the program—market it more strenuously.

- Cite real-life cases where EPCs have paid off.

- Provide examples of what EPCs have accomplished at smaller PHAs.

- Designate and promote a set of best practices for how smaller PHAs should undertake an EPC.

- Explain how the Resident Paid Utilities and other incentive programs work to make sure PHAs are not deterred by resident payment of utilities.

PHA 4

• The PHA looked into an EPC but decided against it because the Authority had enough available cash from capital funding to implement needed energy-saving measures.

- Installed water meters and low-flow showerheads/aerators, replaced toilets as needed (estimate 80–90 percent low-flow now), capped outside spigots, changed to LED lighting as needed, high-efficiency windows, etc., and units are well insulated.

- Have 198 units and don’t have any additional energy conservation needs that would warrant an EPC.



Lack of return on investment.



Administrative burden too high for small PHA; more effective to implement energy conservation measures on their own than to complete all the paperwork required for an EPC.

• PHA converted to RAD closing in November 2017.

- Capital fund alone was enough to do what has been done for energy but not enough to make all other desired improvements at the PHA.

- Al owed them to get out from under unfunded mandates of public housing.

- After the first year, expect to already have excess funds for other improvements.

• Recommendations for improving the EPC program:

- Al ow for direct payments so that PHA can act as its own contractor and use direct funding to do all the units at once. Would prefer hiring temporary employees or using housing staff to carry out an EPC because of bad experiences working with contractors (for example, using less than spec equipment).

- Administrative burden too high for a small PHA.

• Looking into the EPC program helped the PHA realize the energy improvements they could make and at least opened eyes to what they could do to save energy for them and their tenants.

61

PHA 5

• Did not know about the EPC program or if anyone had looked into it in the past.

- Have completed other energy investments (new heat pumps, insulation, windows, etc.) but haven’t analyzed resulting savings.

- Units are in good condition, have been managed well, and money spent wisely.

• Looked into the RAD program but prefer to keep units as public housing.

• Doesn’t have any problems with the programs coming out of HUD.

62

References

Bower, Anna E., Michael E. Canes, Stuart D. Funk, Jyothsna Prabhakaran, Amy Deora, and Rob Hazelton. 2020. Review of Energy Performance Contracts in Public Housing. Washington, DC: U.S. Department of Housing and Urban Development.

U.S. Department of Housing and Urban Development (HUD). 2016. Energy Performance Contracting in HUD’s Public Housing Stock: A Brief Review. Washington, D.C.: HUD, Office of Policy Development and Research, Affordable Housing Research and Technology Division.

63

U.S. Department of Housing and Urban Development Office of Policy Development and Research

Washington, DC 20410-6000

September 2021

Document Outline

- Review of Energy Performance Contracts in Small and Very Small Public Housing Authorities

- Foreword

- Contents

- Exhibits

- Executive Summary

- Chapter 1 Introduction and Background

- Chapter 2 Method

- Phase 1 Survey

- Corrections to the Phase 1 Data

- Implications for Survey Response Analysis

- Phase 2 Survey

- Followup Interviews

- Updated Analysis of Operating Data

- Chapter 3 Survey Analysis and Results

- Statistical Weighting

- Statistical Hypothesis Testing—Comparing Group 1 with Group 3

- Development of Ratings and Averages

- Calculation of Weighted Average Scores

- Hypothesis Testing

- Within Group Differences—Units Subject to an Energy Performance Contract versus Units Not Subject to an Energy Performance Contract

- Sources of Physical Condition Improvements

- Why Not Undertake an Energy Performance Contract?

- How Can HUD Improve Its Energy Performance Contract Program?

- Chapter 4 Followup Interviews

- Introduction

- Representation by Group

- Results

- Interviewee Recommendations

- Chapter 5 Updated Analysis of Utility Expense Level Data

- Previous Analysis—Procedures

- Original Data Source

- Previous Results

- Updated Analysis

- Other Tests

- Chapter 6 Conclusions and Recommendations

- Conclusions

- Recommendations

- Appendix A Responses to Survey Questions

- Population and Responses by Region

- Detailed Survey Questions and Responses

- Appendix B Summary Notes from Survey Followup Telephone Interviews

- Group 1: Have Done an Energy Performance Contract

- Public Housing Agency 1

- PHA 2

- PHA 3

- PHA 5

- PHA 6

- PHA 7

- Group 2: Started but Did Not Complete an Energy Performance Contract

- PHA 1

- PHA 2

- PHA 3

- Group 3: Never Did an Energy Performance Contract

- PHA 1

- PHA 2

- PHA 3

- PHA 4

- PHA 5

- References

Table of Contents

Review of Energy Performance Contracts in Small and Very Small Public Housing Authorities

Foreword

Contents

Exhibits

Executive Summary

Chapter 1 Introduction and Background

Chapter 2 Method

Phase 1 Survey

Corrections to the Phase 1 Data

Implications for Survey Response Analysis

Phase 2 Survey

Followup Interviews

Updated Analysis of Operating Data

Chapter 3 Survey Analysis and Results

Statistical Weighting

Statistical Hypothesis Testing—Comparing Group 1 with Group 3

Development of Ratings and Averages

Calculation of Weighted Average Scores

Hypothesis Testing

Within Group Differences—Units Subject to an Energy Performance Contract versus Units Not Subject to an Energy Performance Contract

Sources of Physical Condition Improvements

Why Not Undertake an Energy Performance Contract?

How Can HUD Improve Its Energy Performance Contract Program?

Chapter 4 Followup Interviews

Introduction

Representation by Group

Results

Interviewee Recommendations

Chapter 5 Updated Analysis of Utility Expense Level Data

Previous Analysis—Procedures

Original Data Source

Previous Results

Updated Analysis

Other Tests

Chapter 6 Conclusions and Recommendations

Conclusions

Recommendations

Appendix A Responses to Survey Questions

Population and Responses by Region

Detailed Survey Questions and Responses

Appendix B Summary Notes from Survey Followup Telephone Interviews

Group 1: Have Done an Energy Performance Contract

Public Housing Agency 1

PHA 2

PHA 3

PHA 5

PHA 6

PHA 7

Group 2: Started but Did Not Complete an Energy Performance Contract

PHA 1

PHA 2

PHA 3

Group 3: Never Did an Energy Performance Contract

PHA 1

PHA 2

PHA 3

PHA 4

PHA 5

References