H U D S E C T I O N 8 1 1 P R A

Project Rental Assistance Program

Phase II Evaluation

Final Report

Implementation and

Short-Term Outcomes

U.S. Department of Housing and Urban Development | Office of Policy Development and Research

Visit PD&R’s website

huduser.gov

to find this report and others sponsored by HUD’s Office of Policy Development and Research (PD&R).

Other services of HUD User, PD&R’s research information service, include listservs, special interest reports, bimonthly publications (best practices, significant studies from other sources), access to public use databases, and a hotline (800-245-2691) for help accessing the information you need.

H U D S E C T I O N 8 1 1 P R A

Project Rental Assistance Program

Phase II Evaluation

Final Report

Implementation and

Short-Term Outcomes

Prepared for

U.S. Department of Housing and Urban Development

Prepared by

Melissa Vandawalker

Ian Breunig

Samuel Dastrup

Sara Galantowicz

Gretchen Locke

Austin Nichols

Abt Associates

February 2020

Acknowledgments

The authors of this report grateful y acknowledge the efforts of many individuals who assisted us as we conducted the Section 811 Project Rental Assistance (PRA) Evaluation.

The authors would like to thank the numerous people from the U.S. Department of Housing

and Urban Development (HUD) and the U.S. Department of Health and Human Services (HHS)

who guided the study team in the evaluation. At HUD, Teresa Souza, the Government Technical

Representative for the evaluation, provided insightful input and guidance throughout the study.

Carol Star, Mark Shroder, and Michael Hol ar at HUD’s Office of Policy Development and Research (PD&R) also provided valuable feedback and insight throughout the evaluation. The authors thank Lynn Rodgers of PD&R for her assistance with acquiring HUD administrative data used in the

quantitative analysis. At HUD’s Office of Multifamily Housing, the study team would like to thank Katina Washington, Marvis Hayward, and Jennifer Larson for providing valuable data, advice,

and information on HUD’s Section 811 program. The authors also thank Martha Egan and John

Sorenson at the Centers for Medicare and Medicaid Services and Emily Rosenoff of HHS’s Office

of the Assistant Secretary for Planning and Evaluation for their assistance and expertise throughout the evaluation.

Many individuals at Abt Associates contributed to the evaluation and this report. Jill Khadduri and Terry Moore, the evaluation’s Project Quality Advisors, provided guidance and feedback throughout all phases of the project. Several staff conducted in-person surveys with residents or interviews with grantees and partnering agency staff: Mol y Brune, Ian Caughlan, Lauren Christopher, Tanya de Souza, Hayley Didrickson, Elizabeth Giardino, Kate Hausdorff, Meaghan Hunt, Olivia Jung, Gabrielle Katz, Caroline Logan, Jennifer Pettis, Louise Rothschild, Kaitlin Sheedy, Wil iam Vil alba, Kel y Weiss, and Katharine Witgert. Alon Evron, Debbie Johnson, Mol y Brune, Gabriel e Katz, Michel e Thompson, Kel y Weiss, Louise Hadden, and Ian Caughlan assisted with the data analyses and data use agreements with agencies that provided data for the study. The team also appreciates Suzanne Erfurth, Bry Pol ack, and Erin Miles who provided editing, production, and graphic design, and

David Howard who provided financial and contracting support.

Thanks are also due to leadership and staff of Section 811 PRA grantees and partnering agencies who participated in interviews with the study team and provided data and information on their

programs to the study team. Property management staff at Section 811 PRA and PRAC properties

also helped the study team with data requests related to recruiting tenants for interviews. Final y, the study team gives special thanks to the Section 811 residents who participated in surveys and shared their invaluable experiences and insights with us.

Disclaimer

The contents of this report are the views of the contractor and do not necessarily reflect the views or policies of the U.S. Department of Housing and Urban Development or the U.S. Government.

H U D S e c t i o n 811 P R A P ro g r a m – P h a s e I I E v a l u a t i o n

Contents

Executive Summary .................................................................................................................................xi

1. About the PRA Program Evaluation and This Report ....................................................................... 1

1.1 Section 811 PRA Grant Requirements ...................................................................................................................... 1

1.2 National Status of PRA Program Implementation .................................................................................................. 2

Status of PRA Grants as of September 2018 ..........................................................................................................................2

1.3 The Evaluation of the Section 811 PRA Program .................................................................................................... 2

Phase I Evaluation (2014-2016) ................................................................................................................................................4

Phase II Evaluation (2016-2019) ...............................................................................................................................................4

1.4 The Phase II Evaluation Design ................................................................................................................................. 4

Phase II Evaluation Objectives, Research Questions, and Data Sources ..............................................................................4

The Evaluation’s Comparison Groups .....................................................................................................................................5

Limitations of the Evaluation’s Design .....................................................................................................................................6

1.5 Structure of This Report ............................................................................................................................................. 7

2. PRA Programs in the Six Study States ............................................................................................. 9

2.1 PRA Grants in the Study States ................................................................................................................................. 9

2.2 State PRA Target Populations ................................................................................................................................... 9

2.3 PRA Program Profiles in the Six Study States ...................................................................................................... 10

California ................................................................................................................................................................................ 10

Delaware ................................................................................................................................................................................. 11

Louisiana ................................................................................................................................................................................ 11

Maryland ................................................................................................................................................................................ 11

Minnesota ............................................................................................................................................................................... 12

Washington ............................................................................................................................................................................ 12

2.4 PRA Program Status in the Study States ............................................................................................................... 12

Status of Units under Contract and under Lease ................................................................................................................. 12

PRA Residents Assisted During 2015-2018 .......................................................................................................................... 13

Duration of PRA Resident Tenancy ....................................................................................................................................... 14

Chapters 3-7: Early Evidence from the PRA Program .......................................................................... 15

3. Demographic Characteristics of PRA Residents and Comparison Groups ................................... 17

3.1 Household and Demographic Characteristics ...................................................................................................... 17

Household Size ...................................................................................................................................................................... 17

Age ......................................................................................................................................................................................... 18

Race, Ethnicity, and Gender .................................................................................................................................................. 18

Household Income ................................................................................................................................................................ 19

4. Neighborhood Characteristics for PRA Residents and the Comparison Groups .......................... 21

4.1 Approach to Comparison of Neighborhood Characteristics ............................................................................... 21

4.2 Neighborhood Demographic and Household Characteristics ............................................................................22

Residential Density and Urban/Rural Classification .............................................................................................................22

Percentage of Residents Who Report a Disability ................................................................................................................23

Education and Income ...........................................................................................................................................................23

Racial y and Ethnical y Concentrated Areas of Poverty Neighborhoods .............................................................................23

Neighborhood Livability Indicators ........................................................................................................................................24

4.3 Access to Services and Transportation .................................................................................................................25

4.4 Resident Perception of Neighborhood ...................................................................................................................27

Service Provider Perceptions of Neighborhoods Where PRA Units Are Available ..............................................................28

iii

H U D S e c t i o n 811 P R A P ro g r a m – P h a s e I I E v a l u a t i o n

Contents

5. PRA and PRAC Property Characteristics ....................................................................................... 29

5.1 Section 811 Property Characteristics .....................................................................................................................29

Building Type ..........................................................................................................................................................................30

Property Size ..........................................................................................................................................................................30

Age and Development Status of PRA Units under Contract ................................................................................................31

For-Profit Status of Owner .....................................................................................................................................................31

5.2 Residents’ Satisfaction with Their Housing ..........................................................................................................32

5.3 Physical Quality of PRA Units ..................................................................................................................................32

PRA Resident Views on the Physical Quality of Their Housing ............................................................................................33

Maintenance Issues Since Resident Moved into Unit ...........................................................................................................33

Program Partner Perceptions of the Quality of PRA Units ...................................................................................................34

5.4 Unit Accessibility Features and Needs of Residents............................................................................................34

Reported Accessibility Needs of PRA Residents .................................................................................................................35

5.5 Integration of PRA Units within Properties............................................................................................................35

Resident Perspectives on Community Integration and Independence ................................................................................36

6. PRA and PRAC Community-Based Services .................................................................................. 39

6.1 Availability and Use of Community-Based Services ............................................................................................39

Housing Locator Services ..................................................................................................................................................... 41

Transition Services .................................................................................................................................................................42

Ongoing Tenancy Supports ...................................................................................................................................................42

6.2 Resident Experience with Community-based Services and Alignment with Needs .......................................43

Resident Experience with Community-based Services .......................................................................................................43

Quality of Life and Health Status ...........................................................................................................................................43

6.3 Housing Tenancy and Program Exits ......................................................................................................................44

7. Healthcare Conditions and Utilization ............................................................................................ 47

7.1 Healthcare Conditions and Prior Healthcare Utilization ......................................................................................48

Chronic and Potential y Disabling Conditions .......................................................................................................................48

7.2 PRA Impacts on Healthcare Utilization ..................................................................................................................49

Prior Healthcare Utilization of PRA Residents and the Comparison Groups ....................................................................... 51

Early PRA Tenancy Findings .................................................................................................................................................52

7.3 Comparison of Healthcare Utilization for PRA Residents and Each Comparison Group ................................52

PRA and PRAC Comparison .................................................................................................................................................53

PRA and NED Comparison ....................................................................................................................................................53

PRA and Other HUD Comparison .........................................................................................................................................54

PRA and Non-HUD Comparison ..........................................................................................................................................56

Summary ................................................................................................................................................................................56

8. Economic Study of the PRA Program ............................................................................................. 57

8.1 Analyzing Costs and Cost-Effectiveness of PRA and Comparison Programs .................................................. 57

Approach to Cost and Cost Al ocation Analyses of PRA and Comparison Programs ........................................................58

Limitations of the Economic Study ........................................................................................................................................59

8.2 Costs of Rental Subsidies ........................................................................................................................................59

Average Monthly Per-Unit Rental Subsidies in the PRA Program ........................................................................................60

Comparison of Actual and Estimated Average Rental Subsidies ........................................................................................60

Comparison of PRA Contract Rent to Fair Market Rent .......................................................................................................61

Comparison of PRA, PRAC, NED, and Other HUD Monthly Per-Unit Rental Subsidies ......................................................61

8.3 Capital Subsidy Costs ...............................................................................................................................................62

Sources of Capital Financing for PRA and PRAC Properties ...............................................................................................62

Comparison of Combined Capital and Rent Subsidies in the PRA and PRAC Programs ...................................................63

8.4 Costs of Healthcare and Disability-related Services Utilization ........................................................................64

Sources of Assistance for Healthcare and Disability Services for PRA Residents .............................................................65

Costs of Relevant Healthcare and Disability-Related Services Assistance through Medicaid ...........................................66

iv

H U D S e c t i o n 811 P R A P ro g r a m – P h a s e I I E v a l u a t i o n

Contents

8.5 Costs of Program Administration ...........................................................................................................................67

PRA Program Staffing in State Housing and Health Agencies .............................................................................................67

Estimated Annual PRA Program Administrative Costs .........................................................................................................69

Sources of Funding for PRA Program Administration...........................................................................................................70

Costs of HUD’s Administration of the PRA Program and Administrative Costs in Comparison Programs.........................71

8.6 Total Estimated Costs and Cost Effectiveness ..................................................................................................... 72

Summary of PRA Program and Comparison Group Costs ..................................................................................................72

Cost-Effectiveness of the PRA Program ............................................................................................................................... 74

9. PRA Program Practices that May Lead to Successful Results ...................................................... 77

9.1 Strategies for Selecting and Prioritizing Target Populations ..............................................................................77

Results of Selection and Prioritization of PRA Target Populations.......................................................................................77

9.2 Removing Barriers to Applicant Eligibility for PRA ...............................................................................................79

Applicant Eligibility Rates by State and Over Time ...............................................................................................................79

Applicants Found Ineligible for the PRA Program .................................................................................................................80

Applicants Found Ineligible for the Properties ......................................................................................................................82

Strategies Used to Remove Barriers to Eligibility .................................................................................................................82

9.3 Identifying and Securing Quality, Cost-Effective PRA Units ..............................................................................83

Percent of Estimated PRA Units Under Contract and Under Lease ...................................................................................84

Study State Adaptations to Secure More PRA Units ............................................................................................................85

9.4 Ensuring PRA Residents Receive Tenancy Supports They Need .......................................................................86

Challenges in Coordinating Tenancy Supports for PRA Residents ......................................................................................86

Strategies for Coordinating Tenancy Supports for PRA Residents ......................................................................................87

9.5 Developing Effective and Sustainable PRA Partnerships....................................................................................88

How State Agencies Define Successful PRA Partnerships ..................................................................................................88

Sustaining PRA Partnerships ................................................................................................................................................89

Expanding Beyond the PRA Program Partnership ...............................................................................................................90

9.6 Summary .................................................................................................................................................................... 91

10. Conclusion on the Overall Effectiveness of the PRA Program ...................................................... 93

10.1 How Effective Have States Been in Meeting the Goals of the PRA Program? ................................................93

Increasing the Supply of Affordable Housing for People with Disabilities ...........................................................................93

Expanding Housing Opportunities for People with Significant Healthcare and Service Needs ..........................................94

Expanding Housing Opportunities in Integrated Settings ....................................................................................................94

Building Institutional Capacity at the State Level ..................................................................................................................95

10.2 Policy Implications and Areas for Future Research ...........................................................................................95

Program Monitoring and Data Sharing ..................................................................................................................................95

Greater Flexibility in Identifying PRA Units That Meet Resident Needs and Preferences ...................................................96

Sustaining and Expanding the PRA Partnerships .................................................................................................................97

Is PRA an Effective Approach? .............................................................................................................................................98

Areas for Future Research .....................................................................................................................................................98

Appendix A: Status of National PRA Program Implementation as of September 2018 .................... 101

A.1 Overview of Section 811 PRA Grant Awards ....................................................................................................... 101

A.2 Status of Units under Agreement for the PRA Program ....................................................................................102

A.3 Status of PRA Grants by State and by Grant Year ............................................................................................. 107

A.4 Characteristics of PRA Units under RAC ...........................................................................................................108

A.5 Status of Applicant Referrals and Lease-Ups .................................................................................................... 113

A.6 Characteristics of Households That Moved into PRA Units ............................................................................. 117

A.7 Exits from PRA Units .............................................................................................................................................. 119

A.8 Housing Choice Vouchers Leveraged .................................................................................................................. 119

Appendix B: Section 811 PRA Phase II Evaluation Methods .............................................................. 121

B.1 Overview of the Section 811 PRA Phase II Evaluation ........................................................................................ 121

B.2 Evaluation Data Sources ........................................................................................................................................122

v

H U D S e c t i o n 811 P R A P ro g r a m – P h a s e I I E v a l u a t i o n

Contents

B.2.1 Evaluation Data Sources ..............................................................................................................................................................122

Integrated Real Estate Management System (iREMS) ................................................................................................ 123

Tenant Rental Assistance Certification System .......................................................................................................... 124

Inventory Management System / PIH Information Center .......................................................................................... 124

Online Property Integrated Information Suite ............................................................................................................. 124

Section 811 PRA Quarterly Reports ............................................................................................................................. 124

B.2.2 Healthcare Data ............................................................................................................................................................................124

CMS Research Data Assistance Center ..................................................................................................................... 124

State Medicaid Data .................................................................................................................................................... 125

B.2.3 Neighborhood Data ......................................................................................................................................................................126

B.2.4 Survey of Section 811 PRA and PRAC Residents .......................................................................................................................126

B.2.5 PRA Program Documents ............................................................................................................................................................127

B.2.6 Interviews with PRA Program Staff ............................................................................................................................................127

B.3 The Analytic Sample for the Impact and Economic Studies .............................................................................128

B.3.1 Impact Study Outcome Measures ...............................................................................................................................................129

B.3.2 Constructing the Analytic Sample ..............................................................................................................................................131

Multi-Stage Process for Constructing the Analytic Sample ........................................................................................ 131

Three Analytic Samples ................................................................................................................................................ 132

Descriptive Demographic Characteristics of the Study Groups ................................................................................. 132

B.3. 3 Impact Analysis ............................................................................................................................................................................132

Adjustments to Hypothesis Tests due to Multiple Comparisons ................................................................................133

B.3.4 Analysis of Program Exits ...........................................................................................................................................................134

B.3. 5 Health Conditions and Healthcare Utilization Measures ...........................................................................................................138

Chronic and Potential y Disabling Conditions ..............................................................................................................138

Healthcare Utilization Outcome Measures...................................................................................................................139

B.3.6 Collecting Medicaid Administrative Data for the Non-HUD Comparison Group ......................................................................140

B. 4 Approach to Economic Study ............................................................................................................................... 143

B.4.1 Cost Structures of HUD Programs Serving Adults with Disabilities .........................................................................................143

B.4.2 Cost Analysis Data Sources .........................................................................................................................................................143

B.4.3 Cost Allocation Analyses .............................................................................................................................................................144

Housing Rental Subsidies ............................................................................................................................................ 145

Housing Capital Subsidies ........................................................................................................................................... 145

Costs of Healthcare and Disability Services Utilization ............................................................................................... 146

Costs of Program Administration ................................................................................................................................. 147

References ........................................................................................................................................... 149

vi

H U D S e c t i o n 811 P R A P ro g r a m – P h a s e I I E v a l u a t i o n

Contents

List of Exhibits

Exhibit 1.1: Evaluation Objectives, Research Questions, and Data Sources ...............................................................................4

Exhibit 1.2: Summary of Program Features for HUD Comparison Programs ..............................................................................6

Exhibit 2.1: PRA Grant Amounts and Planned Units in the Study States, by State and Grant Year ............................................9

Exhibit 2.2: Target Population of Section 811 PRA Programs ...................................................................................................10

Exhibit 2.3: Status of Planned 811 PRA Units in the Study States, September 2018 ...............................................................13

Exhibit 2.4: PRA-Assisted Residents in the Study States by Previous Living Situation, September 2018 ................................13

Exhibit 2.5: PRA Residents Exiting Housing in the Study States, Program Years 2015–2018 ..................................................14

Exhibit 3.1: Household and Bedroom Size of PRA, PRAC, NED, and Other HUD-Assisted Households

in the Study States, March 2018 ..............................................................................................................................17

Exhibit 3.2: Demographic and Socioeconomic Characteristics of PRA, PRAC, NED, and Other HUD-

Assisted Households in the Study States, March 2018 ...........................................................................................18

Exhibit 3.3: Household Income of PRA, PRAC, NED, and Other HUD-Assisted Households in the Study

States, March 2018 ...................................................................................................................................................19

Exhibit 3.4: Sources of Household Income of PRA, PRAC, NED, and Other HUD-Assisted Households

in the Study States, March 2018 ..............................................................................................................................20

Exhibit 4.1: Number of Households, Properties, and Census Tracts in Neighborhood Analysis, by Study

Group in the Study States, March 2018 ...................................................................................................................21

Exhibit 4.2: Neighborhood Characteristics of HUD-Assisted Households in Study Groups, March 2018 ................................22

Exhibit 4.3: Neighborhood Demographic Characteristics of HUD-Assisted Households in Study Groups, March 2018 .........23

Exhibit 4.4: Comparison of Neighborhood Livability Indicators of HUD-Assisted Households in Study

Groups, March 2018 .................................................................................................................................................25

Exhibit 4.5: PRA and PRAC Neighborhood Accessibility Measures ..........................................................................................26

Exhibit 4.6: Section 811 PRA and PRAC Neighborhood Measures ...........................................................................................27

Exhibit 5.1: Section 811 PRA Building Types for Occupied Units in the Study States, September 2018 .................................30

Exhibit 5.2: Size of Properties with PRA Units and PRAC Units in the Study States, September 2018 ....................................31

Exhibit 5.3: Year of Construction or Last Rehabilitation for PRA and PRAC Properties in the Study States,

September 2018 .......................................................................................................................................................31

Exhibit 5.4: Comparison of Section 811 PRA and PRAC Resident Satisfaction with Housing ..................................................32

Exhibit 5.5: Section 811 PRA Resident Report of Property Condition .......................................................................................33

Exhibit 5.6: Unresolved and Resolved Maintenance Issues Reported by PRA Residents ........................................................34

Exhibit 5.7: Reported Accessibility Needs of Section 811 PRA and PRAC Residents ..............................................................35

Exhibit 5.8: Section 811 Community Integration Measures .......................................................................................................37

Exhibit 6.1: The Three Phases of Community-based Services ..................................................................................................40

Exhibit 6.2: Use of Tenancy Supports by PRA and PRAC Residents ........................................................................................41

Exhibit 6.3: Statistically Significant Differences in Reported Health and Quality of Life of Section 811 Residents ...................44

Exhibit 6.4: Reasons Tenants Had Left PRA Housing as of September 2018 ...........................................................................45

Exhibit 6.5: Hazard Ratios for Reasons for Program Exits for PRA and PRAC Residents, 2013–2018 .....................................46

vii

H U D S e c t i o n 811 P R A P ro g r a m – P h a s e I I E v a l u a t i o n

Contents

Exhibit 7.1: Healthcare Utilization Comparison Samples ...........................................................................................................48

Exhibit 7.2: Unweighted Comparisons of Chronic and Potentially Disabling Conditions among PRA

Residents and Individuals in the Comparison Groups Enrolled in Medicaid in 2015 and 2016 ..............................49

Exhibit 7.3: Unweighted Comparisons of Healthcare Utilization, 2015 (prior to moving in for PRA residents) .........................51

Exhibit 7.4: Average Rates of Healthcare Utilization of PRA Residents Relative to PRAC Residents, 2016 .............................53

Exhibit 7.5: Average Rates of Healthcare Utilization by PRA Residents Relative to NED Voucher Holders, 2016 ....................54

Exhibit 7.6: Average Rates of Healthcare Utilization by PRA Residents Relative to Recipients of Other HUD

Assistance, 2016.......................................................................................................................................................55

Exhibit 7.7: Average Rates of Healthcare Utilization for PRA Residents Relative to Medicaid Enrollees Not

Receiving HUD Assistance, 2016 .............................................................................................................................55

Exhibit 8.1: Monthly Rental Subsidy and Contract Rent for Units in PRA Program ..................................................................60

Exhibit 8.2: Monthly Rental Subsidy for PRAC, NED, and Other HUD ......................................................................................61

Exhibit 8.3: Sources of Capital Subsidy Costs for PRA and PRAC Properties ..........................................................................62

Exhibit 8.4: Annualized Per-Unit Capital Subsidy for PRA and PRAC .......................................................................................63

Exhibit 8.5: Annualized Per-Unit Total Housing Subsidy for PRA and PRAC .............................................................................64

Exhibit 8.6: Healthcare and Disability-Related Services Costs for PRA, PRAC, NED, and Other HUD Residents ....................67

Exhibit 8.7: Section 811 PRA Program State Agencies in Six States ........................................................................................67

Exhibit 8.8: State Housing and Health Agency Staff Administering the PRA Program, in Full-Time Equivalents ......................68

Exhibit 8.9: Estimated Annual Grantee and State Agency Partner PRA Program Administrative Costs ...................................70

Exhibit 8.10: Section 811 PRA Grant Administrative Fees Awarded and Received, Program Years 2015-2018.......................71

Exhibit 8.11: Per-Unit Annual Cost Estimates for PRA and Comparison Programs ..................................................................74

Exhibit 9.1: Grantee-Reported Living Situation of PRA Applicants and Residents, September 2018 .......................................78

Exhibit 9.2: Applicants Found Eligible for the PRA Program in the Six Study States, Program Years 2015–2018 ....................80

Exhibit 9.3: Reasons Applicants were Found Ineligible for the PRA Program, Program Years 2015–2018 ...............................81

Exhibit 9.4: Percentage of Applicants That Did Not Meet PRA Program Requirements, Program Years 2015–2018 ..............81

Exhibit 9.5: Reasons Applicant Lease Applications Were Not Accepted, Program Years 2015–2018 ......................................82

Exhibit 9.6: Percentage of PRA Applicants Whose Lease Applications Were Not Accepted by Owners,

Program Years 2015–2018 ........................................................................................................................................83

Exhibit 9.7: Status of Planned Section 811 PRA Units by State, September 2018 ...................................................................84

Exhibit 9.8: Units Identified, Under Contract, and Under Lease for PRA, September 2018 ......................................................85

Exhibit 9.9: Change in Estimated PRA Units as a Result of Average Expected Increases in Contract Rent .............................86

Exhibit A.1: FY12 and FY13 PRA Grant Funding Awards and Planned PRA Units ..................................................................102

Exhibit A.2: RACs Executed with FY12 Grantees through September 2018, by State ............................................................103

Exhibit A.3: Contracts Executed with FY13 Grantees through September 2018, by State .....................................................104

Exhibit A.4: Status of ARACs with FY12 Grantees through September 2018, by State ..........................................................105

Exhibit A.5: Status of ARACs with FY13 Grantees through September 2018, by State .........................................................105

Exhibit A.6: Occupancy Status of Units under RAC as of September 2018, by Grant Year ....................................................106

Exhibit A.7: Implementation Status of FY12 Grantees as of September 2018, by State .........................................................107

Exhibit A.8: Implementation Status of FY13 Grantees as of September 2018, by State .........................................................108

viii

H U D S e c t i o n 811 P R A P ro g r a m – P h a s e I I E v a l u a t i o n

Contents

Exhibit A.9: Section 811 PRA Units under RAC and under Lease as of September 2018, by Bedroom Size .........................109

Exhibit A.10: Section 811 PRA Units under RAC as of September 2018, by Accessibility .....................................................109

Exhibit A.11: Grantee Commitment for Units Rent Levels as of September 2018, by Grant Year and State ..........................110

Exhibit A.12: Contract Rent and Rental Assistance Payment for Units under Lease as of September

2018, by Bedroom Size ..........................................................................................................................................111

Exhibit A.13: Estimated and Actual Rental Assistance Amounts through September 2018, by State ....................................112

Exhibit A.14: Section 811 PRA Grants Target Populations (FY12 Grantees) ...........................................................................113

Exhibit A.15: Section 811 PRA Grants Target Populations (FY13 Grantees) ...........................................................................114

Exhibit A.16: Referral Source of Applicants through September 2018, by Grant Year ............................................................115

Exhibit A.17: Living Situation of Applicants through September 2018, by Grant Year ............................................................116

Exhibit A.18: Living Situation of Applicants on the Waitlist through September 2018, by Grant Year .....................................116

Exhibit A.19: Living Situation of Ineligible Applicants through September 2018, by Grant Year .............................................117

Exhibit A.20: Previous Living Situation of PRA Residents through September 2018, by Grant Year ......................................118

Exhibit A.21: Housing Tenure of Current PRA Residents through September 2018 ................................................................118

Exhibit A.22: Reasons Tenants Left PRA Housing through September 2018 ..........................................................................119

Exhibit A.23: HCVs Leveraged for PRA Properties, by Grant Year and State ..........................................................................120

Exhibit B.2.1: HUD Administrative Data Sources .....................................................................................................................123

Exhibit B.2.2: Surveys Completed by State and Type of Property ...........................................................................................127

Exhibit B.2:3: Completed Administrative Interviews for PRA Evaluation .................................................................................128

Exhibit B.3.1: Categories of Measured Impacts Compared to PRA ........................................................................................129

Exhibit B.3.2: Impact Study Research Matrix ..........................................................................................................................129

Exhibit B.3.3: Percentage of HUD Tenants on September 30, 2017, Enrolled in Medicaid in 2012–2013

or Medicare in 2014–2015 ......................................................................................................................................131

Exhibit B.3.4: Unweighted and Unadjusted Counts of PRA and PRAC Program Exits, 2013–2018 .......................................134

Exhibit B.3.5: Cox Proportional Hazards Regression for Rates of Program Exits for PRA and PRAC

Residents—Any Exit Type .......................................................................................................................................135

Exhibit B.3.6: Hazard Ratios for Reasons for Program Exits for PRA and PRAC Residents, 2013–2018 ...............................135

Exhibit B.3.7: Cox Hazard Ratio for Program Exits Due to Owner Initiated Nonpayment of Rent...........................................136

Exhibit B.3.8: Cox Hazard Ratio for Owner Initiated Reasons Other Than Nonpayment of Rent ............................................136

Exhibit B.3.9: Cox Hazard Ratio for Tenant Initiated Reasons .................................................................................................137

Exhibit B.3.10: Cox Hazard Ratio for Program Exits Due to Death ..........................................................................................137

Exhibit B.3.11: Condensed Categories of Chronic and Potentially Disabling Conditions .......................................................138

Exhibit B.3.12: Measures Describing Healthcare Utilization by Medicaid Enrollees Using Data Collected from States .........139

Exhibit B.3.13: CCS Diagnosis Categories that Meet the Predictive Criteria ..........................................................................141

Exhibit B.4.1: Cost Analysis Data Sources ...............................................................................................................................143

Exhibit B.4.2: Housing Subsidy Analysis Samples ...................................................................................................................144

Exhibit B.4.3: Healthcare Utilization and HCBS Cost Analysis Measures ...............................................................................147

ix

H U D S e c t i o n 811 P R A P ro g r a m – P h a s e I I E v a l u a t i o n

x

H U D S e c t i o n 811 P R A P ro g r a m – P h a s e I I E v a l u a t i o n

Executive Summary

or lower than for other HUD programs but that program

administrative costs are higher.

The Section 811 Project Rental

Assistance Program

The U.S. Department of Housing and Urban Development

(HUD)’s Section 811 Project Rental Assistance (PRA)

Authorized by the Frank Melville Supportive Housing

Program provides rental housing assistance to non-

Investment Act of 2010,1 the PRA program provides

elderly people with disabilities. In this second phase

project-based rental assistance to extremely low

of its evaluation of the PRA program, HUD sought to

income, non-elderly people with disabilities. The

determine the impact of the program on residents’

program responds to the goals of the Supreme Court’s

housing tenancy and use of home and community-based

1999 decision in Olmstead v. L.C.2 to al ow people with

services, characteristics of properties and neighborhoods

disabilities to live in the least restrictive settings possible

where assisted residents live, and residents’ healthcare

that meet their needs and preferences. The PRA program

diagnoses and utilization. In order to assess the program’s

is a joint initiative between HUD and the U.S. Department

effectiveness, the study compared short-term outcomes

of Health and Human Services’ (HHS) Centers for

of the PRA program against outcomes for residents in the

Medicare & Medicaid Services (CMS). The goal of the

Section 811 Capital Advance/Project Rental Assistance

PRA program is to expand access to high-quality,

Contract program (referred to as PRAC in this report),

affordable housing and voluntary, community-based

outcomes for people with disabilities in other HUD rental

services to allow eligible people to live successfully

assistance programs, and outcomes for a group of similar

in the community. To assess the implementation and

people who receive Medicaid but are not assisted by HUD

outcomes of the PRA approach, the Melvil e Act required

programs.

an independent evaluation.

The evaluation found that the PRA program assists people

The PRA program was designed to respond to a number

who are different from people with disabilities in HUD’s

of policy priorities:

other housing assistance programs in their demographic

• To increase the supply of affordable housing for people

characteristics, the types and sizes of properties they live

with disabilities in a cost-effective way while continuing

in, and the characteristics of the neighborhoods where

to serve households with extremely low incomes.

they live. PRA residents have lower incomes, have more

chronic and disabling conditions, and are more likely to

• To provide affordable, community-based housing

have had long-term stays in inpatient settings. Looking at

options for people who might otherwise be, or be

early outcomes for a sample of units in just six states, both

at risk of becoming, homeless or unnecessarily

housing unit and neighborhood quality are lower for PRA

institutionalized. PRA residents must meet HUD

units than for PRAC units. PRA units have greater access

eligibility requirements for age, income, and disability,

to public transportation and are in neighborhoods with

and be eligible for Medicaid-funded or other home and

greater walkability, but PRA residents do not feel as safe in

community-based services (HCBS).

their neighborhoods.

• To offer integrated housing settings where people with

PRA residents receive tenancy supports similar to PRAC

disabilities live in multifamily housing for people both

residents, and healthcare utilization rates are similar for

with and without disabilities.

residents of the two programs. Utilization rates for long-

• To encourage collaborations between state housing

term inpatient care are lower for PRA residents than for the

and health agencies that result in long-term strategies

comparison group that does not receive HUD assistance,

for providing permanent, affordable housing options

and utilization rates for case management services are

for people with disabilities and coordinated access to

higher. Rates of healthcare utilization for PRA residents

services.

do not differ significantly from rates for residents of other

To date, 27 state housing agencies are administering PRA

HUD housing assistance programs.

grant programs and expect to provide rental assistance

Our assessment of the cost-effectiveness of PRA in

for an estimated 6,000 households. The housing agencies

relation to other HUD programs that assist people with

established interagency partnership agreements with

disabilities found that rental subsidy costs are similar

state health agencies that administer community-based

services funded through Medicaid.

1

Frank Melville Supportive Housing Investment Act of 2010 § 42 U.S.C. 8013 (P.L. 111-374).

2

Olmstead v. L.C. (98-536) 527 U.S.581 (1999).

xi

H U D S e c t i o n 811 P R A P ro g r a m – P h a s e I I E v a l u a t i o n

Executive Summary

Evaluating the PRA Program

(PRAC) program, which provides capital grants to

develop housing exclusively for people with disabilities

The PRA program differs from PRAC and other HUD

and project rental assistance for operational costs.

programs that assist similar populations in a number of

Like PRA, PRAC owners must ensure resident access

ways—in the way in which the housing is identified and

to services.

brought into the program, in the type of rental assistance,

in the program’s cost structure, and in whether and

• Receiving assistance through HUD’s Non-Elderly

how coordinated access to services is provided. These

Disabled (NED) voucher program, which provides

differences affect the experience of PRA residents, their

tenant-based rental assistance to non-elderly people

housing location, access to services, and program costs.

with disabilities who may lease units of their choice

that meet HUD’s requirements.

An initial, Phase I Evaluation3 (2014–2016) examined

the early implementation of the PRA program in 12

• Receiving assistance through (other HUD) programs

states, as state housing agency grantees established

available to eligible low-income people with and

agreements with property owners to lease units to PRA

without disabilities; this category includes Housing

residents, determined outreach and eligibility procedures

Choice Vouchers, public housing, and multifamily

to identify eligible applicants, and began moving people

assisted housing.

into housing. Given the complexities of launching the new

• Receiving Medicaid but not living in HUD-assisted

program and that many grantees identified most or al

housing (non-HUD).

of their PRA units in properties under development, few

The Phase II evaluation uses administrative data on

applicants had been housed by the end of the Phase I

individuals’ demographic characteristics and healthcare

evaluation.

utilization patterns, neighborhood characteristics, property

This Phase II Evaluation (2016–2019) assessed

characteristics for the PRA and PRAC programs, and

the ongoing PRA implementation experience as

costs associated with the PRA and other HUD programs.

programs matured and the PRA program’s outcomes

The study team also compares healthcare utilization for

and effectiveness in six states. The selected states—

people in the non-HUD comparison group. Evaluators also

California, Delaware, Louisiana, Maryland, Minnesota,

reviewed program documents, interviewed PRA program

and Washington—were chosen because they had housed

administrators and other program partners, and surveyed

the largest number of PRA residents by 2017 when the

a sample of approximately 400 residents living in PRA and

evaluation’s research design was finalized.

PRAC properties.

The Phase II evaluation was designed to answer these

Key Findings from the

questions:

• How do short-term impacts of the Section 811 PRA

Phase II Evaluation

program compare to outcomes for comparison groups

How PRA Residents Differ from Similar

made up of similar people living in other settings?

Residents Assisted by Other HUD Programs

• What is the relationship between PRA features and

In order to estimate short-term impacts, and to place

strategies and program results?

our findings in context, we assessed the characteristics

• What are the costs of the PRA program, and how do

of PRA residents relative to people served in other HUD

they compare to costs for other HUD programs serving

programs. This analysis uses 2015 Medicaid data within

similar populations?

the six selected states.

To estimate PRA program impacts, the study team

• On average, PRA residents are younger and have lower

constructed four statistical y matched comparison groups

incomes than non-elderly people with disabilities in

comprising people similar to PRA residents based on

other HUD programs.

their demographic characteristics, chronic and disabling

• PRA residents are less likely to live in single-person

conditions, and healthcare utilization patterns prior to PRA

households than PRAC residents, but more likely than

program implementation. The comparison groups are

residents in NED and the other HUD programs.

drawn from non-elderly people with disabilities from the

six study states in the fol owing categories:

• A larger share of PRA residents is African-American,

and a smal er share is non-Hispanic white or Hispanic

• Receiving assistance through HUD’s Section 811

than residents in the comparison groups.

Capital Advance/Project Rental Assistance Contract

3

https://www.huduser.gov/portal/section-811-process-evaluation.html

xii

H U D S e c t i o n 811 P R A P ro g r a m – P h a s e I I E v a l u a t i o n

Executive Summary

• Based on 2015 Medicaid data, the prevalence of

feeling safe where they live, but not to the same extent as

chronic and disabling conditions tended to be higher

PRAC residents do.

for PRA residents than for those in the comparison

Key findings are:

groups.

• Units under contract for PRA (but not necessarily

• Likewise, before being assisted by PRA, PRA residents

occupied by PRA residents yet) are heavily

tended to utilize healthcare services such as inpatient

concentrated in larger, newer properties with more

hospital services, emergency department services,

than 50 units, in either walk-up or elevator buildings.

and medical transportation more often than people in

Most properties with PRA units under contract (85

the comparison groups. They were also more likely to

percent) were built or rehabilitated since 2000. By

have a long-term stay in an institutional setting, such

comparison (and as limited by statute), nearly all PRAC

as a nursing facility, acute care hospital, or inpatient

residents live in smal er properties, general y with

rehabilitation facility, than al comparison groups prior

fewer than 25 units, and with a smal er share of newer

to receiving PRA assistance. This was expected, given

properties (60 percent built since 2000).

that states often target PRA units to people leaving

institutions.4

• On average, PRA units make up 10 percent of total

units in properties with units under contract, wel

How Short-Term Outcomes of the PRA

below the 25 percent cap. While units set aside under

Program Differ from Outcomes of the

other state or local programs count towards the cap,

Study’s Comparison Groups

information on these units was not available to the

These descriptive findings informed the statistical

study team. Units occupied by, but not set aside for,

construction of the four comparison groups comprising

people with disabilities are not included in the cap.

people living in other settings who are similar in

Anecdotal y, we heard that some properties would

demographics and health characteristics to those assisted

exceed the cap if all of these units are included.

in the PRA program in the six study states. Constructing

• Significantly more PRAC residents reported feeling

such comparison groups al ows us to attribute outcomes

safe in their buildings, 92 percent, compared to 77

for PRA residents to the PRA program, rather than to

percent of PRA residents. Slightly higher shares of

differences in the populations served.

PRAC residents (80 percent) report they like where

they live than PRA residents (76 percent), but this

Quality of Properties and Neighborhoods

difference is not statistical y significant.

PRA units must be located in affordable housing

• Significantly fewer PRA residents (70 percent) reported

developments built with other sources of capital funding,

their units are in excel ent or good condition compared

with no more than 25 percent of total units set aside

to PRAC residents (83 percent).

for people with disabilities. The PRA program also has

incentives for grantees to assist more households by

• We measured whether residents feel integrated in

subsidizing rents lower than HUD’s Fair Market Rent

their community by asking whether they know other

(FMR) that is the basis for determining subsidy payments

people in their buildings and in the neighborhood.

in the Housing Choice Voucher (HCV) and some other

PRA residents were significantly less likely than PRAC

HUD programs. These requirements underscore the

residents to report knowing people in their buildings

program’s goals of housing people in mixed population

(81 percent vs 93 percent) or in their neighborhoods

properties, where both those with and without disabilities

(38 percent vs 65 percent).

live, in a cost-effective, person-centered way.

We found a number of statistical y significant differences

We analyzed administrative data on PRA and PRAC

between the neighborhoods where properties with

properties and our survey of a sample of PRA and PRAC

PRA units are located and those where comparison

residents to determine if PRA units meet program goals

group members live. On average, PRA residents live in

and residents’ needs and preferences. We found that

neighborhoods with higher residential density (that is,

PRA residents live in neighborhoods with higher poverty

buildings with 50 or more units) and lower rates of single-

rates and lower levels of education and higher residential

family owner-occupancy than the comparison groups.

densities than similar people in other HUD programs

PRA residents live in neighborhoods with greater access

(PRAC, NED, and other HUD). On average, PRA residents

to public transit and higher rates of “walkability” than the

reported liking their buildings and neighborhoods and

comparison groups, factors that could contribute to their

quality of life and potential y to improved health.

4

This result may also be partly due to inclusion criteria for the sample. A proportion of individuals in the PRAC, NED, and other HUD groups had moved into their residences prior to 2015 so were less likely to have used long-term inpatient care in 2015.

xiii

H U D S e c t i o n 811 P R A P ro g r a m – P h a s e I I E v a l u a t i o n

Executive Summary

• Relative to the comparison groups, census data

Access and Use of Community-Based

indicate PRA residents also live in neighborhoods

Services and Tenancy Supports

where a significantly higher share of non-elderly adults

The PRA program requires residents be eligible for

(age 35 to 64) self-report a disability, lower shares

Medicaid-funded HCBS or similar state plan services to

of all adults have an Associate degree or higher, and

ensure that residents will have the supports they need

more households have incomes below the poverty

to live successful y in the community. Medicaid can

line. Further, PRA residents live in neighborhoods with

fund certain tenancy support services to help Medicaid

statistical y significant higher exposure to harmful

beneficiaries find, apply for, move to, and remain stably

environmental toxins, according to federal data from

housed in community-based housing, although the exact

the Environmental Protection Agency (EPA).

mix of services varies by state. It can also pay for other

• PRA residents are significantly less likely to report

community-based services that ensure beneficiaries’

feeling safe in their neighborhoods (68 percent)

health and wel -being, such as personal care assistance,

compared to PRAC residents (87 percent).

home healthcare, or transportation assistance. These

However, despite some potential chal enges to their

services are intended to support residents’ health status

neighborhood environments, the majority (73 percent)

and successful community living experience. Community-

of PRA residents report they like their neighborhoods.

based services are available under Medicaid waiver

This percentage is less than the share of PRAC

programs, state plan services, and community-funded

residents (84 percent) who express satisfaction

providers. Not all PRA residents are necessarily eligible to

with their neighborhoods, but this difference is not

receive all services available in their communities.

statistical y significant. (The study did not conduct

We surveyed PRA residents in the six study states

surveys of residents in other HUD programs, so their

about their use of and experience with the services they

perception of their neighborhood is unknown.)

receive in their homes and their perceived quality of life

The analyses of property administrative data and of

and health status. We also surveyed similar residents in

resident survey responses are not representative of al

PRAC properties to see whether their experiences

properties and neighborhoods where PRA residents

differed from PRA residents. In PRAC, the nonprofit

will eventual y live. The analyses represent only a subset

sponsors that developed and operate PRAC housing are

of the units and households that will eventual y be

responsible for ensuring residents have the services and

assisted by PRA. The properties represent less than

community supports they need to remain in their homes.

half of the estimated PRA units the six study states plan

Services in both programs are voluntary for residents.

to assist with their PRA programs.5 Likewise, the resident

Results showed:

survey responses represent the experience of a subset

• The majority of PRA and PRAC residents report that

of PRA residents at an early point of their tenancy, and

the tenancy supports and other services they receive

do not reflect the experience of all residents being

meet their needs. Significantly more PRA residents

assisted by PRA at the time or who will be assisted by

reported receiving help with their lease application to

PRA in the future.6

move into their apartment.

In addition, the evaluation’s results only apply to the six

• Overal , both PRA and PRAC residents rated the

states participating in the study and are not representative

quality of their services wel , but some residents in

of the PRA program in all of the states that have PRA

both groups report gaps in services. Notably, among

programs. The states were selected based on the

the one-quarter of each group who reported needing

implementation status of their programs after two years

help with medications, 65 percent of PRA residents

of grant funding, based on the number of PRA units

reported they had gone without medication because

leased in FY17. In many cases, the states that were able

there was no one to help them, compared to 15

to implement their programs more quickly than others had

percent of PRAC residents, a statistical y significant

prior experience with supportive housing programs or had

difference.

previous state-level agency partnerships.

• PRA and PRAC residents report no statistical y

significant differences in healthcare services received,

amount of care provided by friends and family, or

quality of care received from caregivers.

5

The analysis of characteristics of properties of PRA units is limited to 78 properties and with 632 PRA units under contract as of September 2018

and for which we have administrative data.

6

The resident survey was conducted with 194 residents living in PRA units between January and May 2018.

xiv

H U D S e c t i o n 811 P R A P ro g r a m – P h a s e I I E v a l u a t i o n

Executive Summary

• Most PRA and PRAC residents rate their quality of life

post-acute care services, hospice care, and prescription

and overall health as at least okay, but significantly

drugs among dual-enrol ed individuals. Medicaid only

more PRAC residents rate their quality of life and

pays for specific services not covered by Medicare and

overall health as good or excel ent than PRA residents.

sometimes covers the cost of premiums, deductibles,

• PRA and PRAC residents have similar rates of exits

co-pays, or co-insurance (benefits vary across states).

(about 20 percent a year), but PRA residents are more

We cannot be certain that we captured services that

likely to leave for non-payment of rent than PRAC

were entirely paid by Medicare. Thus, it is likely we have

residents do.

underestimated healthcare utilization by PRA residents

and the comparison groups. Moreover, PRA residents

Healthcare Utilization of PRA Residents

were less likely than the PRAC, NED, and other-HUD

We found that, in less than one year after PRA residents

groups to be dual-enrol ed, so we may have overestimated

moved into PRA units,

the impact of PRA on healthcare utilization during tenancy

to some degree.

• PRA residents tended to use inpatient hospital,

emergency department, medical transportation, and

Costs of PRA and Comparison to Other HUD

long-term inpatient services at lower rates than similar

Programs Serving Similar Populations

individuals in the comparison groups, but few of the

The PRA program leverages rental assistance in

differences were statistical y significant.

multifamily developments built with other capital funding

• We did find statistical y significant differences in

sources such as the Low-Income Housing Tax Credit

healthcare utilization after receiving housing assistance

(LIHTC) program. To promote cost-effectiveness, the

between PRA residents and people not receiving

program seeks to maximize the number of units assisted

HUD assistance: lower use of long-term inpatient

at the lowest feasible per unit subsidy cost, while

care services and greater use of case management

maintaining the long-term affordability requirements of

services. The absence of statistical y significant

the units. Additional y, PRA residents must have access

differences among the HUD programs in utilization of

to Medicaid-funded or state plan services that help them

health care services suggests that housing subsidies

transition to and remain stably housed in community-

to help people with disabilities remain in community-

based housing and ideal y reduce use of costly long-term

based housing may matter more than the type of

care and emergency department services.

housing assistance. Because of small sample sizes

To assess the PRA program’s cost-effectiveness relative

and the short fol ow-up period, this inference should

to other programs, we col ected program documents and

be viewed with caution, however.

analyzed available administrative data on program costs

• PRA residents were more likely to use personal

for the PRA program and the comparison group programs

care assistance or case management services,

that are assisted by HUD (PRAC, NED, other HUD

the study’s proxies for Medicaid-funded HCBS.

programs). Specifically, we analyzed capital costs, rental

These differences may reflect greater access to or

subsidy costs, healthcare and disability-related services

coordination of services, or a history of unmet

(paid or unpaid), and program administrative costs.

needs prior to PRA tenancy.

The cost structures across programs are very different,

This analysis provides early evidence that the PRA

the PRA program is still in the relatively early stages

program might have a substantive long-term impact on

of implementation, and the cost data available to the

healthcare utilization in a population with many unmet

study team are not complete across all the comparison

healthcare needs. There are caveats to drawing definitive

groups, for all cost categories. The study also found that

conclusions, however. The PRA tenancy period in this

PRA residents in the study states had higher prevalence

evaluation was one year or less, and it is likely too short

of chronic and disabling conditions and tended to use

a period to detect or attribute significant changes in

healthcare services at higher rates than individuals in the

patterns of healthcare utilization to the PRA program,

comparison groups.

particularly in rare outcomes like transitions to long-term

Given these caveats, our preliminary findings are:

care institutions.

• Rental subsidy costs for PRA residents are higher than

Additional y, while we estimate that between 20 and 40

for PRAC residents, but lower than for NED and the

percent of PRA residents and members of our comparison

other HUD-assisted housing programs. Per unit annual

group are dual-enrol ed in Medicaid and Medicare,

rental subsidies range from $6,841 for PRA units to

we had access only to Medicaid data. Medicare is the

$7,872 for NED vouchers.

primary payer for hospitalizations, physician services,

xv

H U D S e c t i o n 811 P R A P ro g r a m – P h a s e I I E v a l u a t i o n

Executive Summary

• Estimated total housing costs (capital and rental

Securing PRA Units under Contract

subsidies) are $11,800 per unit, per year for PRA units,

As of September 2018, national y PRA grantees and their

compared to between $12,000 and $13,000 per unit

partners have secured contracts for approximately 2,200

per year for PRAC units. The estimated annual cost of

of the 6,000 units the program is expected to assist. The

rental assistance in the PRA program is $6,941 while

PRA program has successful y attracted owners wil ing

capital subsidy costs are estimated at $4,969 annual y.

to enter long-term rental assistance contracts, general y

(Capital subsidies are either unknown or not applicable

at rents below the program’s limit set at HUD’s Fair

for the other comparison groups.) In the PRA program,

Market Rents. Roughly one-third of units committed to

many capital costs are incurred by non-HUD programs

the program are under lease, although most residents

such as the Low-Income Housing Tax Credit.

had been housed less than one year at the time of this

• Program administrative costs are much higher for the

evaluation.

PRA program ($5,780 per unit, annual y) compared to

The majority of PRA residents report that they like where

the comparison group costs of less than $1,000 per

they live and feel safe in their neighborhoods, but a

unit annual y. Grantee costs represent just less than

quarter of residents report concerns with property quality

half (43 percent) of the administrative costs, state

and safety. A fifth of PRA residents report unresolved

agency partner costs represent about 50 percent,

maintenance issues. PRA residents are less likely to

and the cost to HUD represents about 7 percent.

report that they feel safe in their building or neighborhood

PRA costs may go down as the program matures

compared to PRAC residents. While PRA units are

and more residents are housed, potential y driving

located in neighborhoods with higher rates of walkability

down per-unit costs.

and access to public transportation than most of the

• In al , total annual program costs are $17,577 per PRA

comparison groups, PRA units are located in census

unit compared to almost $14,000 for PRAC units.

tracts with higher concentrations of poverty, lower

• The annual estimated cost of healthcare and disability

levels of education, and lower levels of owner-occupied

services for PRA residents is $51,179, slightly higher

housing. PRA residents also live in neighborhoods with

than for PRAC ($50,321), and substantial y lower

higher exposure to harmful environmental toxins. Several

than for NED ($56,025). For residents of other HUD

grantees have sought waivers to increase targeted rents,

programs, the annual estimated healthcare and

given the challenges of attracting units with modest

disability costs were much lower, $34,204.

rents. If granted, higher rents may attract more owner

interest and give PRA residents more choices of units and

Strategies to Address

neighborhoods.

Implementation Challenges

Identifying and Selecting the PRA Target Populations

PRA grantees and their partners are successful y housing

Identifying the Right Unit for the Right Person,

the vulnerable groups that grantees target. In the six study

at the Right Time, Continues to be the Central

Challenge of the PRA Approach

states, about half of the 1,459 planned units are occupied.

Almost half of PRA residents were previously living in

As documented by Phase I of the evaluation, the PRA

institutions (27 percent) or experiencing homelessness (20

program is challenging to implement. The program’s

percent) before moving to a PRA unit. PRA residents to

administration and cost structure differ in a number

date have histories of high rates of chronic and disabling

of ways from HUD’s other rental subsidy programs.

conditions and higher rates of healthcare utilization than

In addition, grantees primarily target populations with

people in HUD’s other assistance programs that serve

extensive needs—those who have been living in or are at

non-elderly people with disabilities.

risk of admission to institutions, and those experiencing

or at risk of homelessness. Finding and engaging

While the PRA program is reaching and engaging

eligible PRA applicants and matching them to available,

applicants, ineligibility continues to be an issue. Many

appropriate units that meet their needs and preferences—

applicants do not meet PRA program requirements for

where and when they are ready to move—is very

income or Medicaid eligibility. Even those who do may

challenging. States are working to meet these challenges

not meet the leasing requirements (for example, credit

in multiple ways.

and criminal records checks) at the property where they

wish to live. Grantees have greater success reaching

these populations and clarifying eligibility when outreach

strategies are tailored to the needs and current

xvi

H U D S e c t i o n 811 P R A P ro g r a m – P h a s e I I E v a l u a t i o n

Executive Summary

(pre-PRA) living situations of each group. Finding effective,

needs, and reasons for program exits. Grantees and

efficient ways to manage eligibility determination and

their state partners may also want to monitor differences

waiting lists also facilitates timelier housing placement,

in tenancy outcomes by target population to see if

as do strategies to work with property owners to mitigate

some populations are more successfully maintaining

concerns about poor credit or criminal histories.

community-based housing than others. Results after

less than one year in housing appear promising but may

Achieving Stable Housing and Access to Community-

based Services

not be definitive. The ongoing study is challenging, given

how complicated and costly it is to acquire and match

PRA residents should have access to the community-

HUD and Medicaid data. We encourage HUD and CMS

based services they need to ensure they can remain in

to pursue opportunities to streamline data sharing in

their homes as long as they like and to promote positive

ways that protect individual privacy and support rigorous

health outcomes. Given the short tenure of most PRA

research. In addition to pursuing opportunities to share

residents, we cannot say definitively that these goals are

data among federal agencies, HUD and CMS should

being reached, but early evidence indicates that PRA

explore similar opportunities to share data with state

residents use fewer high-cost healthcare services after

housing or Medicaid agencies. Such partnerships could

they are housed than they did in the pre-occupancy period

include technical assistance for state agencies in linking

or relative to similar populations living in other housing

and interpreting data.

settings. This provides early evidence that positive

outcomes may be observed in the future.

It is not clear that PRA grantees will be able to continue

securing high-quality units at rents below FMR, especially

Sustaining PRA Partnerships to Ensure Effective Ongoing

in high-cost areas. Overall PRA residents report a positive

Implementation of the PRA Program

experience with their housing and neighborhoods but

The ultimate goal of the PRA program is to create

not to the same degree as PRAC residents. HUD should

institutional knowledge and capacity within states to

exercise flexibility in working with grantees who

further expand the availability of supportive housing

seek waivers to increase rents to FMR. This strategy

for people with disabilities. At the core of this effort are

potential y has dual benefits. It should help attract owners

sustainable partnerships between health and housing

with high-quality housing and provide more housing

agencies that can bring together their respective

choices to PRA applicants. It will have cost implications

resources and expertise. These partnerships grow over

however, as average per-unit costs may increase.

time, and many have their antecedents in previous state or

Incentives in future PRA grant Notices of Funding

CMS initiatives.

Availability (NOFAs) that promote locating units in higher

quality neighborhoods, rather than incentives for setting

The grantees we evaluated see their partnerships as

contract rents lower than the maximum al owed, could

successful and offer insight into strategies to forming and

be another tool to attract PRA units in neighborhoods

deepening them. These include regular meetings and

PRA residents perceive as safe. As could strengthening

communication, recognizing and valuing the expertise

inspections requirements for units placed under contract

of each partner, and automating or documenting key

for PRA.

knowledge and functions so they are not lost when

individual staff move on. As documented in the cost

PRA partnerships between state housing and health

analysis for this study, however, the intensity of this effort

agencies have the potential to help break down silos

contributes to relatively high PRA program administrative

across systems that have traditional y not been wel -

costs compared to other HUD programs.

coordinated, but program administrators report that they

are time-consuming and costly. Costs may go down in

Policy Implications for State

the longer term, but HUD and CMS should continue to

and Federal Stakeholders

support technical assistance to grantees and their

Based on the results of this study, we see early evidence

partners to build capacity, share information and tools

that the PRA program is achieving its aims. Grantees are

across grantees, and institutionalize knowledge so that

moving eligible households to community-based housing,

staff turnover is less disruptive.

and early outcomes appear promising. The research

HUD should explore how the PRA cap of 25 percent

raises several policy implications and suggestions for

units set aside for people with disabilities interacts

further inquiry.

with state incentives and property owner experience.

Going forward, HUD should continue to monitor

PRA grantee reporting indicates that PRA units total

tenancy outcomes in program tenure, unmet support

just 10 percent of all units in developments with units

xvii

H U D S e c t i o n 811 P R A P ro g r a m – P h a s e I I E v a l u a t i o n

Executive Summary

under contract, well below the 25-percent limit. However,

While the short observation period for PRA-supported

some state affordable housing strategies (notably

residency limits our ability to draw definitive policy

through states’ Low-Income Housing Tax Credit program

implications regarding healthcare impacts, we did observe

al ocation processes) provide incentives for higher set-

some differences in service utilization over the short

asides of housing for people with disabilities that may

term that could translate into long-term trends. People

conflict with the PRA cap. Further, anecdotal y we heard

with disabilities in our study groups who were receiving

that some developments have additional people with

housing assistance through HUD had lower rates of

disabilities living in their properties who are not in set-

institutional care than those not supported by HUD

aside units.

programs. Community-based supports such as use of

What the “right” set-aside level should be to ensure

personal care attendants, are on average less costly than

community-integrated housing is difficult to assess. If

institutional care and can contribute to improved health

states reduce incentives in other programs to set aside

status and reduction in unplanned and emergency care.

units for people with disabilities to align with the PRA

CMS should continue to work with states to support

program’s caps, it may reduce the overall expansion of

provision of HCBS, through Medicaid or other state

the supply of units for this population. HUD should work

funding sources, coupled with housing supports, to

with states to explore how their incentive structures affect

assist people with disabilities to live independently

the shared federal-state goals of expanding housing

and promote more cost-effective utilization of

opportunities for people with disabilities while permitting

healthcare services.

them to live in integrated settings.

xviii

H U D S e c t i o n 811 P R A P ro g r a m – P h a s e I I E v a l u a t i o n

Chapter 1. About the PRA Program Evaluation and This Report

About the PRA

services (HCBS) or similar state plan services. HCBS

provide opportunities for Medicaid beneficiaries with

Program Evaluation

a wide range of disabilities to receive services in

their own home or community rather than institutions

or other isolated settings. Nonprofit owners of

and This Report

PRAC housing ensure that residents have access to

voluntary, community-based services.

• While PRAC funds primarily developments exclusively

for people with disabilities, PRA subsidies may be used

only in developments where no more than 25 percent

The Department of Housing and Urban Development

of units are set aside for people with disabilities.

(HUD) supports affordable housing opportunities for

people with disabilities through several housing assistance

• PRA households must have extremely low household

programs. Since 1991, the Section 811 Supportive

incomes, with no more than 30 percent of Area Median

Housing for Persons with Disabilities Program has

Income (AMI), while PRAC households can have up to

been the primary source of new housing developed

50 percent of AMI.

exclusively for non-elderly people with disabilities. The

The PRA program was designed to respond to a number

Section 811 supportive housing model provides residents

of policy priorities:

affordable housing and access to appropriate, voluntary

• To increase the supply of affordable housing for

supportive services. Since the Section 811 program’s

people with disabilities in a cost-effective way, while

inception, the Section 811 Capital Advance and the Project

continuing to serve households with extremely low

Rental Assistant Contract (PRAC) program has provided

incomes.

interest-free capital grants and operating subsidies to

nonprofit housing sponsors to develop and operate

• To provide affordable, community-based housing

properties that exclusively house very low-income people

options for people who might otherwise be, or be

with disabilities. The PRAC program assists approximately

at risk of becoming, homeless or unnecessarily

34,000 people living in group homes, small multifamily

institutionalized.

properties, or condominiums.

• To offer integrated housing settings where people

In 2010, the Frank Melville Supportive Housing

with disabilities live in multifamily housing that assists

Investment Act of 2010 authorized the Section 811

people both with and without disabilities.

Project Rental Assistance (PRA) program, providing an

• To encourage collaboration between state housing

alternative approach to providing permanent supportive

and health agencies that results in long-term strategies

housing for non-elderly persons with disabilities. The PRA

for providing permanent, affordable housing options

program is a joint initiative between HUD and the U.S.

for people with disabilities and structured access to

Department of Health and Human Services (HHS) Centers

services.

for Medicare & Medicaid Services (CMS).

The Melvil e Act required an evaluation of this new

The PRAC and PRA programs both assist low income,

approach to providing housing and services for non-

non-elderly people with disabilities, but the PRA program

elderly people with disabilities. This chapter provides

differs from the PRAC program in important ways:

background on the PRA program’s requirements and

• While PRAC offers capital grants, the PRA program

the status of program implementation. It then describes

provides project-based rental assistance only for units

HUD’s multi-phase evaluation strategy and provides

in affordable housing developments built with other

details on the Phase II evaluation design upon which this

federal or state funding, such as Low-Income Housing

report is based. The chapter’s last section provides an

Tax Credits (LIHTC).

overview of the remainder of this Final Report.

• While PRAC relies on nonprofit sponsors to develop

1.1 Section 811 PRA

and operate the housing, PRA grants are awarded to

state housing agencies that must partner with state

Grant Requirements

health agencies to ensure access to services and

The Melvil e Act that authorized the Section 811 PRA

supports.

program established requirements for properties that

• Those receiving PRA assistance must be eligible

accept PRA subsidies and for residents to be eligible

for Medicaid-funded home and community-based

to live in them. HUD developed additional program

1

H U D S e c t i o n 811 P R A P ro g r a m – P h a s e I I E v a l u a t i o n

Chapter 1. About the PRA Program Evaluation and This Report

requirements. Section 811 PRA program funds are

up 34 percent of FY12 units. FY13 grantees have leased

awarded in grant competitions, announced by federal

4 percent of their planned units. Two-thirds of all PRA

NOFAs. There have been two PRA funding rounds to date:

residents national y have been assisted by PRA for less

Section 811 Project Rental Assistance Program Requirements

a first round for Fiscal Year 2012 (FY12) in February 2013,

than a year.

and a second round for Fiscal Years 2013 and 2014 in

At the time they moved into FY12 grantee units,

March 2015 (FY13).

State Agency Partnerships

nearly one-third of households (32 percent) had been

The NOFAs specify program requirements for grant

living in institutions, and nearly one-fourth were

A partnership agreement between the state housing agency and state Medicaid and/or HHS agency is applicants, grantees, properties and property owners, and

experiencing homelessness (23 percent) directly

required as part of the application for PRA grant funds. The agreement outlines the state housing agency’s residents. Key requirements are summarized in the text

prior to being assisted by PRA. Six percent of residents

commitment to administering the rental subsidy program and the health agency’s commitment to identifying box on the next page.

moved from group homes, adult care homes, or other

and conducting outreach to the target population(s) to be served by the state’s PRA program, as well as residential settings for people with disabilities. Six percent

ensuring that residents are connected to appropriate, voluntary supportive services.

1.2 National Status of PRA

moved other types of housing such as living on their own

Program Implementation

or with roommates in the community or living with family.

Of households that moved into FY13 grant units, nearly

Eligibility of Properties for Rental Assistance

Congress has approved funding for two rounds of

one-fourth had been institutionalized (24 percent), and

The state housing agency grantees select properties to receive PRA subsidies. Eligible properties can be Section 811 PRA program grants, expected to assist

more than one-fifth were experiencing homelessness (21

new or existing multifamily developments in which the development costs are subsidized by other sources.

approximately 6,000 households. The Melvil e Act

percent).

These sources include LIHTC, HUD’s HOME and Community Development Block Grant (CDBG) programs,

authorized up to $300 mil ion in PRA funds to be awarded

Demand for PRA units exceeds supply. As a result,

and other federal, state, or private sources.

between fiscal years 2011 and 2015. HUD awarded

many of the state PRA programs maintain waiting lists

a demonstration round of grants to 13 state housing

of potential PRA applicants. As of September 30, 2018,

agencies for FY12 in February 2013 and awarded a

grantees in the 27 participating states reported 5,991

Rental Assistance Contract Requirements

second round of grants for FY13 in March 2015. Across

applicants on their waiting lists for units available

the two funding rounds, HUD awarded funding to 30 state

Property owners must agree to a 30-year use restriction for providing a specified number of units for PRA.

from FY12 grants and 3,302 applicants for units from

housing agencies.

Owners must execute a Rental Assistance Contract (RAC) with the state housing agency for at least 20

FY13 grants.

years. (Both are contingent on continued Section 811 appropriations.) To ensure that community integration Of the awardees, 27 states had begun PRA programs

National y, 1,229 households have moved into PRA units

goals are achieved under the Section 811 PRA program, the Section 811 statute specifies that no more than as of September 2018.7 The 27 grantees entered into

since the PRA program began in 2015. Cumulatively,

25 percent of the units in the affordable housing development can be set aside for supportive housing or Cooperative Agreements with HUD for a combined

approximately 20 percent of PRA households (217)

have an occupancy preference for people with disabilities. PRA units should also be dispersed throughout $229 mil ion: $88 mil ion in FY12 grant funds and $141

have exited the PRA program over that period.

the property and not segregated to one area of a property.

mil ion in FY13 funds. The FY12 grants will subsidize an

Approximately one-third of these exits were owner-

estimated 2,283 units, and the FY 13 grants are expected

initiated (32 percent): 8 percent for nonpayment of rent

to subsidize 3,772 units.8 The FY12 grants must be

and 24 percent for other reasons not specified. Another

disbursed by September 30, 2025; the FY13 grants must

Section 811 PRA Rents

third of exits were initiated by tenants: 25 percent left for

be disbursed by September 30, 2026.

Grantees determine the maximum rents that property owners may charge for PRA units. PRA rents may not other housing, and 9 percent left for other reasons. An

exceed the applicable FMR or small area FMR level for the location of the property unless supported by a Status of PRA Grants as of September 2018

additional 18 percent of residents died, 8 percent moved

market study. The FY12 and FY13 grant competitions included incentives for grantees to commit to PRA back into institutional care, and 9 percent left for other or

Nationally, 75 percent of the FY12 units are under

rents even lower than FMR.

unknown reasons or moved out without giving notice.

contract with owners for PRA. PRA grantees are making

progress in attracting owners to the program and securing

1.3 The Evaluation of the Section

agreements for units to be subsidized by PRA. The FY12

Resident Eligibility for PRA

grantees had contracts in place with property owners for

811 PRA Program

PRA funds may only be provided for units for households with at least one person with a disability who an estimated 1,718 PRA units, or 75 percent of the units’

The PRA program represents a new model for providing

is between the ages of 18 and 61 at the time the person is first assisted by PRA subsidies. PRA residents grantees planned for in their Cooperative Agreements with

affordable housing and access to community-based

must also be eligible for home and community-based services funded under Medicaid waivers, Medicaid HUD. The FY13 grantees had entered into contracts with

supportive services for non-elderly people with disabilities.

state plan options, or other comparable programs. Households assisted by PRA must have extremely low owners for an estimated 516 PRA units, or 13 percent of

The model offers considerable flexibility to states to

household income (no more than 30 percent of AMI).

the units planned.

select target populations, housing types and locations,

Grantees funded in FY12 are leasing a third of their

and service strategies that reflect available resources,

awarded PRA units. A total of 945 households were living

address state policy priorities, and meet potential PRA

Resident Contribution to Rent

in PRA-subsidized units as of September 2018, making

residents’ needs and preferences. It is not clear whether

Similar to households in other HUD rental subsidy programs, residents living in PRA units pay rent based on 7

State housing agencies in three states that were awarded PRA grants in FY12 and FY13 chose not to participate in the program: North Carolina their incomes. Total tenant payment (rent and utilities) is calculated as the greatest of 30 percent of adjusted (FY12), the District of Columbia (FY13), and Kentucky (FY13).

8

monthly income, 10 percent of monthly gross income, or a state-determined minimum welfare rent.

The estimate is based on the grantee’s projections about rent levels and subsidies for PRA units. The actual number of households will depend on actual rents and subsidies.

2

H U D S e c t i o n 811 P R A P ro g r a m – P h a s e I I E v a l u a t i o n

Chapter 1. About the PRA Program Evaluation and This Report

requirements. Section 811 PRA program funds are

awarded in grant competitions, announced by federal

NOFAs. There have been two PRA funding rounds to date:

Section 811 Project Rental Assistance Program Requirements

a first round for Fiscal Year 2012 (FY12) in February 2013,

and a second round for Fiscal Years 2013 and 2014 in

March 2015 (FY13).

State Agency Partnerships

The NOFAs specify program requirements for grant

A partnership agreement between the state housing agency and state Medicaid and/or HHS agency is applicants, grantees, properties and property owners, and

required as part of the application for PRA grant funds. The agreement outlines the state housing agency’s residents. Key requirements are summarized in the text

commitment to administering the rental subsidy program and the health agency’s commitment to identifying box on the next page.

and conducting outreach to the target population(s) to be served by the state’s PRA program, as well as ensuring that residents are connected to appropriate, voluntary supportive services.

Eligibility of Properties for Rental Assistance

The state housing agency grantees select properties to receive PRA subsidies. Eligible properties can be new or existing multifamily developments in which the development costs are subsidized by other sources.

These sources include LIHTC, HUD’s HOME and Community Development Block Grant (CDBG) programs,

and other federal, state, or private sources.

Rental Assistance Contract Requirements

Property owners must agree to a 30-year use restriction for providing a specified number of units for PRA.

Owners must execute a Rental Assistance Contract (RAC) with the state housing agency for at least 20

years. (Both are contingent on continued Section 811 appropriations.) To ensure that community integration goals are achieved under the Section 811 PRA program, the Section 811 statute specifies that no more than 25 percent of the units in the affordable housing development can be set aside for supportive housing or have an occupancy preference for people with disabilities. PRA units should also be dispersed throughout the property and not segregated to one area of a property.

Section 811 PRA Rents

Grantees determine the maximum rents that property owners may charge for PRA units. PRA rents may not exceed the applicable FMR or small area FMR level for the location of the property unless supported by a market study. The FY12 and FY13 grant competitions included incentives for grantees to commit to PRA rents even lower than FMR.

Resident Eligibility for PRA

PRA funds may only be provided for units for households with at least one person with a disability who is between the ages of 18 and 61 at the time the person is first assisted by PRA subsidies. PRA residents must also be eligible for home and community-based services funded under Medicaid waivers, Medicaid state plan options, or other comparable programs. Households assisted by PRA must have extremely low household income (no more than 30 percent of AMI).

Resident Contribution to Rent

Similar to households in other HUD rental subsidy programs, residents living in PRA units pay rent based on their incomes. Total tenant payment (rent and utilities) is calculated as the greatest of 30 percent of adjusted monthly income, 10 percent of monthly gross income, or a state-determined minimum welfare rent.

3

H U D S e c t i o n 811 P R A P ro g r a m – P h a s e I I E v a l u a t i o n

Chapter 1. About the PRA Program Evaluation and This Report

the PRA approach will be an improvement on the PRAC

questions about the PRA program’s outcomes and

approach or on other HUD programs that provide housing

effectiveness. It focuses on PRA programs in six states:

for people with disabilities. To help answer this question,

California, Delaware, Louisiana, Maryland, Minnesota,

the Melvil e Act that authorized the PRA program also

and Washington. These states were selected because

required an evaluation of the program’s effectiveness. In

they housed the largest numbers of PRA residents

response, HUD has undertaken a multi-phase evaluation

when the study’s research design was finalized in 2017,

to learn how this new model is implemented and what the

giving the evaluation the best chance to detect program

outcomes are for PRA-assisted households.

outcomes for PRA residents. As of September 2018, when

data collection for the Phase II evaluation closed, these

Phase I Evaluation (2014-2016)

six states were assisting more than 500 households,

An initial, Section 811 PRA Phase I Evaluation assessed

representing 62 percent of all PRA residents at the time.

the early implementation of the PRA program in the first

12 states to receive PRA grants.9 Covering the period

1.4 The Phase II Evaluation Design

between October 2014 and June 2016, the evaluation

documented how state housing and health agencies and

Phase II Evaluation Objectives, Research

their partners developed outreach and referral procedures

Questions, and Data Sources

to identify eligible applicants for PRA units and how they

The overarching goals of the Phase II Evaluation are

identified and contracted with property owners to lease

to assess the outcomes and effectiveness of the PRA

units to PRA applicants. The Phase I evaluation report

program and to compare the results to outcomes for

assessed the early implementation experience. Given

similar populations living in other housing settings. The

delays in program implementation and challenges with

research objectives and research questions for the Phase

identifying units that met the requirements of the PRA

II Evaluation and the data sources to address them are

program, however, very few PRA residents were housed

shown in Exhibit 1.1. The evaluation draws on a rich variety

by the end of the Phase I study. It was thus too early to

of data obtained from administrative sources as well as

evaluate the effects of the program on residents.

through in-person interviews with program administrators.

In addition, the study team conducted more than 400

Phase II Evaluation (2016-2019)

in-person surveys with PRA and PRAC residents to gain

This report is the result of the Section 811 PRA Phase

further insights into their program experiences and their

II Evaluation, based on research conducted between

self-reported quality of life and health status.

2016 and 2019. The Phase II evaluation addresses

E xhibit 1.1: Evaluation Objectives, Research Questions, and Dat a S ources

Research Questions

Objectives

Data Sources

• In-person surveys administered to PRA and

Assess the effects of the PRA program on individuals’

PRAC residents

How do short-term impacts of the Section 811 PRA

quality of life and care, housing and neighborhood,

• Housing, healthcare utilization, and neighborhood

program compare to outcomes for similar people

and utilization and access to health services and

administrative data from HUD, CMS, state

served in other HUD housing programs?

supports compared to similar people living in other

Medicaid agencies, and publicly available federal

settings.

datasets

• PRA program documents and administrative data

• Document reviews and follow-up administrative

Estimate the costs and cost effectiveness of providing

What are costs of the PRA program and how do they

interviews with staff from PRA grantees, state

housing and services to PRA program residents

compare to costs for other HUD programs serving

Medicaid agencies, and other PRA program

compared to the costs for similar populations assisted

similar populations?

partners

in other housing settings.

• Administrative data on housing and services

costs from HUD and CMS, respectively

• PRA program documents and administrative data

Continue documenting the implementation of the

• Administrative interviews with HUD staff who

PRA program in six study states, particularly how

What is the relationship between PRA program

administer the PRA program

implementation strategies have changed as the

features and strategies and program results?

• Administrative interviews with staff from PRA

programs have matured and how those strategies

grantees, state Medicaid agencies, and other

may contribute to program results.

PRA program partners

9

https://www.huduser.gov/portal/section-811-process-evaluation.html

4

H U D S e c t i o n 811 P R A P ro g r a m – P h a s e I I E v a l u a t i o n

Chapter 1. About the PRA Program Evaluation and This Report

The Evaluation’s Comparison Groups

to meet tenants’ physical health, mental health, and other

A key feature of the Phase II Evaluation’s design is a

needs for the duration of the 40-year capital advance

comparison of participant characteristics and outcomes

period. PRAC sponsors must either provide services

for non-elderly people with disabilities across a variety of

directly or partner and coordinate with service providers

housing settings. The study compares Section 811 PRA

in the community. Like PRA, services are voluntary for

residents to four distinct comparison groups, for three

residents. Unlike PRA, PRAC properties predominately

purposes.

house single individuals with disabilities. The cost

structure of the PRAC program is also different in that

First, the study examines descriptively how non-

sponsors receive interest-free capital grants and on-going

elderly people with disabilities across living

operating subsidies to ensure affordability. The subsidy is,

situations are similar or different. The study compares

like PRA, project-based. That means residents who move

demographic and socioeconomic characteristics,

may not take their subsidy with them.

health diagnoses and chronic conditions, and historical

healthcare utilization patterns of PRA residents to non-

Group 2: NED

elderly people with disabilities living in other settings.

This group is made up of people assisted by HUD’s Non-

This helps answer the question, whom is the PRA

Elderly Disabled (NED) voucher program. These vouchers

program serving and how do they compare to people with

are administered by local public housing authorities

disabilities living in other settings?

(PHAs) and provide tenant-based subsidies to eligible

Second, we use statistical techniques to compare

applicants. NED voucher holders may rent a unit of their

outcomes for PRA residents to outcomes for similar

choice that meets HUD and the PHA’s rent and housing

people in other housing settings. This helps answer the

quality guidelines. NED voucher holders who move

question, how would PRA residents have fared in other

may take their subsidy with them. The NED voucher is

housing settings, in the absence of the PRA program?

a housing subsidy only; HUD does not require a formal

mechanism for ensuring voucher recipients’ access to

Third, the study explores the costs of administering

services. Some PHAs partner with service agencies to

housing and services through the PRA program and,

help NED voucher-holders access services.11 There are no

to the extent data are available, how those costs

capital subsidies in the NED voucher program.

compare to the costs of providing housing and

services through other HUD capital or rental subsidy

Group 3: Other HUD

programs. This analysis helps address questions about

Unlike groups 1 and 2, this group is made up of non-

the PRA program’s cost-effectiveness compared to other

elderly people with disabilities who are living in other

settings.

HUD-assisted units that are not restricted to people with

The evaluation’s comparison groups were selected to

disabilities. These housing settings include the Housing

represent a range of housing settings where non-elderly

Choice Voucher program (76 percent of residents in the

people with disabilities may live. Three groups are made

other HUD group), public housing (22 percent), and other

up of people living in HUD-assisted housing. All of the

HUD multifamily programs (2 percent). These programs

HUD programs assist households with very low incomes

assist very low-income renters, including families, the

(not more than 50 percent of AMI), but other program

elderly, and people with disabilities. As in PRA, PRAC,

features vary across programs. The comparison groups

and NED, tenants’ rent payments in these programs are

and some of their important features are as fol ows. See

general y limited to 30 percent of income. In most cases,

Exhibit 1.2 for a comparative matrix of differences across

there is no requirement to ensure access to services,

the HUD programs.

although some public or multifamily developments may

have service coordinators or may partner with service

Group 1: PRAC

agencies to assist residents. Public housing is built with

This group is made up of residents in HUD’s Section 811

HUD capital subsidies and also receives an operating

Project Rental Assistance Contract program. The PRAC

subsidy. There are no capital subsidies associated with

program’s eligibility requirements regarding age, disability,

the HCV. Like NED vouchers, HCV holders who move may

and income are broadly similar to those of the PRA

take their subsidy with them.

program.10 PRAC sponsors must make services available

10 The PRAC program is restricted to households with very low incomes (earning no more than 50 percent of AMI) while the PRA program is restricted to households with extremely low household incomes (earning no more than 30 percent of AMI).

11 NED Category 2 vouchers, a small percentage of NED vouchers, are targeted to people exiting institutions. PHAs applying for Category 2

vouchers must partner with a state Medicaid or health agency to apply for Category 2 vouchers.

5

H U D S e c t i o n 811 P R A P ro g r a m – P h a s e I I E v a l u a t i o n

Chapter 1. About the PRA Program Evaluation and This Report

Group 4: Non-HUD

experiencing homelessness. Because we do not know

Unlike groups 1 through 3, this group consists of people

the housing setting for this group, we cannot assess their

who are enrol ed in Medicaid but who are not assisted

housing experiences (for example, costs, affordability

in any of the HUD programs. The housing setting for

relative to income, tenure, neighborhood quality, and

this group is unknown, but may include people living

so on), but we can compare their healthcare utilization

in institutions, those living with family, those living

characteristics and outcomes to the PRA group.

independently in unsubsidized housing, or people

E xhibit 1.2 : Summar y of Program Features for H U D C omparison Programs

Program Feature

PRA

PRAC

NED

Other HUD

Non-elderly people with

Non-elderly people with

Non-elderly people with

Families, elderly, non-elderly

Target population

disabilities; extremely low-

disabilities; very low-income

disabilities

people with disabilities

income household

household

HCV: tenant-based rental

assistance

Public housing: capital and

Project-based rental

Project-based capital grants

Subsidy type

Tenant-based rental assistance operating subsidy

assistance

and operating subsidies

Multifamily: capital subsidy

and/or project-based rental

assistance

Local housing authorities

State housing agencies

or (for HUD multifamily)

Program administrator

partnering with state health

Non-profit sponsors

Local housing authorities

affordable housing owner/

agencies

manager

HCV: no formal service

strategy

Property service plan

Coordination with state health

describes how residents will

Housing subsidy only, housing

Approach to supportive

Public/multifamily housing:

agency to ensure access to

be ensured access to services

authority may partner with

services

PHA or property owner may

HCBS

provided by the sponsor or

service providers

partner with service providers

partner providers

or provide on-site service

coordinators

HCV: Used in private rental

Mixed population housing;

housing, no restrictions

Used in private rental housing,

not more than 25 percent of

Housing exclusively for people

regarding population type

Housing integration

no restrictions regarding

units set aside for people with

with disabilities

population type

disabilities

Public/Multifamily: Mixed

population housing

Limitations of the Evaluation’s Design

The study attempted to col ect post-occupancy

The evaluation leverages a diverse set of data sources,

healthcare utilization data for as many PRA residents

including information col ected directly from a wide range

as possible, for as long as possible after they moved

of program stakeholders, from state agency administrators

into their units. Given lags in data availability and the

to service providers and property managers to PRA

relatively slow pace of PRA lease-ups, we only observe

subsidy recipients. There are, however, limitations to the

approximately 7 months of healthcare utilization after

analysis we were able to conduct:

residents moved into their units. Results from this brief

follow-up period may not reflect longer-term patterns

• Limited post-occupancy follow-up period for PRA

that may be observed in the future.

residents. The healthcare utilization and outcomes

analysis relies on state Medicaid data to compare

• Resident survey data limited to PRA and PRAC

healthcare utilization before PRA residents moved into

residents. The evaluation budget did not permit

their PRA unit with their post-move-in experiences.

surveys with residents from all of the comparison

6

H U D S e c t i o n 811 P R A P ro g r a m – P h a s e I I E v a l u a t i o n

Chapter 1. About the PRA Program Evaluation and This Report

groups. Priority was given to surveying PRA and PRAC

these costs are al ocated among funding sources.

residents.

To the extent that data are available, the section

• No cost-effectiveness analysis for the non-HUD

also assesses how cost-effective the PRA program

group. As noted in the comparison group description,

compared to other HUD programs that serve similar

the non-HUD group is made up of a sample of

populations.

Medicaid beneficiaries who are not receiving HUD

• The ninth chapter: Relationship Between PRA

assistance. The sample was drawn from Medicaid

Program Strategies and Program Results explores

data, so the study can report on some demographic

the relationship between elective features or strategies

and health characteristics and on healthcare utilization

established by state PRA partnerships and the early

and costs for this group. There is no information on

results of the program. The section discusses how

housing costs for this group, which precluded a cost-

the selection and prioritization of state-selected PRA

effectiveness analysis for this comparison group.

target population(s) affect whom the PRA program

• Other data limitations. The study drew on a variety

assists, and what strategies have been successful in

of administrative data sources to produce the most

reaching and referring eligible members of the states’

complete and robust analyses possible, but some

target populations. The section also discusses how

analyses were limited by incomplete or unavailable

states have addressed challenges in getting high-

data for some comparison groups. These limitations

quality, cost-effective units under contract for PRA,

are noted throughout the report.

what strategies have been successful in matching

applicants to units that meet their needs, and how

1.5 Structure of This Report

states coordinate tenancy supports for PRA residents.

Final y, the section discusses what strategies

Including this introductory chapter, this report consists of

state agency staff report have been successful in

nine chapters, a conclusion, and several appendices:

building and maintaining effective, sustainable PRA

• The second chapter: PRA Programs in the Six

partnerships.

Study States presents an overview of the PRA

• Conclusion: How Effective States Have Been in

programs in the six study states and an update

Meeting the Goals of the PRA Program brings

of the progress made toward their PRA grants

together the results from the report’s three main

as of September 2018, the third full year of grant

sections. The chapter offers a final assessment of how

administration.

successful state agencies and their partners have been

• The third chapter: Characteristics of PRA

in meeting the goals of the PRA program, and how

Residents and How They Differ from Comparison

early outcomes of the PRA program compare to similar

Groups presents descriptive data on who the PRA

people not assisted by PRA.

program serves and how PRA residents compare to

• Appendices: The appendices include an update of

non-elderly people with disabilities who are served by

all 27 PRA grant programs as of September 2018

other HUD programs.

(Appendix A) and a description of the study’s data

• The fourth through seventh chapters: Short-Term

sources and methods (Appendix B).

Outcomes of PRA Residents and How They Differ

from Comparison Groups presents early evidence

of how outcomes of the PRA program compare

to outcomes of other HUD programs and housing

situations for non-elderly people with disabilities.

Outcomes are compared in three areas: neighborhood

characteristics and resident satisfaction with their

neighborhood, property characteristics and resident

satisfaction with their property, home and community-

based services and housing tenancy, and healthcare

diagnoses and utilization.

• The eighth chapter: Costs of the PRA Program

and How They Compare to Other HUD Programs

discusses the housing, services, and program

administrative costs of the PRA program, and how

7

H U D S e c t i o n 811 P R A P ro g r a m – P h a s e I I E v a l u a t i o n

8

H U D S e c t i o n 811 P R A P ro g r a m – P h a s e I I E v a l u a t i o n

Chapter 2. PRA Programs in the Six Study States

PRA Programs in the

programs in those states expect to assist with their grants.

Between 2015 and 2018, more than 700 households

Six Study States

have been assisted by PRA in these states. In general,

the Section 811 PRA programs are assisting people in

two distinct, but potential y overlapping groups—people

living in nursing facilities or other facilities with institutional

levels of care and people experiencing or at risk for

homelessness.

This evaluation focuses on PRA implementation and

outcomes in 6 of the 27 states that are administering

2.1 PRA Grants in the Study States

PRA grants: California, Delaware, Louisiana, Maryland,

Minnesota, and Washington. This chapter describes

HUD awarded $66.5 million in PRA grants to the state

the states’ programs and their progress through

housing agencies in the six study states over two

September 2018, representing the first full 3 years of PRA

funding rounds. For the FY12 round of funding, the PRA

implementation. This chapter provides a brief overview

partnerships in the six states combined $44.7 mil ion

of the PRA program’s design in the study states and the

in grant funds for an estimated 948 PRA units (Exhibit

progress they made in implementing the program in their

2.1). Three of the study states (California, Maryland, and

state as of September 2018.

Minnesota) also received PRA grants in FY13, the second

HUD awarded $66.5 mil ion in PRA grants to the state

year of funding, for an additional combined $25 mil ion

housing agencies in the six study states in the FY12 and

in grant funds. The two funding rounds are expected to

FY13 funding rounds. As of September 2018, grantees in

assist an estimated 1,373 units of rental assistance—948

the six states had entered into agreements with property

units for FY12 and 425 units for FY13. Each PRA “unit”

owners for more than half of the 1,373 units the PRA

represents 5 years of rental assistance.

E xhibit 2.1: P R A Grant A mount s and Planned Unit s in the S tudy S t ates, by S t ate and Grant Year FY12

FY13

State

Grant Amount

Planned Units

Grant Amount

Planned Units

California

$11,870,256

233

$11,985,436

200

Delaware

$5,246,276

148

Louisiana

$8,489,928

199

Maryland

$10,917,383

150

$9,808,054

150

Minnesota

$3,085,500

85

$3,000,000

75

Washington

$5,739,717

133

Total

$45,348,060

948

$24,793,490

425

Note: Planned units based on number in Cooperative Agreements with HUD as of September 2018.

Sources: “Section 811 Project Rental Assistance: Bringing Supportive Service Rental Housing to Scale. Status Report to Congress,” April 2017, and data provided by HUD’s Office of Multifamily Programs.

2.2 State PRA Target Populations

Some grantees view the PRA program as a resource for

achieving compliance with the Americans with Disabilities

Section 811 PRA rental subsidies are set aside for

Act (ADA) (as affirmed by the Olmstead decision) to

households that meet the program’s requirements for

al ow people with disabilities to live in the least restrictive

age, household income, and eligibility for HCBS. Within

settings possible that meet their needs and preferences,

these federal requirements, the PRA state agency

and for helping to achieve the goals of state plans to

partnerships have flexibility in determining which

reduce homelessness. PRA partnerships also consider

populations their PRA program will serve. PRA grantees

available resources from state-administered supportive

identify target populations based on the unmet needs for

housing programs or initiatives as they determine where

people with disabilities and their states’ policy priorities.

the PRA program fits in their state context.

9

H U D S e c t i o n 811 P R A P ro g r a m – P h a s e I I E v a l u a t i o n

Chapter 2. PRA Programs in the Six Study States

All six states participate in the Money Fol ows the Person

People experiencing homelessness are eligible for PRA

(MFP) rebalancing demonstration program, a federal

units in all six states. Four states (California, Louisiana,

initiative to give people needing long-term services and

Maryland, and Minnesota) specifical y identify people

supports (LTSS) more choice about where they live and

who are homeless or at risk for homelessness as one

receive care, and to increase the capacity of state LTSS

of their target populations. State agencies in these

systems to serve people in community settings.12 Four

states partnered with homeless outreach organizations

states (California, Delaware, Minnesota, and Washington)

to identify people who could benefit from PRA. People

specifically target PRA units to individuals participating

experiencing homelessness may be receiving services

in their state’s MFP program and coordinate outreach to

under Medicaid waiver programs, through state or locally

potential PRA residents within their MFP programs.

funded programs, or may not be receiving any services at

California and Maryland also target participants of

the time they apply to the PRA program.

specific HCBS Medicaid waiver programs. Under

Exhibit 2.2 lists the target population of the study states

HCBS waivers, states can “waive” certain Medicaid

FY and FY13 grants. See the chapter titled PRA Program

program requirements in order to provide people long-

Practices that May Lead to Successful Results for more

term care services and supports in their home, rather than

information about how the selection and prioritization of

in an institutional setting. CMS must approve any waivers,

targeted populations in the study states affected who

and the number and types of Medicaid waivers vary

applies for PRA and who ends up being assisted by PRA.

among states.

E xhibit 2.2 : Target Population of Section 811 P R A Programs

State

Target Populations of FY12 and FY13 PRA Grants

• People residing in inpatient facilities and enrolled in the California Community Transitions Money Follows the Person (MFP) program California

• People receiving Medi-Cal long-term home and HBCS waiver or state plan services who are at risk for placement in inpatient facilities

• FY13 grants also target people who are homeless or at risk for homelessness

• People exiting the Delaware Psychiatric Center and/or with serious and persistent mental illness Delaware

• People exiting institutions with emphasis on MFP participants

• Identified as at risk of being admitted to long-term care facilities

• People who are inappropriately institutionalized

Louisiana

• Homeless people

• People who are at risk of homelessness (including those living in transitional housing) and those at risk of institutionalization

• People with disabilities living in institutions

• People at risk of institutionalization due to current housing situation, such as those who are homebound or living in sub-standard housing

Maryland

• People who want to move from a group home, Alternative Living Unit, or Residential Rehabilitation Program or HCBS Options Waiver participants moving from a licensed assisted living facility to independent renting

• Homeless people

• People experiencing long-term homelessness who have severe mental illness

Minnesota

• People with physical or mental disabilities exiting institutional settings and who are assisted by Minnesota’s MFP program

• People served through the Roads to Community Living program

• People with developmental disabilities served through the Developmental Disabilities Division Washington

• People with functional disabilities served through the Home and Community Services Division

• People with mental illnesses served through the Division of Behavioral Health and Recovery Source: 2012 and 2013 PRA funding applications and information provided through administrative interviews with state housing and health agencies.

2.3 PRA Program Profiles

California

in the Six Study States

California’s PRA program is a partnership among five state

agency partners: the California Housing Finance Agency

This section provides brief profiles of the PRA programs in

(CalHFA), California Tax Credit Al ocation Committee, the

the six study states.

Department of Housing and Community Development;

the Department of Health Care Services (DHCS), and

12 https://www.medicaid.gov/medicaid/ltss/money-follows-the-person/index.html

10

H U D S e c t i o n 811 P R A P ro g r a m – P h a s e I I E v a l u a t i o n

Chapter 2. PRA Programs in the Six Study States

the Department of Developmental Services. CalHFA

Mental Health. DHSS case managers or DHSS-approved

and their state partners applied for the maximum grant

service providers identify applicants for the PRA program.

amount in the FY12 funding round. The state received

FY12 and FY13 grants of approximately $12 mil ion each.

Louisiana

Combined, CalHFA expects to assist 433 units with the

Louisiana’s PRA program builds on the state’s Permanent

two grants. CalHFA issued a NOFA and invited owners of

Supportive Housing (PSH) program developed in

existing multifamily properties, as well as properties under

response to the 2005 hurricanes in the Gulf region.

development, to apply for PRA subsidies.

The state received a FY12 grant of $8 mil ion, which is

The FY12 California PRA program targets persons

expected to fund 199 units. The PSH program targets

with disabilities who are eligible for Medicaid and who

people with disabilities who have supportive service

are either living in an institution and interested in living

needs. PRA partners are the Louisiana Housing

in the community or at risk of institutionalization. The

Corporation (LHC) and the Department of Health and

PRA program targets people in California Community

Hospitals. An Executive Management Council made up of

Transitions, the state’s MFP program, and others eligible

representatives from a number of state agencies oversees

for Medicaid services. The FY13 grant added people

the program. With the PRA grant, LHC expanded the PSH

experiencing homelessness as a target population.

program from the Gulf region to other parts of the state.

Two service agencies that are under California’s

PRA also helps address priorities in the state’s Ten-Year

13-agency DHCS are PRA partners: The Department of

Plan to End Homelessness and the terms of a class-

Developmental Disabilities and the Department of Health

action lawsuit settlement under Olmstead ( Barthelemy

Care Services that administers the state’s Medicaid and

v. Louisiana Department of Health and Hospitals14).

MFP programs.

Louisiana identified PRA units through a NOFA for existing

multifamily properties already funded with LIHTC or other

Delaware

housing subsidies.

Delaware’s Section 811 PRA program is administered by

Louisiana’s FY12 PRA grant targets people who are

the Delaware State Housing Authority (DHSA), partnering

inappropriately institutionalized, are homeless, or are

with the Delaware Department of Health and Social

at risk of institutionalization or homelessness. The PSH

Services (DHSS). The state received a FY12 grant of over

program office at LHC coordinates centralized outreach,

$5 mil ion for 148 PRA units. Delaware’s Section 811 PRA

with waitlists and referrals organized and tracked by

program built on Delaware’s State Rental Assistance

parishes. Units targeted for the PRA program are largely

Program (SRAP), which launched in 2011. SRAP provides

funded through the LIHTC program or HUD’s HOME

vouchers for people with low income who are especial y

program.

vulnerable to homelessness, such as youth exiting foster

care or those exiting long-term care institutions. DSHA

Maryland

identified PRA units by establishing points in their LIHTC

In Maryland, the PRA program partners are the

programs for properties to accept PRA units in new and

Departments of Housing and Community Development

rehabilitated properties.

for identifying properties for PRA units; Health and Mental

The target populations for Delaware’s PRA program

Hygiene for providing services to residents and applicants;

are, in order of priority, (1) the population covered under

and Developmental Disabilities for managing applications,

a 2011 Department of Justice Settlement Agreement13

waiting lists, and referrals to units. Maryland’s MFP

who are DHSS clients exiting the Delaware Psychiatric

program, housed in the Department of Health and Mental

Center and/or with serious and persistent mental il ness;

Hygiene, provides substantial staff support to the PRA

(2) DHSS clients exiting institutions, with an emphasis on

program. The state received a FY12 grant of $11 mil ion

MFP participants; and (3) people at risk of admission to

for 150 units and an FY13 grant of approximately $10

long-term care facilities. Four divisions in DHSS administer

mil ion for an additional 153 units. Maryland built its

the program for their respective populations: Division of

PRA program approach on two existing programs. The

Services for Aging and Adults with Physical Disabilities;

Bridge Program provides temporary tenant-based rental

Developmental Disabilities Division; Division of Medicaid

subsidies to assist low-income people with disabilities

and Medical Assistance (which administers the MFP

until they could obtain a housing choice voucher. The

program); and the Division of Substance Abuse and

Weinberg Apartments program provides rental housing

in tax credit properties to people with disabilities with

13 United States v. Delaware, Civil Action No. 11-591-LPS (July 6, 2011)

14 Barthelemy v. Louisiana Dept. of Health and Hospitals (U.S. Dist. Ct., E.D. La., No. 00-1083, Oct. 17, 2001) 11

H U D S e c t i o n 811 P R A P ro g r a m – P h a s e I I E v a l u a t i o n

Chapter 2. PRA Programs in the Six Study States

incomes between 15 and 30 percent of area median

The contractor administers the PRA rental assistance. The

income. The state housing agency identified PRA units in a

Washington Housing Finance Agency and the State Health

mix of existing multifamily properties and properties under

Care Authority (the state Medicaid agency) coordinate

development.

with Commerce and DSHS, but do not play active roles in

Maryland’s FY12 PRA program focused on the

PRA program management. Commerce issued a NOFA

Washington and Baltimore metropolitan areas. The FY13

and invited owners of existing multifamily properties, as

grant expands the program statewide. Maryland’s PRA

well as properties under development, to apply for PRA

target populations are, in order of priority, (1) people

subsidies. The state agencies also actively conducted

who are living in an institution, (2) people at risk of

outreach to property owners of multifamily properties

institutionalization, (3) Community Pathways Waiver and

that received capital subsidies through state affordable

Residential Rehabilitation program participants who are

housing programs.

transitioning to community-based settings and people

The PRA program fil s gaps in Washington’s existing

experiencing homelessness who are enrolled in Medicaid,

housing programs for people with disabilities and provides

and (4) people transitioning from group homes or assisted

a permanent housing option for people served in two

living facilities. The Department of Developmental

state-funded transitional housing programs. Washington’s

Disabilities maintains a combined waiting list, organized

PRA target populations are MFP participants and other

by priority and desired county of residence, for applicants

Medicaid waiver clients with developmental disabilities,

for PRA assistance, the Bridge Program, and Weinberg

mental il ness, or functional disabilities.

Apartments.

2.4 PRA Program Status in

Minnesota

the Study States

Minnesota’s PRA program is administered by the

Minnesota Housing Finance Agency (MN Housing) and

This section provides a brief update on the study states’

the state Medicaid agency, the Department of Human

progress with PRA implementation as of the end of

Services. The state received a FY12 grant of $3 mil ion

September 2018, which is the end of the third full PRA

for 85 PRA units and a FY13 grant for an additional $3

program fiscal year. The data come from the most

mil ion for PRA 75 units. The two agencies have a long

quarterly progress reports that grantees submitted to

history of working together. The PRA program built on

HUD. The quarterly reports update HUD on the number

the state’s MFP and supportive housing initiatives for

of units under contract with owners and under lease

people experiencing homelessness and serious mental

by PRA residents, as well as provide some summary

il ness. In addition, the state views the PRA program as

characteristics of PRA applicants, residents, and units

an important source of community-based housing to help

under contract.

the state respond to an Olmstead settlement. MN Housing

al ocated their FY12 units through a NOFA for owners of

Status of Units under Contract

existing multifamily properties and al ocated their FY13

and under Lease

units through incentives in their state LIHTC program for

As of September 2018, the study states had about

new construction or rehabilitated properties.

half of the total number of planned PRA units under

The Minnesota PRA program targets people experiencing

contract with owners for PRA subsidies. Al PRA

long-term homelessness and serious mental il ness

partnerships in the study states have made progress in

and people with mental il ness or physical disabilities

identifying properties and units, entering into contracts

who are exiting institutions and who are enrol ed in the

with property owners, and leasing PRA units to eligible

MFP program. The state views PRA units as a source of

people with disabilities, as shown in Exhibit 2.3. The

community-integrated housing to support members of the

exhibit compares each state’s progress to the number of

Jensen Settlement Agreement.15

units the grantee expects to assist with its FY12 and FY13

grants. Grantees in the six study states had identified 104

Washington

percent of their planned FY12 PRA units (Louisiana and

Maryland both identified more units than they expect to

In Washington, the Department of Commerce (Commerce)

fund), had 78 percent of planned units under contract with

and the Department of Social and Health Services (DSHS)

owners, and had 51 percent under lease by PRA residents.

are the key PRA program partners. Commerce received

For FY13 grants, 56 percent of planned units were

a FY12 grant of $5.7 mil ion and expects to subsidize

identified, 7 percent were under contract, and 3 percent

133 units. DSHS has three regional housing program

were under lease by PRA residents as of September 2018.

managers who coordinate housing referrals in the state.

15 https://mn.gov/dhs/general-public/featured-programs-initiatives/jensen-settlement/

12

H U D S e c t i o n 811 P R A P ro g r a m – P h a s e I I E v a l u a t i o n

Chapter 2. PRA Programs in the Six Study States

E xhibit 2.3 : S t atus of Planned 811 P R A Unit s in the S tudy S t ates, September 2018

FY12

FY13

% Units

% Units

# of Planned

% of Units

under

% Units

# of Planned

% of Units

Under

% Units

State

Units

Identified

Contract

Leased

Units

Identified

Contract

Leased

California

233

96%

55%

36%

200

17%

0%

0%

Delaware

148

82%

78%

42%

Louisiana

199

128%

128%

62%

Maryland

150

115%

49%

49%

150

86%

5%

4%

Minnesota

85

99%

99%

91%

75

104%

37%

13%

Washington

133

100%

65%

52%

Total

948

104%

78%

51%

425

56%

7%

3%

Source: Abt analysis of 811 PRA Quarterly Reports for the period ending September 30, 2018.

PRA Residents Assisted During 2015-2018

access affordable housing. Approximately 20 percent of

Between 2015 and September 2018, the study states

residents were experiencing homelessness, and another

had assisted 725 households. Grantees report that PRA

20 percent were at risk for homelessness. Six percent of

residents came from a variety of previous living situations

residents moved from a group home, adult care home, or

(Exhibit 2.4). More than a quarter of PRA residents (27

other residential setting, and the previous living situation

percent) moved directly from nursing facilities or other

was not reported for the remaining 14 percent of PRA

institutional care settings, and an additional 13 percent

residents.

were at risk for institutionalization if they cou

Exhibit 2.4: ld not

PRA-Assisted Residents in the Study States

by Previous Living Situation, September 2018

E xhibit 2.4 : P R A-A ssisted Resident s in the S tudy S t ates by Previous Living Situation, September 2018

Other, 14%

Institutionalized, 27%

Living in a Group

Home, Adult Care

Home, or Other

Residential Setting, 6%

At Risk of

Homelessness, 20%

At Risk of

Institutionalization, 13%

Experiencing

Homelessness, 20%

Note: Percentages may not add to 100 percent due to rounding.

Source: Abt analysis of 2015-2018 Section 811 PRA Quarterly Reports in six study states.

13

H U D S e c t i o n 811 P R A P ro g r a m – P h a s e I I E v a l u a t i o n

Chapter 2. PRA Programs in the Six Study States

Duration of PRA Resident Tenancy

E xhibit 2.5 : P R A Resident s E xiting Housing in the

S tudy S t ates, Program Year s 2015 –2018

Given the recency of the program funding and relatively

slow pace of PRA unit occupancy, most residents in

PRA-assisted units had not lived in their units for very long

Total

at the time of the evaluation. Among the 519 households

2015–2018

reported living in PRA units as of September 2018: 22

Leased residents at end of year

504

percent had moved into PRA units within the last 6

Number of residents exiting PRA units

193

months, and 23 percent had moved in between 7 and 12

Percent of leased residents exiting PRA units

19%

months ago. A third (33 percent) had moved in between 1

and 2 years ago, and 21 percent had lived in their units for

Note: Percentage of PRA residents exiting housing in the study states based

on the number of residents residing in PRA units at the end of the fiscal year.

more than 2 years.

Source: Abt analysis of 2015-2018 Section 811 PRA Quarterly Reports through

In the first 4 years of the program, almost one-fifth

the period ending September 30, 2018, in six study states.

of PRA residents exited the program in the study

states. Cumulatively, 193 PRA households (19 percent)

During the research period, these states continued

exited in the first 4 years of the program (Exhibit 2.5).

to outpace other states in housing PRA residents. In

While the number of exits increased over time, the

addition to providing a sizable sample for the evaluation’s

percent of residents who left each year declined as overal

outcomes analyses, these six states’ PRA programs

participation grew. The percent of residents exiting PRA

also present a variety of partnership structures,

units decreased from 23 percent in 2016 and 24 percent in

targeting approaches, outreach and housing placement

2017 to 17 percent in 2018.

strategies, and other program features. They may not be

The states selected for the Phase II Evaluation were

representative of the larger group of 27 states that are

chosen because they had made the most progress leasing

administering PRA grants, but they do reflect a diversity of

PRA units by spring 2017 when the study team finalized

program settings and approaches and can offer lessons

the research design.

for the program going forward.

14

H U D S e c t i o n 811 P R A P ro g r a m – P h a s e I I E v a l u a t i o n

Chapters 3-7: Early Evidence from the PRA Program

Chapters 3-7: Early

Evidence from the

PRA Program

As described in the fourth section of the first chapter, a

primary goal of the Phase II evaluation is to assess the

outcomes and effectiveness of the PRA program and

to compare to outcomes for similar populations living

in other housing settings. The third through seventh

chapters in this section review the early evidence from the

study.

The analyses are based on available individual-level

data from HUD, CMS, and state Medicaid agencies in

the six study states. Where data are available, the study

compared outcomes of PRA residents to outcomes of

residents in the four comparison groups described in the

chapter titled “PRAC, NED, other HUD, and non-HUD.”

The comparison groups are limited to households with

non-elderly people with disabilities. The PRA, PRAC, and

NED comparison groups include al individuals under age

65 in the six study states who receive rental assistance

through those programs. For the other HUD group, we

include all individuals under age 65 who report having

a disability and who receive assistance from the public

housing, HCV, and other HUD multifamily programs.

We supplement the administrative data analyses in this

section with data from our survey of Section 811 residents

in a sample of PRA and PRAC properties. We use the

survey results to compare how PRA residents and PRAC

residents rate their housing, neighborhood, community-

based services and supports, and overall health and

wel -being.

The first chapter in this section presents descriptive

analyses of the demographic characteristics of the study

groups. We then present results of the impact analyses

that compare outcomes for PRA residents and similar

individuals in the comparison groups in neighborhood

outcomes, property characteristics, and resident

experience with community-based services and housing

tenancy outcomes. The seventh chapter presents the

health status and healthcare utilization patterns across the

study’s comparison groups.

15

H U D S e c t i o n 811 P R A P ro g r a m – P h a s e I I E v a l u a t i o n

16

H U D S e c t i o n 811 P R A P ro g r a m – P h a s e I I E v a l u a t i o n

Chapter 3. Demographic Characteristics of PRA Residents and Comparison Groups Demographic

• PRA residents have fewer single-person households

than PRAC residents but more than NED and other-

Characteristics of

HUD programs.

• PRA residents are less likely to be white non-Hispanic,

are younger, and have lower incomes compared to

PRA Residents and

non-elderly people with disabilities served in other

HUD programs in the same states.

Comparison Groups

3.1 Household and Demographic

Characteristics

Household Size

In order to place the findings of the impact analysis in

PRA residents are less likely to be single-person

context, and to construct appropriate comparison groups,

households than PRAC residents, but more likely

the study team analyzed demographic data on PRA

than NED and other-HUD residents. The PRA program

program residents and other HUD program residents. This

assists households of varying sizes, although the majority

analysis also provides insight into who the PRA program

of PRA households are single-person households. The

is supporting, relative to other HUD programs that provide

average household size is 1.3 for PRA, 1.1 for PRAC, 1.9

assisted housing. The data come from several HUD

for NED, and 1.8 for other HUD. As shown in Exhibit 3.1, 78

databases.16

percent of PRA residents live alone, 15 percent live in two-

These demographic comparisons are intended to

person households, and 7 percent live in three-person or

help readers interpret the results of the differences

larger households. By contrast, most PRAC residents (92

in program outcomes presented in the subsequent

percent) live alone. More than half of NED (55 percent) and

chapters. Secondly, we use these demographic and

other-HUD assisted residents (61 percent) live alone. The

household characteristics (race, ethnicity, gender, age,

remaining NED and other-HUD assisted households are

any dependents, and income) to select individuals

approximately evenly distributed between two and three-

for comparison groups who are most similar to PRA

person (or larger) households.

residents. Constructing similar comparison groups means

Both PRA and PRAC residents lived in units with an

differences in program outcomes can be attributed to the

approximate average of 1.0 bedrooms, lower than

program (PRA, PRAC, NED, or other HUD), rather than to

for NED residents (1.9 bedrooms) and other HUD

individual characteristics.

residents (1.5 bedrooms). The distribution of bedroom

The study found numerous differences between PRA

sizes of the units mirrors the household sizes served

households and households in the comparison groups in

across these programs (see Exhibit 3.1).

the six study states:

E xhibit 3.1: Household and Bedroom Size of P R A , P R AC, N ED, and

O ther H U D -A ssisted Households in the S tudy S t ates, March 2018

Characteristics of HUD-

PRA

PRAC

NED

Other HUD-Assisted

Assisted Households

N

540

3,194

2,532

62,661

Household Size

1 Person Household

78%

92%

55%

61%

2 Person Household

15%

6%

22%

19%

3+ Person Household

7%

2%

23%

21%

Average Household Size

1.32

1.1

1.93

1.81

( cont )

16 The non-HUD comparison group does not receive HUD assistance; therefore, we do not have comparable demographic data on that group.

17

H U D S e c t i o n 811 P R A P ro g r a m – P h a s e I I E v a l u a t i o n

Chapter 3. Demographic Characteristics of PRA Residents and Comparison Groups E xhibit 3.1: Household and Bedroom Size of P R A , P R AC, N ED, and

O ther H U D -A ssisted Households in the S tudy S t ates, March 2018 ( cont )

Characteristics of HUD-

PRA

PRAC

NED

Other HUD-Assisted

Assisted Households

N

540

3,194

2,532

62,661

Bedroom Count

0 Bedroom

27%

7%

3%

12%

1 Bedroom

50%

85%

42%

43%

2+ Bedroom

23%

8%

34%

28%

3+ Bedroom a

21%

17%

Average No. of Bedrooms

0.97

1.04

1.86

1.53

a Three and four bedrooms combined with two bedrooms for PRA and PRAC only due to small sample reporting restrictions.

Source: Abt analysis of unweighted household data from HUD databases iREMS and PIC as of March 2018 in the six study states.

Age

residents in the comparison groups. As shown in Exhibit

Adults in PRA households are younger than residents

3.2, a higher proportion of PRA residents identified as

assisted by other HUD programs. On average, PRA

African American or black (45 percent) when compared to

residents are age 45, compared to 46 for PRAC, 47 for

PRAC residents (32 percent), NED recipients (31 percent),

NED and 50 for other HUD-assisted residents (Exhibit

and other HUD residents (43 percent). Similarly, less than

3.2). The PRA, PRAC, and NED programs require that

half of PRA residents identified as white non-Hispanic

the individual with a disability be age 61 or younger on

(42 percent), compared to 51 percent of PRAC residents,

their move-in date. In order to create an appropriate

50 percent of NED residents, and 41 percent of other

comparison population from the other HUD-assistance

HUD residents. PRA and PRAC residents are less likely

programs, we included in the sample only households

to identify as Hispanic (6 percent) than NED resident (13

with adult members with disabilities who are younger than

percent) and other HUD residents (12 percent).

age 65. Differences in age may be related to differences in

PRA programs assist slightly more women than men, 54

residents’ tenure as described in the next section.

percent vs. 46 percent. NED and other HUD-assisted

residents also have more women than men. The opposite

Race, Ethnicity, and Gender

is true for PRAC residents, who are 52 percent male.

A larger share of PRA residents are African-American and

a smal er share are non-Hispanic white or Hispanic than

E xhibit 3.2 : Demographic and S ocioeconomic Characteristics of P R A , P R AC, N ED,

and O ther H U D -A ssisted Households in the S tudy S t ates, March 2018

Characteristics of HUD-

PRA

PRAC

NED

Other HUD-Assisted

Assisted Households

N

540

3,194

2,532

62,661

Age of Individual w/ Disability

18-30

19%

16%

15%

10%

31-40

19%

19%

16%

15%

41-50

19%

19%

18%

17%

51-62

40%

38%

42%

43%

63+

3%

8%

9%

15%

Average Age

45

46

47

50

( cont )

18

H U D S e c t i o n 811 P R A P ro g r a m – P h a s e I I E v a l u a t i o n

Chapter 3. Demographic Characteristics of PRA Residents and Comparison Groups E xhibit 3.2 : Demographic and S ocioeconomic Characteristics of P R A , P R AC, N ED,

and O ther H U D -A ssisted Households in the S tudy S t ates, March 2018 ( cont )

Characteristics of HUD-

PRA

PRAC

NED

Other HUD-Assisted

Assisted Households

N

540

3,194

2,532

62,661

Race and Ethnicity

White/Caucasian (not Hispanic)

42%

51%

50%

41%

Black/African American

45%

32%

31%

43%

Asian

0%

3%

4%

3%

Hispanic

6%

6%

13%

12%

Other or Unknown

7%

8%

2%

1%

Gender

Female

54%

48%

59%

55%

Male

46%

52%

41%

45%

Note: Households could report more than one race or ethnicity so totals for race and ethnicity may not add to 100 percent.

Source: Abt analysis of unweighted household data from HUD databases iREMS and PIC as of March 2018 in the six study states.

Household Income

5 percent for PRA household and 6 percent for PRAC

PRA residents have lower average household incomes

households, compared to 1 percent for NED and other

than any of the other HUD-assisted groups. PRA

HUD households. The PRA program targets households

residents’ average unadjusted annual total income was

with lower household incomes than the other HUD

$8,578, compared with PRAC residents ($10,716), NED

comparison programs. To be eligible, PRA households

residents ($14,729), and other HUD residents ($14,205)

must have extremely low household incomes (no more

(see Exhibit 3.3). Average income is also affected by the

than 30 percent of AMI), compared to 50 percent of AMI

number of households that report zero income which is

for other HUD programs.

E xhibit 3.3 : Household Income of P R A , P R AC, N ED, and

O ther H U D -A ssisted Households in the S tudy S t ates, March 2018

Characteristics of HUD-

Assisted Households

PRA

PRAC

NED

Other HUD-Assisted

Annual Household Income

N

540

3,194

2,532

62,661

Average household income

$8,578

$10,716

$14,729

$14,205

Percentage of households that

5.4%

5.8%

0.9%

1.3%

report zero household income

Source: Abt analysis of unweighted household data from HUD databases TRACS and IMS/PIC as of March 31, 2018, in the six study states.

Differences in household income across programs

but lower than for NED (60 percent) and other HUD (59

may be related to a number of factors. NED and other

percent) households. Only a third of PRA households (34

HUD households are more likely to have more than one

percent) report Social Security disability or retirement

household member than PRA or PRAC households.

income, compared to 56 percent of PRAC households,

Average income is also affected by residents’ source of

57 percent of NED, and 49 percent of other HUD-assisted

income. More than half of PRA residents (54 percent)

households. Fewer PRA households have employment or

receive Supplemental Security Income (SSI). This

business income, 6 percent, compared to 15 percent for

percentage is higher than for PRAC residents (48 percent),

PRAC, 17 percent for other HUD, and 20 percent for NED.

19

H U D S e c t i o n 811 P R A P ro g r a m – P h a s e I I E v a l u a t i o n

Chapter 3. Demographic Characteristics of PRA Residents and Comparison Groups E xhibit 3.4 : S ources of Household Income of P R A , P R AC, N ED,

and O ther H U D -A ssisted Households in the S tudy S t ates, March 2018

Source of Income

PRA

PRAC

NED

Other HUD-Assisted

N

506

2,998

2,450

60,411

Percentage with Supplemental Security Income

54%

48%

60%

59%

Percentage with Social Security

34%

56%

57%

49%

Percentage with general assistance or welfare

15%

17%

20%

19%

Percentage with TANF

6%

2%

8%

10%

Percentage with any wage or own business income b

6%

15%

20%

17%

Percentage with unemployment insurance

0%

0%

1%

1%

Percentage with child support

<3%a

0%

6%

4%

Percentage with pension

0%

1%

3%

6%

Percentage with other non-wage income

4%

5%

8%

8%

a Percentage not shown due to small sample reporting restrictions.

b Percentage with any wage includes “PHA Wage,” “Federal Wage,” “Other Wage,” and “Business Income.”

Note: Households may have more than one source of income. Sample sizes differ from Exhibit 3.3 because of missing source of income information for some households.

Source: Abt analysis of unweighted household data from HUD databases TRACS and PIC as of March 31, 2018, in the six study states.

20

H U D S e c t i o n 811 P R A P ro g r a m – P h a s e I I E v a l u a t i o n

Chapter 4. Neighborhood Characteristics for PRA Residents and the Comparison Groups Neighborhood

The study found numerous statistical y significant

differences in certain characteristics of the neighborhoods

Characteristics for

where PRA residents live and where residents in the

comparison groups live. PRA residents are significantly

more likely to live in neighborhoods with higher

PRA Residents and

concentrations of poverty, lower rates of owner-occupied

units, higher residential density, lower education rates,

the Comparison

and higher percentages of people with a disability than

the neighborhoods where other HUD-assisted, non-

elderly people with disabilities live. While PRA residents

Groups

live in neighborhoods that have somewhat higher rates of

access to public transportation, they are also significantly

more likely to live in neighborhoods with higher exposure

to harmful environmental toxins than those in the PRAC,

NED, and other HUD-assisted groups

Prior research has documented that different housing

assistance approaches and programs can result in

4.1 Approach to Comparison of

differences in the characteristics of neighborhoods where

Neighborhood Characteristics

housing assistance recipients live (Galvez, 2010; Finkel et

al., 2016; McClure, 2008). The quality of life and health for

Using census tract-level data from the American

people with disabilities can be related to features of the

Community Survey (ACS), an annual update of the

properties where they live, but they may also be related to

decennial census, and geocoded data on properties

characteristics of the neighborhoods where they live. This

from HUD administrative data, the study team compared

section compares the characteristics of neighborhoods

average characteristics for neighborhoods where PRA

where PRA-assisted households live to the characteristics

residents and the residents in the study’s comparison

of the neighborhoods where similar people assisted by

groups live.

PRAC, NED, and other HUD programs live. We present

Exhibit 4.1 reports the number of residents in the

findings for four types of neighborhood characteristics:

neighborhood analysis sample, the number of properties

household demographics, share of individuals with

(for PRA and PRAC only), and the number of census tracts

a disability, education and income, and indicators of

in which they live, based on the census tracts provided in

livability.

the HUD data.17

E xhibit 4.1: Number of Households, Proper ties, and C ensus Tract s in

Neighborhood A nalysis, by S tudy Group in the S tudy S t ates, March 2018

Number of Section 811

Number of Census

Individuals in Neighborhood

Properties in Which

Tracts in Which

Comparison Group

Analysis Sample

Households Live

Households Live

PRA

540

58

66

PRAC

6,479

484

574

NED

8,823

--

1,838

Other HUD-assisted

177,434

--

6,803

All six study states

193,276

--

9,281

Note: There may be more census tracts than properties because both PRA and PRAC properties may have multiple locations under some circumstances.

Source: Abt analysis of TRACS and PIC household data from the quarter ending March 31, 2018.

17 A small number of individuals in the NED and other HUD comparison groups were dropped from this analysis because of missing values in the neighborhood data due to small-cell reporting restrictions in the American Community Survey.

21

H U D S e c t i o n 811 P R A P ro g r a m – P h a s e I I E v a l u a t i o n

Chapter 4. Neighborhood Characteristics for PRA Residents and the Comparison Groups 4.2 Neighborhood Demographic and

statewide average for the study states is 59 percent. See

Household Characteristics

Exhibit 4.2.

PRA residents also live in neighborhoods with more

Residential Density and

buildings with 50 or more units than residents in the

Urban/Rural Classification

comparison groups. On average, 22 percent of buildings

in PRA neighborhoods are that large, compared to only 8

The study compared two measures of residential density

percent for PRAC, 7 percent for NED, and 12 percent for

in census tracts where PRA residents and comparison

other HUD-assisted programs. Less than half of buildings

group members live: rates of single-family owner

in neighborhoods where PRA residents live are single-

occupancy and the percentage of buildings with more

family homes or other one-unit structures (45 percent),

than 50 units. These measures can be indicators of

compared to 60 percent for PRAC, 58 for NED, and 53 for

neighborhood stability. Single-family owner-occupants

other HUD-assisted residents.

have a longer average length of time in a property as

compared to renters, so neighborhoods with higher

As shown in Exhibit 4.2, PRA residents are more likely

percentages of single-family owners are likely to have

to live in urban areas than PRAC, NED, and other HUD-

more long-term residents. The number of units by building

assisted groups are. The U.S. Census categorizes census

size also helps us understand how many people live in

tracts or blocks as either urban or rural areas. Urban areas

each neighborhood.

are comprised of Urbanized Areas (UAs) that have more

than 50,000 people or Urban Clusters (UC) that have at

On average, PRA residents live in neighborhoods with

least 2,500 and less than 50,000 people. Rural Areas

a lower owner-occupancy rate than the comparison

encompass all areas not included in an urban area (UA or

groups. Just over a third of properties in PRA residents’

UC). Most PRA residents, 97 percent, live in Urbanized

neighborhoods (36 percent) are owner-occupied,

Areas compared with 82 percent of PRAC residents, 86

compared to almost half of properties (50 percent) in

percent of NED, and 80 percent of other HUD-assisted

PRAC residents’ neighborhoods, 46 percent for NED,

residents.

and 42 percent for other HUD-assisted programs. The

E xhibit 4.2 : Neighborhood Characteristics of H U D -A ssisted Households in S tudy Groups, March 2018

Other HUD-

Statewide

Neighborhood Characteristics

PRA Mean

PRAC Mean

NED Mean

Assisted Mean

Average

N

540

3,197

2,719

66,687

24,112,942

Owner Occupancy Rate of Housing

Percentage of properties that are owner occupied

36.2%

49.7%**

46.0%**

42.2%**

59.4%

Building Size

Percentage of buildings with 1 unit

44.8%

59.6%**

58.1%**

53.1%**

67.4%

Percentage of buildings with 2-9 units

13.1%

16.7%**

17.8%**

18.0%**

12.2%

Percentage of buildings with 10-49 units

18.2%

11.5%**

13.3%**

13.3%**

9.5%

Percentage of buildings with 50+ units

21.7%

8.0%**

6.8%**

12.0%**

6.1%

Urban/Rural Classification

Percentage of households living in Urbanized Areas

96.9%

81.6%**

86.2%**

80.3%**

80.7%

Percentage of households living in Urban Clusters

<3%a

<19% a **

8.5%**

14.9%**

7.9%

Percentage of households living in Rural Areas

<3%a

<1% a **

2.1%**

2.9%**

11.4%

a Exact percentages not shown due to small sample reporting restrictions.

**Indicates a statistically significant difference between PRA and the comparison group with a p-value of .05.

Note: Statewide averages provided for comparison.

Sources: Abt analysis of ACS Factfinder and Affirmatively Furthering Fair Housing Data; and TRACS and PIC household data from the quarter ending March 31, 2018, in six study states.

22

H U D S e c t i o n 811 P R A P ro g r a m – P h a s e I I E v a l u a t i o n

Chapter 4. Neighborhood Characteristics for PRA Residents and the Comparison Groups Percentage of Residents Who

residents live, but similar to the share for residents in the

Report a Disability

other HUD comparison group.

For all age groups, PRA residents live in

PRA residents live in neighborhoods with higher

neighborhoods where a higher share of the

percentages of households with incomes below the

population reports a disability, based on self-reported

poverty line. On average, 28 percent of households in

data from the American Community Survey.18 However,

neighborhoods where PRA residents live have incomes

some differences are small and not statistical y significant.

below the federal poverty level, which is a significantly

The largest difference was in the percentage of residents

higher share than for neighborhoods where the other

with disability age 35-64, who make up 22 percent of

study groups live.20 By contrast, the poverty rate is 18

residents in PRA neighborhoods, compared to 16 percent

percent in PRAC neighborhoods, 22 percent in NED

for PRAC, 18 percent for NED, and 20 percent for other

neighborhoods, and 25 percent in neighborhoods of other

HUD-assisted residents. (Most PRA residents also fal into

HUD-assisted residents.

the 35-64 age group.) See Exhibit 4.3.

Racially and Ethnically Concentrated Areas

PRA residents live in neighborhoods with relatively the

of Poverty Neighborhoods

same percentages of people with disabilities over age 64

and between 18 and 34 as in the comparison groups. The

One-third of PRA residents live in neighborhoods

ACS asks respondents a series of questions about six

identified as Racial y and Ethnical y Concentrated Areas

different types of disabilities.19 Respondents who report

of Poverty, significantly more than in the comparison

any one of the six disability types are considered to have

groups. Racial y/Ethnical y Concentrated Areas of Poverty

a disability.

(R/ECAP) have a non-white population of 50 percent or

more and have more than 40 percent of households living

Education and Income

below the poverty line. HUD’s Affirmatively Furthering

Based on ACS data, PRA residents live in

Fair Housing (AFFH) database defines R/ECAP as census

neighborhoods with a lower percentage of residents

tracts that have high concentrations of poverty with a

with an Associate degree or higher. PRA residents

minority of residents who are non-Hispanic white. Almost

live in neighborhoods where, on average, 46 percent of

a third of PRA residents (30 percent) live in R/ECAP areas,

the adult population has a high school diploma or less

compared to 8 percent for PRAC, 13 percent for NED, and

as their highest level of educational attainment. This is a

18 percent for the other HUD-assisted group. (See Exhibit

higher share than neighborhoods where PRAC and NED

4.3.)

E xhibit 4.3 : Neighborhood Demographic Characteristics of

H U D -A ssisted Households in S tudy Groups, March 2018

Other HUD-

Statewide

Neighborhood Characteristics

PRA Mean

PRAC Mean

NED Mean

Assisted Mean

Average

N

540

3,197

2,719

66,687

63,246,812

Population with Disabilities by Age

Percentage of residents with disabilities age 18-34

8.4%

6.8%**

8.3%

7.8%**

5.5%

Percentage of residents with disabilities age 35-64

21.9%

16.1%**

18.4%**

20.2%**

11.9%

Percentage of residents with disabilities over

5.7%

5.2%**

5.6%

5.6%

5.0%

age 64

Percentage of residents with cognitive disability,

8.9%

6.1%**

7.1%

7.4%**

4.6%

all ages

Percentage of residents with ambulatory disability,

10.5%

7.9%**

8.7%

9.1%**

6.3%

all ages

( cont )

18 Disabled population by age (ACS Factfinder).

19 The six types of disabilities in the ACS are: hearing difficulty, vision difficulty, cognitive difficulty, ambulatory difficulty, self-care difficulty, and independent living difficulty. Retrieved from https://www.census.gov/topics/health/disability/guidance/data-collection-acs.html

20 The federal poverty level is defined by the U.S. Department of Health and Human Services: for 2018 it is $12,140 for the first person and $4,320 for each additional person.

23

H U D S e c t i o n 811 P R A P ro g r a m – P h a s e I I E v a l u a t i o n

Chapter 4. Neighborhood Characteristics for PRA Residents and the Comparison Groups E xhibit 4.3 : Neighborhood Demographic Characteristics of

H U D -A ssisted Households in S tudy Groups, March 2018 ( cont )

Other HUD-

Statewide

Neighborhood Characteristics

PRA Mean

PRAC Mean

NED Mean

Assisted Mean

Average

Education Level of Residents

Percentage of adults age 25 and over with

53.8%

55.9%**

51.8%**

51.7%**

60.0%

associate degree or higher

Percentage of adults age 25 and over with high

81.5%

82.6%**

80.2%

80.0%**

83.4%

school diploma

Poverty Level

Percentage of household below the poverty line

28.2%

18.4%**

21.7%**

24.9%**

13.7%

Racially/Ethnically Concentrated Areas of Poverty

Percentage of properties in R/ECAP Areas

29.6%

7.9%**

12.7%**

17.8%**

4.4%

R/ECAP = Racially/Ethnically Concentrated Areas of Poverty

**Indicates a statistically significant difference between PRA and the comparison group with a p-value of .05.

Note: Statewide averages provided for comparison.

Sources: Abt analysis of ACS Factfinder and Affirmatively Furthering Fair Housing Data; and TRACS and PIC household data from the quarter ending March 31, 2018, in six study states.

Neighborhood Livability Indicators

“walkable.” Walkability is a measure used to characterize

Neighborhood livability may also affect quality of life

the relative ease of pedestrian travel in a census tract

for non-elderly people with disabilities. Using publicly

block group.23 A higher walkability score means that it is

available data at the census tract level from HUD’s AFFH

more likely people walk as a mode of travel in that census

database and from the federal Environmental Protection

block.

Agency, we compared measures of use of public

PRA residents live in neighborhoods with higher

transportation, walkability, and environmental quality in

rates of walkability than PRAC, NED, and other HUD-

neighborhoods where PRA residents and other HUD-

assisted residents. On average, the National Walk Index

assisted residents live.

Score is 12.6 for PRA neighborhoods, compared to 12.0

Using publicly available data from the U.S. Environmental

for PRAC, 10.5 for NED, and 11.6 for neighborhoods where

Protection Agency,21 we compared a transit trips

other HUD-assisted residents live.

index score that measures the likelihood residents in a

Environmental health is measured by the Environmental

neighborhood use public transportation. The score ranges

Health Index from HUD’s Affirmatively Furthering Fair

from 0 to 100, with a higher score representing better

Housing Data and Mapping Tool.24 The index measures

access to public transit. On average, PRA residents live

potential exposure to harmful toxins within a census

in neighborhoods with greater access to public transit

block. Scores range from 0 to 100. The higher the

than the comparison groups (Exhibit 4.4). The transit

score a census tract has, the less exposure to harmful

score is 68 for PRA—similar to those for PRAC (66) and

toxins, and the better the environmental quality. PRA

NED (67), but significantly higher than those for other-HUD

residents lived in neighborhoods with slightly, but

assisted residents (64).

statistically significantly, higher exposure to harmful

The National Walkability Index22 score ranges from 1 to

environmental toxins than those in the PRAC, NED

20, with higher scores for areas that are considered more

and other HUD-assisted groups. The Environmental

21 https://www.epa.gov/smartgrowth

22 Environmental Protection Agency. National Walkability Index. 2010-2012. Published December 23, 2015. https://www.epa.gov/smartgrowth/

smart-location-mapping

23 The Walkability Index is a composite index that characterizes every Census 2010 block group in the U.S. based on its relative walkability.

The rankings were determined at the block group level and have been aggregated to higher-level geographies by way of population-weighted block apportionment.

24 The Environmental Health Hazard Index summarizes potential exposure to harmful toxins at a neighborhood level. The index uses standardized EPA estimates of air quality carcinogenic, respiratory, and neurological hazards. https://www.hudexchange.info/resource/4867/affh-data-and-

mapping-tool/

24

H U D S e c t i o n 811 P R A P ro g r a m – P h a s e I I E v a l u a t i o n

Chapter 4. Neighborhood Characteristics for PRA Residents and the Comparison Groups E xhibit 4.4 : C omparison of Neighborhood Livabilit y Indicator s of

H U D -A ssisted Households in S tudy Groups, March 2018

Other HUD-

Statewide

Neighborhood Characteristics

PRA Mean

PRAC Mean

NED Mean

Assisted Mean

Average

N

540

3,197

2,719

66,687

63,246,812

Transit Measure

Transit Index Score

68.3

66.2

67.0

64.0**

61.8

Walkability

National Walk Index Score (1-20)

12.6

12.0**

10.5**

11.6**

10.5

Environmental Health

Environmental Hazard Index

33.3

37.7**

40.4**

38.2**

34.4

**Indicates a statistically significant difference between PRA and the comparison group with a p-value of .05.

Note: Statewide averages provided for comparison.

Sources: Abt analysis of ACS Factfinder and Affirmatively Furthering Fair Housing Data and analysis of TRACS and PIC household data from the quarter ending March 31, 2018.

Health Index for neighborhoods where PRA residents live

Differences in Survey Responses Between

is an average of 33.3, compared to 37.7 for PRAC, 40.4 for

Section 811 PRA and PRAC Residents

NED, and 38.2 for other HUD-assisted.

4.3 Access to Services and

Evaluators interviewed 403 PRA and PRAC residents

in their homes between January and May 2018. The

Transportation

two groups were selected to be relatively similar

in the length of time they had been assisted by

It is important for residents to be able to access the

Section 811 and also close geographical y. The

services they need to live an independent and full life.

survey results were adjusted and weighted to make

Staff from state agencies in five of the study states

the two groups similar based on demographic

discussed the value of being near transit and paratransit

and healthcare utilization characteristics. This

(transportation services that meet or exceed requirements

al ows us to compare differences between PRA

under the American Disabilities Act (ADA)). Service

residents and otherwise similar PRAC residents.

providers noted that, when transportation is not available,

Throughout the fourth through sixth chapters,

property location is a problem, limiting resident access to

the shaded text boxes compare resident survey

services and supports.

results for PRA and PRAC residents. Statistically

To measure how well residents are able to access services

significant findings at the 5 percent level are

and transportation in their neighborhoods, the study team

noted and highlighted in bold and italics.

surveyed PRA and PRAC residents about how long it

takes them to get places and whether they have problems

navigating their neighborhood. Residents were asked

how long it takes them to get to the nearest grocery store

and pharmacy and whether they often or sometimes have

trouble getting around their neighborhood.

25

H U D S e c t i o n 811 P R A P ro g r a m – P h a s e I I E v a l u a t i o n

Chapter 4. Neighborhood Characteristics for PRA Residents and the Comparison Groups Most residents said it takes them between 15 minutes

More PRAC residents said they have trouble getting

and 1 hour to get to the nearest grocery store or

around their neighborhood because it takes them

pharmacy.25 Few PRA residents reported that they

too long to get where they wanted to go. Nearly all

have trouble or sometimes have trouble (21 percent)

PRAC residents (94 percent) who said they have

getting around their neighborhood. Of those who

trouble getting around their neighborhood cited

reported they did have trouble getting around, the most

this as a reason, compared to 60 percent of PRA

common reasons were that they did not have enough

residents, a statistically significant difference.

money for transportation (69 percent), or it that it takes too

long (60 percent), that there is no public transportation in

More PRA residents cite the lack of public

their area (49 percent), and that the neighborhood is not

transportation as a reason they have trouble

physical y accessible enough for them (31 percent).

getting around their neighborhood (49 percent of

PRA residents compared to 36 percent of PRAC

residents). This difference is not statistical y

PRAC residents and PRA residents are about

significant, however. Similar percentages of PRA

as likely to report problems getting around their

and PRAC residents reported having trouble

neighborhood. Less than a quarter of residents

getting around their neighborhood because they

(21 percent of PRA residents and 23 percent of

did not have enough money for transportation (69

PRAC residents) said they sometimes have trouble

percent of PRA and 66 percent of PRAC), and

getting around their neighborhood (Exhibit 4.5).

because their neighborhood was not accessible

Among those who report issues, the reasons are

enough for them (31 percent for both groups).

somewhat different for PRA and PRAC residents.

E xhibit 4.5 : P R A and P R AC Neighborhood Accessibilit y Measures

Difference in

Measure

PRA N

PRAC N

PRA%

PRAC%

Means

Percentage of residents who report they have

trouble getting around their neighborhood at least

180

207

20.9%

23.4%

-2.6%

sometimes

Because it is not physically accessible enough

35

28

31.4%

31.3%

0.1%

for them

Because their neighborhood has no public

33

28

48.5%

36.2%

12.3%

transportation

Because it takes them too long to get to where

37

29

59.5%

93.8%

-34.4%**

they need to go

Because they do not have enough money for

35

26

68.6%

66.0%

2.6%

transportation

**Indicates a statistically significant difference between PRA and the comparison group with a p-value of .05.

Note: PRAC results were adjusted and weighted for multiple comparisons.

Source: Abt analysis of Section 811 resident survey results administered in six states between January and May 2018.

25 Many PRA residents noted that they do not need to travel to a pharmacy to obtain their medications because they receive prescription drugs through the mail.

26

H U D S e c t i o n 811 P R A P ro g r a m – P h a s e I I E v a l u a t i o n

Chapter 4. Neighborhood Characteristics for PRA Residents and the Comparison Groups 4.4 Resident Perception of

PRAC residents are more likely to report

Neighborhood

liking their current neighborhood (84 percent,

compared to 73 percent of PRA residents), but

We surveyed PRA residents about their perception

the difference is not statistical y significant.

of the neighborhoods where they live in terms of the

When compared to PRA residents, PRAC

neighborhood’s safety, accessibility, public transportation,

and access to services and amenities. We also asked

residents are significantly more likely to

residents to report their overall satisfaction with their

report feeling safe in their neighborhoods,

neighborhood.

87 percent, compared to 68 percent of PRA

The majority of PRA residents reported that they like

residents. (See Exhibit 4.6.)

the neighborhood where they currently live and feel

safe there. Almost three-fourths of PRA residents (73

While about a third of PRA and PRAC residents

percent) said they liked their neighborhood (Exhibit 4.6).

said they wanted to move at the time of the survey

An additional 13 percent of PRA residents reported that

(33.5 percent of PRA and 32.1 percent of PRAC

they sometimes liked the neighborhood where they live.

residents), their reasons were different. PRA

The majority of PRA residents (68 percent) reported that

residents were significantly more likely to want to

they felt safe in their neighborhood. Another 12 percent

move because they don’t feel safe or because their

of residents reported that they sometimes felt safe, and

unit was not well maintained or managed. PRAC

less than 10 percent reported they do not feel safe in their

residents were significantly more likely to want to

neighborhood.

move because they want to live by themselves.

E xhibit 4.6 : Section 811 P R A and P R AC Neighborhood Measures

Difference in

Measure

PRA N

PRAC N

PRA%

PRAC%

Means

Percent of residents who say they like their

180

207

72.8%

83.8%

-11.0%

neighborhood

Percent of residents who say they feel safe in their

182

206

67.6%

86.6%

-19.0%**

neighborhood

Percent of residents who say they want to move

185

206

33.5%

32.1%

1.4%

from their unit

Percentage of residents who want to move

104

106

a

because they don’t feel safe there

<10%a

10.4%**

Percentage of residents who want to move

because their building or unit is not well

104

106

a

<10%a

12.4%**

maintained or managed

Percentage of residents who want to move

104

106

because they would like to live by themselves

<10%a

a

-21.2%**

a Percentages not shown due to small sample reporting restrictions.

**Indicates a statistically significant difference between PRA and the comparison group with a p-value of .05.

Note: PRAC results were adjusted and weighted for multiple comparisons.

Source: Abt analysis of Section 811 resident survey results administered in six states between January and May 2018.

27

H U D S e c t i o n 811 P R A P ro g r a m – P h a s e I I E v a l u a t i o n

Chapter 4. Neighborhood Characteristics for PRA Residents and the Comparison Groups Service Provider Perceptions of

Louisiana, Maryland, and Washington) reported that some

Neighborhoods Where PRA Units Are

applicants may be reluctant to move away from their

Available

current families and support networks. In addition, some

The study team asked service providers who work with

applicants may fear or dislike moving from a small country

PRA applicants and residents and owners of properties

town or suburban area to a larger city. Service providers

where PRA residents live about their perceptions of the

report that these concerns have prompted at least some

neighborhoods where PRA units are located and whether

applicants to turn down an available unit.

the neighborhoods meet the needs of the state’s PRA

Service providers reported that some properties with PRA

target populations.

units are located in undesirable or high crime areas, are far

Service providers reported that the neighborhood factors

from services or shopping, or are not accessible to public

that are most important to PRA applicants are:

transit. Providers also noted that available PRA units did

not always align with the preferences of residents to be

• Being near current family and support networks

near family or to be in an urban area or not. Preferences in

• Having a choice between suburban and urban options

housing location are personal, so it is difficult to generalize

about how neighborhood features influence resident

• Being in a safe neighborhood

satisfaction.

• Having access to public transit and door-to-door

Staff from five state agencies also reported that

transit service

neighborhood safety was an issue for residents living

• Being close to community services, healthcare

in at least one of their properties. In two states, safety

services, and shopping

concerns have deterred residents from accepting

State agency staff reported a number of chal enges

an available unit or even from visiting the property to

finding properties in neighborhoods or metro areas that

view the unit. Interviewees made comments such as

were desirable to program applicants. State housing

“The neighborhoods aren’t great from a safety

agencies have a limited set of owners and properties with

perspective” and “Neighborhood safety is an issue,

which to partner. As a result, some residents move not

particularly in terms of drug activity” or that it is “not

only to different neighborhoods but also to different parts

a nice neighborhood.” These comments are generally

of the state entirely to find an affordable place to live.

associated with comments on high rates of drug activity

Such moves can interfere with a person’s existing support

and/or violence in the area.

network. Service providers from four states (Delaware,

28

H U D S e c t i o n 811 P R A P ro g r a m – P h a s e I I E v a l u a t i o n

Chapter 5. PRA and PRAC Property Characteristics

PRA and

average of 14 units. The majority of PRA units are in

larger developments of more than 50 units, in either

PRAC Property

walk-up or elevator buildings, while only 3 percent of

PRAC residents live in properties with more than 50

units.

Characteristics

• On average in the study states, PRA units make up 10

percent of units in properties that have PRA units. This

is well below the 25 percent cap but does not account

for other units occupied by people with disabilities

who are not PRA-assisted. Some properties have set-

PRA subsidies are awarded to state housing agencies that

asides under other housing subsidy programs.

are responsible for selecting properties where PRA can

be used. Properties must have capital funding from other

• The majority of PRA residents report they like where

sources and meet HUD’s physical quality standards. Units

they live and feel safe, but a third of surveyed residents

set aside for people with disabilities may not exceed 25

report wanting to move. The most common reasons

percent of all units in the property. Rents must not exceed

PRA residents reported for wanting to move were that

the greater of either the Fair Market Rent or Small Area

they would prefer to live in a different neighborhood,

Fair Market Rent26 and must be affordable to residents

their building is not well maintained or managed, or

earning up to 50 percent of area median income. Grantees

they do not feel safe.

were provided incentives in the FY12 and FY13 grant

• PRAC residents are significantly more likely than PRA

competitions if they would agree to contract for units with

residents to report that the condition of their property

rents below FMR.

is excel ent or good and significantly more likely to feel

Within these broad PRA program and rent requirements,

safe in their property. A third of PRAC residents also

state housing agencies have the flexibility to select

report they would like to move from their current home,

properties where PRA can be used. This chapter

but for different reasons.

describes the types of properties that were selected

• Approximately three-fourths of PRA and PRAC

for PRA residents to live in, how they compare to the

residents reported maintenance issues in their units

properties where Section 811 PRAC residents live,

since they moved in, but residents report most

and how PRA and PRAC residents rate the quality

problems have been resolved.

of their properties and satisfaction with them. Using

HUD administrative data, the study team reviewed

5.1 Section 811 Property

the characteristics of properties with PRA units under

Characteristics

agreement with owners and compared them to properties

where PRAC residents live.

Using HUD administrative data as of September 2018,

The study team compared property characteristics for

the study team compared the building characteristics of

PRA and PRAC residents only, and not for the NED and

properties where PRA units are under contract (including

other HUD groups. Property data was not available for

both units under lease by PRA residents and those that

properties housing people with tenant-based assistance

are not being assisted yet) and properties where PRAC

in the NED and the HCV program (HCVs are the majority

residents live. Properties with PRA assistance must have

of the other HUD comparison group). This chapter also

at least five units and comply with the 25 percent cap

draws on the study’s surveys of PRA and PRAC residents

on set-asides for people with disabilities, but otherwise,

about their experience and satisfaction with where they

there are no restrictions on, for example, building type or

live, to determine whether the two groups differ.27

maximum property size.

The study found that:

We expect to see differences in property characteristics

in PRAC properties, given PRAC program rules regarding

• Properties in the six study states where PRA and

property size and configuration. PRAC capital grants

PRAC residents live are very different on average.

may be used to build (1) group homes where residents

Properties with PRA units under contract have an

share kitchen and living areas, (2) independent living

average of 72 units, while PRAC properties have an

facilities where each unit has a kitchen and bath, and (3)

26 The Fair Market Rent is used as a standard for rents in the HCV and other affordable housing programs and is determined annually by HUD.

27 The survey results were regression-adjusted and propensity score weighted to ensure similar populations. For details on adjustments and weighting, see Appendix B.

29

H U D S e c t i o n 811 P R A P ro g r a m – P h a s e I I E v a l u a t i o n

Chapter 5. PRA and PRAC Property Characteristics

condominium units. Most PRAC properties have fewer

units, in either walk-up or elevator buildings.28 Some

than 25 units in a single location or, in some cases, across

50 percent of PRA units are in walk-up properties, and

two or more locations.

35 percent are in elevator buildings. Smal er shares of

units are in row house developments (4 percent), semi-

Building Type

detached buildings (4 percent), or townhouses (7 percent).

PRA units under contract with owners are heavily

concentrated in larger developments of more than 50

Exhibit 5.1: Section 811 PRA Building Types for Occupied

Units in the Study States, September 2018

E xhibit 5.1: Section 811 P R A Building Types for Occupied Unit s in the S tudy S t ates, September 2018

2-5 story

elevator, 17%

Detached, 0.33%

Walk-up, 50%

High rise

elevator, 18%

Row, Semi-detached,

or Townhouse, 15%

Note: Percentages may not add to 100 percent due to rounding.

Source: Abt analysis of iREMS data as of September 2018 in the six study states.

Property Size

(33 percent) have between 2 and 9 units. Only three PRAC

The average number of units per property with PRA

properties in the six study states have 50 or more units;

units under contract is 72. Almost three-fourths (74

these properties house 3 percent of all PRAC residents.

percent) of units under contract are in properties with

An additional 7 percent of PRAC residents are housed in

50 or more units, and 44 percent have between 26 and

properties with between 26 and 49 units.

50 units. The remaining 2 percent of PRA units are in

Also, more PRAC properties are scattered site

properties with fewer than 25 units. (See Exhibit 5.2.)

properties than is the case for PRA. Scattered

As expected, PRAC properties are smaller on average

site properties are those that have units “scattered”

than properties where PRA residents live. The PRAC

throughout multiple locations or properties. Eight percent

properties in our sample have 14 units on average. The

of PRAC properties are reported as being scattered site

majority of PRAC properties (67 percent) have between

properties, compared to just two PRA properties (<1

10 and 25 units, compared to just 4 percent for PRA

percent).

properties. Most of the remaining third of PRAC properties

28 Property characteristics data are from HUD’s iREMS system. Property data was not available for 140 of the 778 units under contract with owners as of September 2018. Data on building type was not available for most PRAC properties.

30

H U D S e c t i o n 811 P R A P ro g r a m – P h a s e I I E v a l u a t i o n

Chapter 5. PRA and PRAC Property Characteristics

E xhibit 5.2 : Size of Proper ties with P R A Unit s and P R AC Unit s in the S tudy S t ates, September 2018

No. of Units in Property

PRA Properties N

PRA Properties %

PRAC Properties N

PRAC Properties %

2-9 units

1

1%

155

33%

10-25 units

3

4%

317

67%

26-49 units

28

36%

2

<1%

50+ units

45

58%

3

<1%

Total

74

100%

477

100%

Missing Unit Size

4

0

Average size

72

14

% scattered-site

<1%

8%

Note: Percentages may not add to 100 percent due to rounding.

Source: Abt analysis of iREMS data as of September 2018 in six study states.

Age and Development Status of

rehabilitated since 2010, and an additional 30 percent of

PRA Units under Contract

residents live in properties built or rehabilitated between

Most PRA residents live in properties that have been

2000 and 2009. Only one property with 11 PRA units was

constructed or substantially rehabilitated in the last

built in the 1970s, and two properties with a combined 13

20 years, but some residents live in older properties

PRA units were built in the 1980s.

built in the 1970s and 1980s. For 78 properties under

The units under contract as of September 2018 represent

contract with owners for PRA as of September 2018,

approximately a third of the total number of units that the

almost two-thirds (62 percent) were constructed or

study states expect to assist with their 2012 and 2013

substantial y rehabilitated since 2010 (Exhibit 5.3). Most

grants. When all the estimated units are ful y leased, the

PRA residents (85 percent) live in these properties:

vast majority of PRA residents will live in properties that

55 percent of PRA residents live in properties built or

are newly constructed or substantial y rehabilitated.

E xhibit 5.3 : Year of C onstruction or L ast Rehabilit ation for P R A

and P R AC Proper ties in the S tudy S t ates, September 2018

PRA

# of PRA

% of PRA

PRAC

PRAC

% of PRAC

Year Constructed or

PRA

Properties

Units in

Units in

PRAC

Properties

Units in

Units in

Substantially Rehabilitated

Properties N

%

Properties

Properties

Properties N

%

Properties

Properties

1970-1979

1

1%

11

2%

0

0%

0

0%

1980-1989

2

3%

13

2%

1

<1%

12

<1%

1990-1999

8

10%

74

12%

190

40%

2,412

47%

2000-2009

18

23%

189

30%

242

51%

2,726

53%

2010-2017

48

62%

345

55%

43

9%

13

<1%

Total

77

100%

632

100%

476

100%

5163

100%

Missing

1

1

1

19

Note: Percentages may not add to 100 percent due to rounding.

Source: Abt analysis of iREMS data as of September 2018 in six study states.

For-Profit Status of Owner

By contrast, owners or sponsors of Section 811 PRAC

More than two-thirds of owners with properties receiving

properties are required to be nonprofit organizations,

PRA units are for-profit entities, representing 73 percent

and 100 percent of PRAC residents live in properties with

of all PRA units under contract as of September 2018.

nonprofit owners.

31

H U D S e c t i o n 811 P R A P ro g r a m – P h a s e I I E v a l u a t i o n

Chapter 5. PRA and PRAC Property Characteristics

5.2 Residents’ Satisfaction

Similar percentages of PRA and PRAC residents

with Their Housing

report that they like where they live. Some 76

percent of PRA residents and 80 percent of PRAC

The 2018 survey of PRA and PRAC residents asked

residents report that they like where they live (Exhibit

respondents about their satisfaction with where they live

5.4). PRAC residents have more-favorable opinions

and how they rate the quality of their property.

regarding the quality and safety of their homes

compared to PRA residents, however. Significantly

The majority of PRA residents reported that they

more PRAC residents reported feeling safe living

liked where they live (76 percent) and felt safe in their

in their building than PRA residents (92 percent,

building (77 percent). Some PRA residents reported that

compared to 77 percent of PRA residents.)

they wanted to move, however (34 percent). The most

common reasons PRA residents reported for wanting

While about one-third of PRAC residents (32

to move were that they would prefer to live in a different

percent, or 66 residents surveyed) report they

neighborhood (41 percent of residents who said they

wanted to move, the reasons stated are different

want to move), their building was not well maintained

from those reported by PRA residents. PRAC

or managed (20 percent), they did not feel safe in their

residents report they want to move because

building (17 percent), and they would like to live closer to

they would prefer to live alone in their own

family or friends (14 percent). Other reasons were that they

apartment (39 percent of PRAC residents

preferred to live in a different type of building, have better

who want to move), would prefer a different

access to public transportation, or pay less-expensive

neighborhood (29 percent), or would like to

rent.

live closer to family or friends (20 percent).

E xhibit 5.4 : C omparison of Section 811 P R A and P R AC Resident S atisfaction with Housing

Difference in

Percentage of Residents Who Report That

PRA N

PRAC N

PRA %

PRAC %

Means

They like the place where they live now

184

207

75.5%

79.8%

-4.2%

The condition of the place is excellent or good

187

208

69.0%

82.8%

-13.9%**

They feel safe living in their building

187

204

76.5%

91.7%

-15.2%**

They want to move from their place of residence

185

206

33.5%

32.1%

1.4%

**Indicates a statistically significant difference between PRA and the comparison group with a p-value of .05.

Note: PRAC results were adjusted and weighted for multiple comparisons.

Source: Abt analysis of Section 811 resident survey results administered in six states between January and May 2018 in six study states.

5.3 Physical Quality of PRA Units

UPCS inspections are not required prior to grantees

executing contracts with property owners, although al

The Section 811 PRA program includes requirements

states report trying to visit the property prior to entering

and processes that help to ensure that units under lease

into an agreement with the owner, if feasible. In addition,

for PRA residents meet a certain standard of physical

as part of the PRA leasing requirements, all new residents

quality. HUD requires property inspections every 3 years

(sometimes accompanied by their case managers)

according to Uniform Physical Condition Standards

conduct a visual inspection and sign off on a checklist

(UPCS) that require properties to be decent, safe,

prior to moving in their unit. In two of the study states,

sanitary, and in good repair. In practice, study states

state agency staff conduct visual inspections of al new

report conducting inspections more frequently than

PRA units prior to new residents moving in.

HUD’s 3-year requirement, usual y because other funding

sources (such as LIHTC) require it. Five of the six study

Inspection results for properties with PRA units were not

grantees conduct formal inspections annual y, and one

available to the research team for this study, but we did

(California) conducts inspections every 2 years.

ask state agencies, program partners, and residents for

their perceptions about housing quality.

32

H U D S e c t i o n 811 P R A P ro g r a m – P h a s e I I E v a l u a t i o n

Chapter 5. PRA and PRAC Property Characteristics

PRA Resident Views on the Physical Quality

“excel ent” (35 percent) or “good” (35 percent) condition.

of Their Housing

The remainder said their unit was in “okay” (18 percent), or

Residents’ views on the quality of their PRA housing

either “not so good” or “bad” (12 percent) condition. (See

are mixed. More than two-thirds (70 percent) of PRA

Exhibit 5.5.)

residents surveyed reported that their unit was either in

E xhibit 5.5 : S

Exhibit 5.5: ection 811 PRA Resident Report of Pr

Section 811 PRA Resident Report of Pr operty Condition

operty Condition

Not so good

or Bad, 12%

Excellent, 35%

Okay, 18%

More PRAC residents

reported that the

condition of their unit

was excellent or good

(83 percent) than PRA

residents (70 percent),

a difference that is

statistically significant.

Good, 35%

Note: N=193 PRA residents.

Source: Abt analysis of Section 811 resident survey results administered in six states between January and May 2018.

Maintenance Issues Since Resident

While the majority of residents report one or more

Moved into Unit

maintenance issues, most residents also report that their

Three-fourths of PRA residents reported at least one

maintenance issues were not persistent and had been

maintenance issue in their units since they moved into

resolved by property management and maintenance staff.

their apartments (Exhibit 5.6). PRA residents reported

At the same time, more than a fifth of PRA residents (21

problems in their unit with pests (28 percent of residents),

percent) reported unresolved maintenance issues. The

plumbing (25 percent), broken air conditioning or heating (24

most common unresolved issue was with pests; 20 percent

percent), or broken appliances (17 percent).

of PRA residents reporting persistent issues with these.

33

H U D S e c t i o n 811 P R A P ro g r a m – P h a s e I I E v a l u a t i o n

Chapter 5. PRA and PRAC Property Characteristics

E xhibit 5.6 : Unres

Exhibit 5.6: olv

Unr ed and Res olved M aintenance I s sues Repor ted by P R A R

esolved and Resolved Maintenance Issues Reported by PRA Residentsesident s

Any maintenance issue since move in

21%

54%

Persistent issues with broken

13%

11%

air conditioning or heating

Unresolved (%)

Persistent problems with broken appliances

8%

9%

Resolved (%)

Persistent problems with pests

20%

8%

Persistent problems with plumbing

10%

15%

0%

10%

20%

30%

40%

50%

60%

70%

80%

Note: N = 193 PRA residents.

Source: Abt analysis of Section 811 resident survey results administered in six states between January and May 2018.

Similar percentages of PRA and PRAC residents

In contrast, some service providers and state agency staff

report having had at least one maintenance issue

described wel - maintained properties. One was described

since they moved in, but fewer PRAC residents

as fol ows: “[Property] is wonderful. Furnishings were

report persistent or unresolved maintenance

supplied for the units. The property is very pleasing

issues. Compared to PRA residents, fewer PRAC

visually. Lots of windows, light and airy . . . It is

residents report persistent problems with broken

impressive—quality of furnishings is high. There is a

appliances (33 percentage points less), broken

sense of dignity for people to move into a place of that

air conditioning or heating (14 percentage points

caliber.”

less), and plumbing (13 percentage points less).

PRAC residents reported slightly more unresolved

5.4 Unit Accessibility Features and

issues with their apartments overall (2 percentage

Needs of Residents

points more), and more unresolved issues with

pests (4 percentage points) than PRA residents.

Given the target populations of the PRA program, it is

likely that some PRA residents will need accessibility

features for people with disabilities in their units. HUD asks

Program Partner Perceptions of the Quality

PRA grantees to report the number of accessible units

of PRA Units

under contract for PRA, but because PRA units can “float”

Staff from the state housing agencies and service provider

within a property, owners cannot typical y identify what

organizations we interviewed reported that PRA units

accessibility features are available until a unit is vacant

rarely fail inspection but that the quality of the program

and available for lease to a PRA resident. For this reason,

ranges widely even within states. Properties range from

accessibility features are unknown for 448 units, or 58

newly built developments with extensive amenities to older

percent of the 778 units under contract in the six study

properties with maintenance and accessibility problems,

states.29

such as persistent vermin infestations or broken elevators.

Of the units under contract for which accessibility is

According to one service provider who works with PRA

reported, 19 percent have accessibility features for

applicants looking for housing, “Some properties are

residents. According to PRA grantee reported data, the

better than others. Some I wouldn’t want to live in.” In

majority of reported accessible units have accessibility

three states, interview respondents indicated there were

features for people with mobility impairments (13 percent).

one or more properties with serious maintenance issues.

Only 2 percent of units under contract are reported to

29 In their quarterly PRA reports, grantees report the share of accessible units in the units under contract, but missing data rates are high, so these data should be viewed with caution. Further, while owners may have agreed to make accessible units available, they do not commit to lease specific units. Rather, units “float” within the development.

34

H U D S e c t i o n 811 P R A P ro g r a m – P h a s e I I E v a l u a t i o n

Chapter 5. PRA and PRAC Property Characteristics

have accessibility features for hearing impairments, 1

issues with completing reasonable accommodation

percent for visual impairments, and 3 percent for some

requests for their units, and most indicated that the

combination of impairments.

requested accommodations were provided.

Reported Accessibility Needs

Since the study team interviewed only Section 811

of PRA Residents

residents, and not applicants, we cannot speak to the

accessibility needs of those who may be eligible for PRA

State agency and service provider staff report that

units but who have not received assistance yet. Property

the smal number of available accessible units has not

owners and service providers reported that, while some

prevented the match of PRA applicants to a unit that

applicants have turned down units because the units

meets their needs. According to state agency staff, few

did not meet their accessibility needs, the issue is not

PRA applicants have reported needing ful y accessible

widespread.

units.

While most PRA residents may not need ful y accessible

Residents we surveyed in PRAC properties

units, they may request modifications as reasonable

were significantly less likely than PRA

accommodation after they move in. In resident surveys

residents to report that they needed changes

conducted for this study, about a quarter of respondents

to their units or asked for special equipment.

living in PRA units (24 percent) said they had requested

More than a third of PRA residents (35 percent)

modifications to their units, and 20 percent said they

asked for changes or special equipment,

had asked for special equipment (see Exhibit 5.7). Most

compared to 16.4 percent of PRAC residents.

requests were minor, such as instal ing grab bars, lowering

High proportions of both groups reported they

countertops, accommodating a service animal, or moving

received the accommodations they asked for (81

a client to a first-floor unit. Interviewees did not report

percent for PRA, and 99 percent for PRAC).

E xhibit 5.7: Repor ted Accessibilit y Needs of Section 811 P R A and P R AC Resident s

Difference in

Residents Who Reported That:

PRA N

PRAC N

PRA %

PRAC %

Means

They have either needed changes to their building or requested

186

205

34.8%

16.4%

18.4%**

special equipment

They reported needing a change or special equipment and their

case manager or property manager made the changes they

32

19

81.3%

98.6%

-17.3%

requested sometimes or all the time

**Indicates a statistically significant difference between PRA and the comparison group with a p-value of .05.

Note: PRAC results were adjusted and weighted for multiple comparisons.

Source: Abt analysis of Section 811 resident survey results administered in six states between January and May 2018.

5.5 Integration of PRA

grantee or the funding source for the property, and the

Units within Properties

owner’s decision about how many units to set aside for

PRA subsidies within the limits set by the program.

A goal of the PRA program is to house people with

Through September 2018, state housing agencies entered

disabilities in properties where people with and without

into contracts with owners for an average of 10 PRA units

disabilities live. PRA grant funds provide rental subsidies

per property. On average for all properties with PRA

in properties with no more than 25 percent of their units

units under contract, PRA units make up 10 percent

set aside for PRA or people with disabilities served by

of units, considerably less than the PRA program’s

other housing programs. Beyond the 25 percent cap,

25-percent limit.

the proportion of PRA units in a given property reflects

The number of PRA units under contract with owners is

several factors. These factors include the number of other

the maximum number of units that, based on program

(non-PRA) supportive housing units in the property, other

rules and the owner’s preference, can be assisted by

property requirements or restrictions established by the

35

H U D S e c t i o n 811 P R A P ro g r a m – P h a s e I I E v a l u a t i o n

Chapter 5. PRA and PRAC Property Characteristics

PRA funding at that property. The actual number of units

The study found significant differences in

occupied by PRA residents could be less than the number

reports of community integration for PRA and

under contract for a few reasons. Not all units under

PRAC residents. PRAC residents were also

contract may be available for PRA residents because they

significantly more likely to report knowing

are still leased by other non-PRA tenants. Applicants may

others in their neighborhood (65 percent)

also lack interest in specific properties, resulting in fewer

compared to PRA residents (38 percent).

lease-ups to PRA residents.

(See Exhibit 5.8.) How well residents know

Setting limits on the percentage of units set aside

their neighbors may be a reflection of how long

specifical y for people with disabilities does not

residents have lived in their neighborhood.

necessarily mean that the percentage of people with

PRAC survey respondents were selected to be

disabilities living in these properties is limited to 25

relatively similar to PRA residents as to how

percent. According to HUD’s Picture of Assisted Housing

long they had lived in their unit when they were

data,30 an estimated 21 percent of HUD-assisted

surveyed, but we did not adjust the survey

households include someone with some type of disabling

results for length of tenure.

condition, making it likely that properties will have other

individuals with disabilities besides PRA residents living

PRAC residents were significantly more

in them. A few property owners interviewed by the study

likely to say they could see their friends

team estimated that the percentage of residents living in

and family when they wanted to see them

their buildings with physical or mental health disabilities

than PRA residents were, 88 percent of

was closer to 50 percent.

PRAC residents compared to 72 percent

of PRA residents, a statistically significant

Resident Perspectives on Community

difference. Compared to PRA residents,

Integration and Independence

slightly fewer PRAC residents said they

could be alone when they wanted to be (88.5

To measure how well Section 811 residents are integrated

percent compared to 95 percent of PRA

into their communities, the survey asked residents whether

residents) or eat when they wanted to (95

they knew and talked to other people in their building and

percent of PRAC residents compared to 99.5

neighborhoods. Most PRA residents (82 percent) said they

percent of PRA residents). These differences

knew other people in their building. While only 38 percent

were not statistical y significant, however.

of PRA residents say they know the other people in their

neighborhood, 68 percent of PRA residents say they have

spoken to other people in their neighborhood more than

twice in the past month.

The survey also asked residents about their autonomy

and independence in personal life choices. Most PRA

residents (95 percent) said they could be alone when they

wanted to be, and almost all (99.5 percent) said they could

eat when they wanted to. The majority of PRA residents

(72 percent) said they could see family and friends when

they wanted to see them.

30 https://www.huduser.gov/portal/datasets/assthsg.html

36

H U D S e c t i o n 811 P R A P ro g r a m – P h a s e I I E v a l u a t i o n

Chapter 5. PRA and PRAC Property Characteristics

E xhibit 5.8 : Section 811 C ommunit y Integration Measures

Exhibit 5.8: Section 811 Community Integration Measures

Impact:

Percentage of Residents Who Know

82%

-10.4**

Others in Their Building

93%

38%

-27.0**

PRA

Percentage of Residents Who Know

Others in Their Neighborhood

65%

PRAC

-10.5**

Percentage of Residents Who Have

67.8%

Talked to People in Their Neighborhood

More Than Twice Past Month

78.3%

0%

10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

% of Residents

**Indicates a statistically significant difference between PRA and the comparison group with a p-value of .05.

Note: PRAC results were adjusted and weighted for multiple comparisons.

Source: Abt analysis of Section 811 resident survey results administered in six states between January and May 2018.

37

H U D S e c t i o n 811 P R A P ro g r a m – P h a s e I I E v a l u a t i o n

38

H U D S e c t i o n 811 P R A P ro g r a m – P h a s e I I E v a l u a t i o n

Chapter 6. PRA and PRAC Community-Based Services

PRA and PRAC

status. PRAC residents were selected to participate in the

survey based on length of time in housing assistance and,

Community-

to ensure that they were in areas with similar access to

services, on geographic proximity.

Overal , we learned that:

Based Services

• The majority of PRA and PRAC residents report

receiving tenancy supports and other services that

meet their needs. Overal , they report a positive

experience with their home-based care, but some

The Section 811 program, including both PRA and PRAC

report gaps in services.

approaches, is a supportive housing program that seeks

• PRA and PRAC residents report no significant

to ensure residents have access to high-quality affordable

differences in ongoing tenancy supports and

housing and the community-based services and supports

community-based services received, the amount of

they need to be successful tenants in the community.

care they receive from friends and family, and their

PRA residents are required to be eligible for Medicaid or

quality of care by service providers.

state programs that provide home and community-based

• Most PRA and PRAC residents rate their quality of life

services (HCBS). As a result, PRA residents should have

and overall health as at least “okay,” but significantly

access to voluntary services and supports that assist

more PRAC residents than PRA residents rate

them as they move into and get settled in their new

their quality of life and overall health as “good” or

homes. PRA residents should also have access to ongoing

“excel ent.”

support services to help residents stay in their apartments

for as long as they want to live there. As discussed in

• PRA and PRAC residents have similar rates of exit, but

the first chapter, HUD’s other housing programs for

PRA residents are significantly more likely than PRAC

non-elderly people with disabilities also offer housing

residents to leave for owner-initiated reasons.

affordable to low-income renters, but residents housed

in other settings may or may not have ensured access to

6.1 Availability and Use of

community-based services and supports.

Community-Based Services

This chapter focuses on the community-based services

Tenancy supports encompass a wide range of services

that support residents in their homes. These services

to help people who have been living in an institution

include both tenancy supports that help PRA residents

or experiencing homelessness to transition to and live

apply for, move to, and remain stably housed in

independently in the community. People with disabilities

community-based housing and other community-based

or chronic conditions may need assistance in finding

services that ensure residents’ health and wel being (for

affordable community-based housing and moving into

example, personal care assistance, home healthcare, or

and establishing their new home. They may also need

transportation assistance).

personal care, behavioral health support, or other ongoing

To learn more about the differences between PRA

arrangements to support their daily lives. PRA grant funds

and PRAC, the study team interviewed staff from PRA

cannot be used to pay for services. Services are general y

partnering agencies in the six study states in late 2017 and

funded by Medicaid and provided through Medicaid

early 2018. The study team also conducted interviews with

waivers or programs. To ensure residents have the home-

a sample of service providers who assist PRA residents

and community-based services they need, the state

as they move into their homes and who provide ongoing

housing agency must partner with the agency responsible

community-based services, as well as with a sample of

for administering Medicaid or other healthcare programs

property managers at properties where units are under

in the state to apply for PRA grant funds.

contract for PRA. Final y, the study team surveyed a

The community-based services available to PRA residents

sample of PRA and PRAC residents living in the study

can be characterized in three phases: (1) housing locator

states in 201831 to learn about their tenancy experiences,

services to help people find affordable community

including access to tenancy supports and other services,

housing options; (2) transition services as they move

and their assessments of their quality of life and health

31 The study team conducted surveys with 403 Section 811 PRA and PRAC residents between January and May 2018 in six states: California, Delaware, Louisiana, Maryland, Minnesota, and Washington. PRAC residents were selected based on how long they had lived in their units, to be similar to PRA residents, who have a shorter average tenancy.

39

H U D S e c t i o n 811 P R A P ro g r a m – P h a s e I I E v a l u a t i o n

Chapter 6. PRA and PRAC Community-Based Services

into community housing; and (3) ongoing personal and

6.1 shows examples of the types of services offered within

community supports that help people live independently

each phase.

in the community. Community-based services initially

offered in one phase may continue into the next. Exhibit

E xhibit 6.1: T he T hree Phases of C ommunit y-based Ser vices

• Help individuals understand and locate affordable housing in their community

• Assist with transportation to view housing options and neighborhoods

• Help individuals complete and submit housing applications

• Help individuals meet housing eligibility requirements

ousing H ervices

- Obtain proper identification and proof of income or benefits

tor S

- Settle outstanding utility bills

- Correct credit reports to meet credit standards

Types of

Loca

- Helping provide explanations for or expunge criminal records

• Help individuals appeal rejections by landlords

• Help individuals pay security deposit requirements

• Assist individuals in procuring furniture or household goods

ces

• Assist with the physical move to the new property, including packing, transportation, and unpacking vier

• Accompany individuals on their move-in day

• Assist with move-in paperwork, inspections, and obtaining keys to the building, unit, and/or mailbox

• Ensure needed medical equipment is delivered and set up prior to move-in

ition S

• Set up community services

rans

- Establish a healthcare provider

f T

- Transfer prescriptions to a community pharmacy

o

- Apply for food and utility assistance benefits

- Set up meal delivery

Types

• Orient individuals to a new neighborhood

• Ongoing Tenancy Supports

- Assistance with establishing financial services, such as a checking account

- Assistance with developing and following a household budget

- Prompts to follow community rules

ts

- Linkages to social supports that foster a sense of belonging

or

- Assistance with applications for utility or food assistance

- Mediation to resolve disputes with neighbors or property management

- Assistance with annual recertification of income and eligibility

• Other Community-based Services

ngoing Supp

- Nursing care

f O

- Home health aides

- Assistance with eating, bathing, or dressing

- Medication reminders

Types o

- Durable medical equipment

- Homemaker services

- Meal preparation

- Supported employment

- Transportation

Source: Abt Associates

40

H U D S e c t i o n 811 P R A P ro g r a m – P h a s e I I E v a l u a t i o n

Chapter 6. PRA and PRAC Community-Based Services

Housing Locator Services

The majority of PRA and PRAC residents surveyed

Housing locator services help individuals find and

reported they received assistance from community-

apply for affordable community housing. Housing

based housing locator service providers in

locator service providers take people to view available

identifying and applying for affordable housing.

apartments, help them complete rental applications

Compared to PRAC residents, significantly

and gather necessary documentation to support the

more PRA residents report receiving support

applications, and help applicants access funds to pay

with applying to housing. While 77 percent of

security deposits.

PRA residents reported receiving help with their

housing application to move into their current

The intensity of the housing locator assistance varies

unit, only 59 percent of PRAC residents reported

by state. For example, in Delaware, qualified nursing

they received the same assistance. See Exhibit

home residents who express an interest in moving to the

6.2.

community are mailed a notice from the grantee when

a PRA unit becomes available. It is then the resident’s

Fewer than half of both PRA and PRAC residents

responsibility to visit the apartment and complete

report they looked at least one other apartment than

an application. In contrast, caseworkers in Louisiana

where they currently live in their housing search:

accompanied potential tenants to visit properties and

44 percent of PRA residents and 46 percent of

helped them complete applications. In some states,

PRAC residents said they looked at more than one

property managers reported they sometimes assisted

available unit. Of those who looked at more than one

PRA applicants with completing an application if no case

available apartment, 90 percent of PRA residents

manager was present.

and 91 percent of PRAC residents said they looked

at apartments in more than one neighborhood.

E xhibit 6.2 : Use of Tenancy Suppor t s by P R A and P R AC Resident s

Exhibit 6.2: Use of Tenancy Supports by PRA and PRAC Residents

Received Help

77%

on Their Lease Application

59%

44%

PRA

Received Information

on How to Live on Their Own

34%

PRAC

Received Help

70%

Getting Settled into Their Home

69%

0%

20%

40%

60%

80%

100%

% of Residents

**Indicates a statistically significant difference between PRA and the comparison group with a p-value of .05.

Note: PRAC results were adjusted and weighted for multiple comparisons.

Source: Abt analysis of Section 811 resident survey results administered in six states between January and May 2018.

41

H U D S e c t i o n 811 P R A P ro g r a m – P h a s e I I E v a l u a t i o n

Chapter 6. PRA and PRAC Community-Based Services

Transition Services

(44 percent) reported receiving information on

Transition services begin once an applicant is

how to live on their own when moving into their

approved for a specific PRA unit. Transition services

unit than PRAC residents (34 percent), but this

are those related to the physical move into a PRA

difference is not statistical y significant (Exhibit 6.2).

apartment, which can include obtaining household

goods, moving and unpacking, and stocking food and

Ongoing Tenancy Supports

cleaning supplies. They also include connecting utilities

and establishing independent living skil s. PRA residents

Ongoing tenancy supports are personal and

receive transition services mainly from community-based

community supports that help an individual continue

service providers, primarily funded by Medicaid. State,

to live independently in his or her home. Ongoing

local, and charitable organizations may also provide

personal and community supports include two kinds of

funding for transition services in some states, but funding

services: 1) tenancy supports that assist individuals with

is limited. In rare instances, property owners may employ

their responsibilities as tenants; and 2) community-based

a service coordinator who assists residents with obtaining

long-term services and supports that help individuals with

small furniture items or provides a starter supply of

disabilities with personal care needs, transportation, or

household goods. Transition service providers also often

other community linkages.

assist residents with applying for benefits such as utility

Once residents are established in their PRA units, the

assistance and SNAP (food stamps), to help support their

frequency of their interactions with community and

independent living.

property-based service providers varies widely. The type

Funding for transition services is covered for Medicaid

and intensity of service provider interaction is, in large

MFP participants but not for other enrol ees. Service

part, based on individual residents’ established service

providers we interviewed told us that outfitting an

needs. Each individual who is served by Medicaid or a

apartment is one of their major expenses and that funding

similar state plan services typical y has a case manager

for these items is always a patchwork, cobbled together

who develops a service plan specifical y for that individual.

through a variety of sources. Service providers in the

The service plan identifies the services that the individual

six study states general y have excel ent connections to

needs and is eligible to receive. Individuals in the MFP

state and county social services agencies, as well as to

program are eligible to receive case management and

private philanthropic entities that assist with transitions

tenancy supports for 12 months after they move into their

to community-based living. Many PRA residents do not

units.

own furniture or other household goods such as cooking

However, some services may be limited by what is

utensils or sheets and towels. For example, in parts of

al owable under a particular Medicaid waiver or by

Minnesota, a county agency offers support with buying

available community resources. For example, service

furniture. In some states, resources to assist residents

providers in some communities report a shortage of

with the physical move to a PRA residence are scarce. A

home health aides or other home and community-based

property manager from Delaware said, “When it comes

providers.

to the physical moving, boxing up everything…there is

nothing…. They have to hire movers and a lot of them

Similar percentages of PRAC and PRA residents

can’t afford it.” Some transition managers are extremely

report needing help with medication, meals, and

resourceful and leverage all available federal, state,

bathing. Thirty percent of PRA residents and 27

county, and philanthropic resources to help residents

percent of PRAC residents report needing help

move in and set up their homes. Other transition managers

taking medication. Similarly, 29 percent of PRA

were frustrated by an inability to fill certain needs, such

residents and 26 percent of PRAC residents report

as assisting with food stamp eligibility or procuring

needing help preparing meals and snacks, and

household cleaning products.

22 percent of PRA residents and 19 percent of

PRAC residents report needing help with bathing.

Similar percentages of PRA and PRAC residents

surveyed report receiving help with getting settled

into their home: 70 percent of PRA residents and

69 percent of PRAC residents. More PRA residents

42

H U D S e c t i o n 811 P R A P ro g r a m – P h a s e I I E v a l u a t i o n

Chapter 6. PRA and PRAC Community-Based Services

The majority of PRA and PRAC residents report

PRAC residents also rated their experience with

receiving ongoing tenancy supports. More than

people who work with them highly. The difference

half of PRA (52 percent) and PRAC residents (55

is not statistically significant. Most PRAC residents

percent) report receiving ongoing help with activities

report that the people who work with them know the

like paying bil s, resolving conflicts with landlords or

services they need (89 percent), that they treat them

neighbors, and following the rules of the property.

the way they want to be treated (80 percent), and

that they show up to work on time most of the time

In addition to paid caregivers, 44 percent of PRA

(78 percent).

residents also report receiving additional weekly

help from family and friends, compared to 54

The survey did reveal that a small share of PRA and

percent of PRAC residents. Respondents mentioned

PRAC residents are not receiving the services they

that family or friends may run errands or shop for

need. Only 12 percent of PRA residents reported

groceries for them. Family members who live with

that they need more help with ongoing tenancy

the respondent may provide daily help with meals,

than they are currently getting. While more PRAC

bathing, or dressing. PRA residents report receiving

residents report needing more help with tenancy

more hours of help from family and friends than

support than they are getting (26 percent), this

PRAC residents receive. Of residents who receive

difference is not statistical y significant.

support from family and friends, 44 percent of

PRA residents receive at least 5 hours of such

Only a small number of residents reported going

assistance per week, compared to only 27 percent

without the help they need with medication, meals,

of PRAC residents. The difference in hours of

or bathing. Similar percentages of PRA and PRAC

week of assistance is not statistical y significant.

residents report going without the help they

need, with one exception. A significantly higher

percentage of PRA residents who report needing

6.2 Resident Experience with

help taking their medications have gone without

Community-based Services and

taking medication when needed (64 percent),

compared to only 15 percent for PRAC residents.

Alignment with Needs

Resident Experience with

Quality of Life and Health Status

Community-based Services

The study team asked PRA and PRAC residents to rate

Most of the PRA residents who reported receiving help

their quality of life and overall health as “excel ent,” “good,”

with everyday activities32 (68 percent of PRA residents

“okay,” or “poor.” The majority of PRA residents (58

surveyed), reported positive experiences with the help

percent) rate their quality of life as either excellent

they received. Most PRA residents with paid caregivers

or good, and 42 percent rate their overall health as

reported that the people who work with them are

excellent or good. While the majority (87 percent) of

knowledgeable about the support they need (more than

residents report that their mental or emotional health is

94 percent33), that they come to work when they are

excel ent, good, or okay, the majority of PRA residents (69

supposed to most of the time (91 percent), and that they

percent) also report that they had symptoms of mental or

treat them the way they want to be treated (89 percent).

emotional health conditions in the past week.

32 As defined as a response of “Yes,” “No,” “Don’t know,” or “Refuse” to the question of “Do the people who work with you know what kind of help you need with everyday activities, like getting ready in the morning, getting groceries, or going places in your community?”

33 Exact percentage not shown due to small sampling restrictions.

43

H U D S e c t i o n 811 P R A P ro g r a m – P h a s e I I E v a l u a t i o n

Chapter 6. PRA and PRAC Community-Based Services

More PRAC residents than PRA residents

Two-thirds of PRAC residents (67 percent)

reported their overall quality of life and overall

report their mental and emotional health as

emotional and mental health as “excellent” or

excellent or good, compared to 49 percent of

“good.” Almost three-fourths of PRAC residents

PRA residents, also a statistically significant

(73 percent) report that their overall quality

difference. Most residents also report periodic

of life is “excellent” or “good,” compared to

issues with their mental or emotional health.

58 percent of PRA residents, a statistically

Approximately two-thirds of both PRA (69 percent)

significant difference. (See Exhibit 6.3).

and PRAC residents (64 percent) report that they

have been bothered by “feeling sad, blue, nervous,

or cranky” at least sometimes in the past week.

E xhibit 6.3 : S t atistically Significant Dif ferences in Repor ted

Health and Qualit y of Life of Section 811 Resident s

Difference in

Percentage of Residents Who Report

PRA N

PRAC N

PRA %

PRAC %

Means

Their quality of life is excellent or good

186

206

58.1%

72.7%

-14.7%**

Their mental or emotional health is excellent or good

183

207

49.2%

67.2%

-18.1%**

**Indicates a statistically significant difference between PRA and the comparison group with a p-value of .05.

Note: The PRAC survey results were adjusted and weighted based on demographic and healthcare utilization characteristics of PRA and PRAC residents.

Source: Abt analysis of Section 811 PRA Resident Survey administered January through May 2018 in the six study states.

6.3 Housing Tenancy and Program

lived in their housing for a little over a year. The PRA

Exits

residents with the longest tenure had been housed for

less than four years; some 45 percent of PRA residents

A goal of affordable, community-based housing is to

had been assisted for less than a year. Not surprisingly,

avoid unnecessary institutionalizations of people with

the average length of tenancy of the PRAC group is much

disabilities, and to al ow people to live successful y in the

longer than for PRA residents. PRAC residents on average

community with supports. The PRA option may lead to

had lived in their housing for 7 years.

longer stays for residents than similar programs that assist

According to grantee-reported data, 193 PRA households

non-elderly people with disabilities, or PRA residents may

had exited their unit in the first 3 years of the PRA

have different reasons for leaving the program. Using

program. PRA tenants initiated 27 percent of these exits,

HUD administrative data, the study team compared rates

and owners initiated 30 percent (Exhibit 6.4). Of the 51

of program exit and reasons for exits for PRA and PRAC

households that left of their own accord, 36 exited for

residents.

other housing, and 15 left for other, unknown reasons.

As described in the first chapter, PRA residents have

Of the 59 exits that were owner-initiated, 24 were for

had fairly short durations of tenancy as a result of the

nonpayment of rent and 35 were for other lease violations.

recency and slow implementation of the PRA program.

An additional 19 percent of residents died, and 8 percent

On average, as of September 2018, PRA residents had

moved to institutional care.

44

H U D S e c t i o n 811 P R A P ro g r a m – P h a s e I I E v a l u a t i o n

Chapter 6. PRA and PRAC Community-Based Services

E xhibit 6.4 : Reasons Tenant s Had Lef t P R A Housing as of September 2018

Reason

PRA N

PRA %

Tenant initiated–left for other housing

36

19%

Tenant initiated–other

15

8%

Owner initiated–nonpayment of rent

24

12%

Owner initiated–other

35

18%

Death

37

19%

Institutionalized

16

8%

Unit transfer FY12 to FY13

<10a

<6%a

Unknown/disappeared

<10a

<6%a

Other

17

9%

Total

193

100%

a Exact values are not shown due to small sample reporting restrictions.

Source: Abt analysis of 2015-2018 Section 811 PRA Quarterly Reports.

Using HUD administrative data, the study team compared

due to other owner-initiated reasons besides nonpayment

program exit rates and reasons for exiting for PRA and

of rent.

PRAC residents.34 PRA and comparable PRAC residents

Exhibit 6.5 shows hazard ratios for reasons for exiting the

leave the Section 811 program at similar rates. For both

program. Each row is a separate Cox model regression

PRA and PRAC, in the first 3 years after they moved

reporting the coefficient on PRA residents relative to

in, residents exited the program at a rate of just over

PRAC. For example, the hazard ratio of 5.74 for owner-

20 percent per year. After the third year, the rate of exit

initiated nonpayment of rent shows that PRA residents

leveled off for both groups.

are 5.74 more likely to exit for nonpayment of rent than

Because PRAC residents in our sample have, on average,

PRAC residents. In this exhibit, the only statistically

earlier move-in dates than PRA residents and thus have

significant differences in reasons for exits are for owner-

a longer period over which to exit, PRAC residents who

initiated nonpayment of rent and for tenant-initiated

exited had been assisted by PRAC for an average of 17

reasons. However, this large difference in hazards for PRA

months. Through September 2018, PRA residents who

corresponds to a very low prevalence, on the order of half

exited had been assisted by PRA for an average of 11

a percent a year (compared to about one in a thousand for

months.

PRAC). In contrast, the point estimate on tenant-initiated

Based on reason-for-exit codes in TRACS administrative

(other) reasons corresponds to a larger gap in prevalence

data, PRA residents are almost six times more likely to

of exit, with exits on the order of 14 percent a year for PRA

exit for nonpayment of rent (owner-initiated) than are

and 7 percent a year for PRAC; however this difference

comparable PRAC residents. We also find suggestive

could be due to chance (that is, the hazard ratio does not

evidence that the likelihood of tenant-initiated exits may be

differ statistical y from 1).

somewhat lower in PRA. Our analysis indicates such exits

are half as likely for PRA residents as compared to PRAC

residents, but the difference is not statistical y significant.

The analysis did not find any statistical differences in the

likelihood of exits due to death or primary householder or

34 We estimated a Cox proportional hazard model of program exit. This model uses the same coefficients and propensity score weighting approach as the other impact models. The model estimates the likelihood of exiting the program taking into account the varying lengths of time a resident has already received assistance in addition to individual baseline demographic and health characteristics. Coefficients of the model are interpreted as hazard ratios, which in our model indicate the ratio of share of PRA residents expected to exit the program in a given month to the share of PRAC

residents expected to exit that program in a given month. Hazard ratios greater than one reflect higher likelihood of exit for PRA as compared to PRAC, while ratios less than one reflect a lower likelihood of exit.

45

H U D S e c t i o n 811 P R A P ro g r a m – P h a s e I I E v a l u a t i o n

Chapter 6. PRA and PRAC Community-Based Services

E xhibit 6.5 : Ha zard Ratios for Reasons for Program E xit s for P R A and P R AC Resident s, 2013 –2018

Exit Definition

Hazard Ratio

Standard Error

P-Value

Owner initiated for nonpayment of rent

5.74

1.84

0.00

Owner initiated—other

0.99

0.24

0.95

Tenant initiated

0.50

0.09

0.00

Death

0.90

0.25

0.69

Source: Abt analysis of program exit data from TRACS for PRA and PRAC residents as of September 2018.

46

H U D S e c t i o n 811 P R A P ro g r a m – P h a s e I I E v a l u a t i o n

Chapter 7. Healthcare Conditions and Utilization

Healthcare Conditions services when we compared PRA residents’ healthcare

utilization patterns after moving-in to similar people

and Utilization

living in other settings. In less than 1 year of fol ow-up

after moving in, PRA residents tended to use inpatient

hospital, emergency department, medical transportation,

and inpatient long-term care services at lower rates than

similar individuals in the comparison groups. In contrast,

we found that PRA residents were more likely to use

The PRA program provides stable, affordable housing,

personal care attendant or case management services.

with connections to Medicaid or state programs that

Because the comparison groups have been matched

provide home and community-based services. HUD and

on individual characteristics, including their healthcare

HHS hypothesize that this housing and services model wil

utilization in 2015, lower use of medical services may

decrease unnecessary institutionalization, improve access

reflect greater access to or coordination of services, while

to healthcare, and reduce the use of emergency services.

the greater use of personal care attendants and case

To assess whether that will happen, in Chapters 3 through

management services may reflect a history of unmet

6, we looked at whom the program is serving, what types

need prior to PRA tenancy. Since few of the estimated

of properties and neighborhoods they live in, and the

differences were statistically significant, some of the

tenancy supports they need and receive. In this chapter,

observed differences could be due to chance.

we look at PRA residents’ use of healthcare services

before and after being assisted by PRA, including how

To examine healthcare characteristics and utilization

use during PRA tenancy differs from healthcare use by the

patterns, the study team obtained 2 years (2015–2016) of

study’s comparison groups.

medical claims and managed care encounter data from

the Medicaid agencies in the six study states.35 We sent

We found more than two-thirds of PRA residents had

unique identifiers for the members of the PRA and the

a mental health condition, and this was higher than for

three HUD comparison groups (PRAC, NED, and other

people in the NED, other HUD, and non-HUD groups. PRA

HUD) to the Medicaid agencies, who then linked the

residents were also more likely than the other groups to

PRA resident identifiers to data on Medicaid enrol ees

have a chronic or disabling condition and had higher rates

and to three HUD comparison groups. We also obtained

of healthcare utilization in the pre-occupancy period—that

Medicaid data from each state for a sample of Medicaid

is, before weighting the comparison groups to match them

enrol ees who were not assisted by HUD programs. For

based on pre-occupancy healthcare utilization.

brevity, this comparison group is referred to as the “non-

Propensity score weighting is used to calculate averages

HUD” group.

in a way that places more importance on individuals

We used the 2016 Medicaid data to examine PRA

in the comparison group with characteristics similar to

residents’ healthcare utilization for those who moved in

the individuals in the PRA program, which makes the

during 2016, and for individuals in the HUD comparison

comparison group’s weighted averages representative of

groups who moved into their units between 2013 and

the PRA group’s averages had they not been assigned

2016.36 Exhibit 7.1 shows the number of PRA residents and

to PRA units. After propensity score weighting, we

individuals in the comparison groups.

found few statistical y significant differences in the use

of expensive emergency or institutional long-term care

35 These were the most recent data available for all states. Two states, Minnesota and Washington, provided claims and encounter data for 2 years and 6 months (from January 2015 through June 2017).

36 The earliest year that a PRA resident had moved into their unit was 2013. Additionally, we also include individuals in the PRA and comparison groups who live in Minnesota and Washington and moved in during the first half of 2017 since they could be linked to post-occupancy data on utilization.

47

H U D S e c t i o n 811 P R A P ro g r a m – P h a s e I I E v a l u a t i o n

Chapter 7. Healthcare Conditions and Utilization

E xhibit 7.1: Healthcare U tilization C omparison S amples

PRA

PRAC

NED

Other HUD

Non-HUD

State

N

%

N

%

N

%

N

%

N

%

California

35

14%

106

9%

<10a

<4%a

61

9%

814

1%

Delaware

24

9%

122

11%

<10a

<4%a

103

16%

4,987

8%

Louisiana

61

24%

223

20%

44

16%

110

17%

40,000

68%

Maryland

19

7%

168

15%

0

0%

78

12%

5,461

9%

Minnesota

68

26%

235

21%

≥10

≥4%

133

21%

2,183

4%

Washington

50

19%

288

25%

209

76%

161

25%

5,701

10%

Total

257

1,142

275

646

59,146

a Exact percentages are not shown due to small sample reporting restrictions.

Notes: Since the non-HUD group was not linked to HUD administrative data and data on demographic characteristics came from Medicaid data, it was necessary to restrict both the PRA and non-HUD sample to individuals enrolled in Medicaid for at least 1 month in 2015. This dropped less than 11 PRA residents from analyses comparing PRA residents to the Non-HUD sample in Delaware, Minnesota, Washington, and Louisiana.

Sources: Abt Associates analysis of TRACS and PIC household data from the quarter ending on March 31, 2018; and Medicaid claims and encounter data for 2015-2016 (June 2017 for Minnesota & Washington) received from the State Medicaid agencies for California, Delaware, Louisiana, Maryland, Minnesota, and Washington.

7.1 Healthcare Conditions and

Chronic and Potentially Disabling Conditions

Prior Healthcare Utilization

We used diagnoses listed on 2015 Medicaid claims

and encounter data37 and applied criteria defined by

PRA residents have a range of disabilities and chronic

the Centers for Medicare & Medicaid Services (CMS)

conditions and therefore varying service needs and rates

to identify individuals with any of 27 common chronic

of healthcare utilization. In this section, we present 2015

conditions or 33 other chronic or potential y disabling

Medicaid data to describe residents’ rates of chronic

conditions.38 We grouped the 60 chronic and potential y

and disabling conditions and healthcare utilization for the

disabling conditions into 19 categories. For example,

period prior to receiving PRA assistance. To understand

we grouped depression, anxiety, post-traumatic stress,

the characteristics of PRA residents relative to the other

bipolar personality, and psychotic disorders into

study groups, we examined these rates in the comparison

one category related to mental health. Hip fractures,

groups prior to weighting the groups to match them to

osteoporosis, and rheumatoid arthritis were categorized

the PRA residents. In this analysis, we found that the

as musculoskeletal, while chronic kidney disease,

prevalence of chronic and disabling conditions tended to

diabetes, and hypothyroidism were categorized as

be higher among PRA residents than in the comparison

endocrine and renal. The full list of conditions for each

groups. Likewise, PRA residents tended to utilize

category is listed in Appendix B.

healthcare services at higher rates than individuals in the

More than half of PRA residents have mental

comparison groups. Greater prevalence of chronic and

health conditions (71 percent) and musculoskeletal

potential y disabling conditions among PRA residents

conditions (61 percent.) Just under one-third (32

could be related to the target groups the states selected

percent) have endocrine or renal conditions, and a little

for their PRA programs. PRA is more explicitly targeting

less than one-quarter have pulmonary conditions (24

people leaving institutions or leaving homelessness than

percent) (Exhibit 7.2). Prior to weighting the comparison

other HUD programs in our study. The requirement that

groups to match them to the PRA residents, PRA

PRA residents must be eligible to receive Medicaid HCBS

residents were most similar to PRAC residents in the

or similar services may also be playing a role. Although

prevalence of chronic and disabling conditions. Only

PRAC residents also have coordinated access to services,

cardiovascular conditions and other conditions (including

there are no such Medicaid eligibility requirements in

fibromyalgia, ulcers, or viral hepatitis) were more

PRAC or NED or the other HUD programs.

37 Managed Care Organizations contracted with a Medicaid program submit “encounter” claims with the same information about the healthcare service provided, the treating provider, and other patient-level detail found in fee-for-service claims but, for some states, the encounter data does not include information on reimbursement to providers.

38 https://www.ccwdata.org/web/guest/condition-categories

48

H U D S e c t i o n 811 P R A P ro g r a m – P h a s e I I E v a l u a t i o n

Chapter 7. Healthcare Conditions and Utilization

prevalent among PRA residents than PRAC residents.

or sensory impairments between PRA residents and any

Developmental disorders, mental health conditions, and

of the comparison groups. Nearly all of the other 13

mobility impairments were more prevalent among PRA

categories of conditions were more prevalent among

residents than in the other HUD group. There was no

PRA residents than among individuals in the NED and

difference in the prevalence of cancer, HIV/AIDS, leukemia

non-HUD groups.

and lymphoma, liver conditions, neurological disorders,

E xhibit 7.2 : Unweighted C omparisons of Chronic and Potentially Disabling C onditions among

P R A Resident s and Individuals in the C omparison Groups Enrolled in Medicaid in 2015 and 2016

PRA

PRAC

NED

Other HUD

Non-HUD

(N=251)

(N=1,112)

(N=268)

(N=624)

(N=59,146)

Category/Measure

Percent

Percent

Percent

Percent

Percent

Cancer

<4%a

1.3%

<4%a

<2%a

1.1%

Cardiovascular

15.9%

11.0%**

4.9%**

13.8%

5.4%**

Developmental disorders or disabilities

16.7%

21.7%

7.1%**

7.1%**

10.7%**

Endocrine and renal

31.9%

27.0%

20.5%**

26.8%

15.3%**

HIV/AIDS

<4%a

3.6%

<4%a

2.9%

1.9%

Leukemia and lymphomas

<4%a

<1%a

<4%a

<2%a

0.2%

Liver conditions

4.8%

3.6%

<4%a

3.8%

2.2%

Mental health

70.9%

67.8%

40.7%**

47.8%**

39.4%**

Mobility impairments

6.8%

5.6%

<4%a**

2.2%**

1.5%**

Musculoskeletal

60.6%

62.1%

69.0%**

58.8%

28.2%**

Neurological disorders

13.9%

16.8%

10.8%

9.5%

10.0%

Obesity

13.5%

10.1%

6.0%**

14.1%

6.5%**

Ophthalmic

4.8%

5.5%

4.5%

6.1%

1.7%**

Other chronic conditions b

45.4%

41.5%

26.9%**

45.2%

27.1%**

Other conditions c

24.7%

18.7%**

20.5%

19.7%

15.9%**

Peripheral vascular disease

4.8%

4.0%

<4%a**

3.0%

1.4%**

Pulmonary

23.5%

18.1%

13.4%**

23.4%

12.0%**

Sensory impairment

< 4%a

3.1%

<4%a

1.8%

0.8%

Tobacco use disorders

17.5%

13.4%

<4%a**

13.0%

15.6%

**Statistically significant difference compared to PRA residents at the 5 percent level ( p<.05).

a Exact percentages not shown due to small sample reporting restrictions.

b Includes anemia, hyperlipidemia, hypertension, and benign prostatic hyperplasia.

c Includes fibromyalgia, pressure or chronic ulcers, and viral hepatitis.

Notes: PRA residents moved in during 2016 (or first two quarters of 2017 for Minnesota & Washington); PRAC, NED, and other HUD residents moved into their unit during 2013-2016 (or first two quarters of 2017 for Minnesota & Washington). Non-HUD individuals were enrolled in Medicaid for at least part of 2015 and 2016.

Sources: Abt Associates analysis of TRACS and PIC household data from the quarter ending on March 31, 2018; and Medicaid claims and encounter data for 2015-2016 (June 2017 for Minnesota & Washington) received from the State Medicaid agencies for California, Delaware, Louisiana, Maryland, Minnesota, and Washington.

7.2 PRA Impacts on

care” services (that is, hospital admissions, emergency

Healthcare Utilization

department visits, and medical transportation) and three

measures related to the long-term use of institutional

Using the Medicaid data from the study states, we

services. Some of the measures are subsets of other

created seven measures to examine the use of “acute-

measures (for example, 30-day readmission can only

49

H U D S e c t i o n 811 P R A P ro g r a m – P h a s e I I E v a l u a t i o n

Chapter 7. Healthcare Conditions and Utilization

fol ow admission, so anyone who was not admitted is not

• Any admission for long-term inpatient care (yes/no)

at risk of readmission).

• Any use of personal care attendant services (yes/no)

Because the PRA program requires residents to be eligible

• Any use of case management services (yes/no)

for Medicaid or state-funded HCBS, we were interested

in differences in the “take-up” of HCBS between PRA

To calculate these measures, we used the 2016 Medicaid

residents and individuals living in other settings. There

data to examine PRA residents’ healthcare utilization for

was not a way to reliably and consistently identify HCBS

those who moved in during 2016 and for people in the

in every state’s data, and the bundle of services offered

PRAC, NED, and other HUD comparison groups who

varies by state. To proxy HCBS use, we created two binary

moved into their units between 2013 and 2016.40,41 We

measures for whether PRA residents or the comparison

accounted for the number of months in 2016 that PRA

groups used any personal care attendant (PCA) or case

residents were enrol ed in Medicaid, after they moved

management services, post-occupancy. PCA services are

into their PRA unit, by calculating individual utilization

designed to help persons with disabilities with everyday

rates as the number of events per quarter (that is, three

activities (such as bathing or dressing) and are typical y

months).42,43 The 2015 Medicaid data were used to weight

services included in Medicaid HCBS programs. Case

the comparison groups to match PRA residents based

managers coordinate the various services offered in

on various chronic and disabling conditions and prior

Medicaid HCBS programs. Case management services

rates of healthcare utilization. HUD administrative data

may also be offered to a broader population, for reasons

(or Medicaid data for the non-HUD group) were used to

unrelated to HCBS for people with disabilities.

match based on demographic characteristics.

We compared average rates of healthcare utilization

We limited this comparison to individuals who are similar

among PRA residents and the comparison groups using

to PRA residents in terms of race and ethnicity, gender,

the following measures:

age, any dependents, income, rent, prior Medicaid

enrol ment, chronic and disabling conditions, and prior

• Number of inpatient hospital admissions (per quarter)

healthcare utilization,44 so that differences in population

• Number of inpatient days (per quarter)

characteristics would not be driving the results. In the

• Number of 30-day readmissions to a hospital

figures below impacts that were statistical y significant

(per quarter)

either at the 1-percent or 5-percent levels, before

adjusting for multiple hypothesis testing, are shown with

• Number of inpatient hospital admissions for mental

an asterisk (* for significance at the 10-percent level,

health conditions (per quarter)

** for significance at 5-percent level, and *** for

• Number of emergency department visits (per quarter)

significance at the 1-percent level). 45

• Number of emergency department visits not resulting

After we adjusted for multiple comparisons,

in inpatient admission (per quarter)

however, the only differences that are statistically

significant are those comparing PRA residents to

• Number of emergency and non-emergency medical

otherwise similar Medicaid enrollees not receiving

transportation events (per quarter)

HUD assistance. Therefore, some of the observed

• Number of long-term inpatient days (per quarter)39

differences discussed in this section could be due to

• Number of admissions for long-term inpatient care

chance, and some relatively large differences may

(per quarter)

be reflecting chance variation in small samples. The

39 We use the term “long-term inpatient days/care” to refer to all inpatient admissions lasting longer than 28 days, for any reason other than hospice or substance abuse, and regardless of the type of provider (for example, nursing facility, acute care hospital, long-term care hospital, inpatient rehabilitation facility, intermediate care facility, and so on). We could not confidently distinguish the type of inpatient provider from the claims in all six study states, therefore we could not identify individuals admitted to a nursing facility or other type of facility with the specific intention of residing there to receive “long-term care.”

40 We also include individuals in the PRA and comparison groups who live in Minnesota and Washington and moved in during the first half of 2017

since they could be linked to post-occupancy data on utilization.

41 The earliest year that a PRA resident had moved into their unit was 2013.

42 We calculated the rates as the total number of events of interest, divided by the number of months in the period, multiplied by three. For the non-HUD group, we examined their utilization in 2016 accounting for the number of months they were continuously enrolled in Medicaid.

43 The average number of months of follow-up in 2016 (and 2017 for Washington and Minnesota residents) for PRA residents was 6.8 months, 12.0

months for PRAC residents, 15.6 months for the NED group, 11.8 months for the other HUD group, and 9.4 months for the non-HUD group.

44 The non-HUD group could not be matched to PRA residents based on any dependents, income, or rent since they were not linked to HUD

administrative data.

45 We adjusted p-values using the Benjamini-Hochberg multiple testing correction to understand how likely any statistically significant impact estimates were due simply to chance.

50

H U D S e c t i o n 811 P R A P ro g r a m – P h a s e I I E v a l u a t i o n

Chapter 7. Healthcare Conditions and Utilization

housing situation is unknown for individuals in the

services more often than the comparison groups prior to

non-HUD group and may include people who receive

being assisted by PRA, possibly reflecting their greater

housing assistance from sources other than HUD.

prevalence of chronic and disabling conditions and

Detailed descriptions of the variables, comparison groups,

lack of appropriate and stable housing. Prior to PRA

and methodology used in the analysis below are provided

assistance, residents enrol ed in Medicaid during 2015

in Appendix B.

utilized inpatient hospital services, emergency department

services, and medical transportation services more often

Prior Healthcare Utilization of PRA Residents

than people in the PRAC, NED, other HUD, and non-HUD

and the Comparison Groups

groups. As expected, given the states often target PRA

units to people leaving institutions and these people

We compared average rates of healthcare utilization

represent more than a quarter of PRA residents assisted

among PRA residents before they were assisted by

between 2015 and 2018, they also used long-term

PRA to average rates of utilization among comparison

inpatient care more often than all comparison groups prior

groups, prior to weighting them to match them to the PRA

to PRA assistance.46

residents. The PRA group tended to utilize healthcare

E xhibit 7.3 : Unweighted C omparisons of Healthcare U tilization, 2015 ( prior to moving in for P R A resident s ) PRA

PRAC

NED

Other HUD

Non-HUD

(N=251)

(N=1,112)

(N=268)

(N=624)

(N=59,146)

Category Measure

Mean

Mean

Mean

Mean

Mean

Days admitted to an inpatient hospital,

1.98

0.87

0.16**

0.38**

0.56**

per quarter

Number of inpatient hospital admissions,

0.18

0.09**

0.03**

0.06**

0.07**

per quarter

Number of 30-day readmissions to a hospital, per

0.05

0.02

0.004**

0.01**

0.01**

quarter

Number of inpatient hospital admissions for

0.08

0.03

0.01**

0.01**

0.03**

mental health conditions, per quarter

Number of emergency department visits,

1.11

0.49**

0.27**

0.44**

0.66**

per quarter

Number of emergency department visits not

0.96

0.41**

0.25**

0.38**

0.62**

resulting in inpatient admission, per quarter

Number of emergency and non-emergency

1.96

1.51

0.48**

1.33

0.39**

medical transportation events, per quarter

Days of long-term inpatient care, per quarter

8.42

1.35**

0.82**

0.32**

5.33

Number of admissions for long-term inpatient

0.05

0.01**

0.01**

0.004**

0.02**

care, per quarter

Any admission for long-term inpatient care

11.6%

2.5%**

<4%a**

<2%a**

4.8%**

Any use of personal care attendant services

10.3%

15.4%**

11.9%

12.0%

3.1%**

Any use of case management services

19.5%

20.7%

4.9%**

10.7%**

3.7%**

**Statistically significant difference compared to PRA residents at the 5 percent level ( p<.05).

a Exact percentages not shown due to small sample reporting restrictions.

Notes: PRA residents moved in during 2016 (or first two quarters of 2017 for Minnesota & Washington). PRAC, NED, and other HUD residents moved into their unit during 2013-2016 (or first two quarters of 2017 for Minnesota & Washington). Non-HUD individuals were enrolled in Medicaid for at least part of 2015 and 2016. We could not identify personal care attendant services using Maryland Medicaid fee-for-service claims or encounter data, thus Maryland was dropped from comparisons of personal care attendant use.

Sources: Abt Associates analysis of TRACS and PIC household data from the quarter ending on March 31, 2018; and Medicaid claims and encounter data for 2015.

46 This result is also partly due to inclusion criteria for the sample. A proportion of individuals in the PRAC, NED, and other HUD groups had moved into their residences prior to 2015 so were less likely to have used long-term inpatient care in 2015.

51

H U D S e c t i o n 811 P R A P ro g r a m – P h a s e I I E v a l u a t i o n

Chapter 7. Healthcare Conditions and Utilization

Prior to being assisted by PRA, PRA residents were

transitions to long-term care institutions. Additional y,

somewhat less likely than PRAC residents to use PCA

weighting and regression models cannot adjust for

services, more likely than the non-HUD group, and

unobserved factors that influence rates of healthcare

no more likely than the NED and other HUD groups.

utilization that could differ between PRA residents and

The PRAC and NED programs target individuals with

the comparison groups, including Medicaid eligibility

disabilities who may need PCA or similar support services

requirements. Although we match the comparison group

to live independently in the community, while the other

to PRA residents based on 60 different types of chronic

HUD and non-HUD sample is more diverse. Prior to PRA

and disabling conditions, the groups may still differ with

tenancy, the PRA residents were no more likely than PRAC

respect to the severity of those conditions and overal

residents to use case management services, but more

cognitive or functional status.

likely to use these services than the NED, other HUD, and

In addition, while many people with disabilities

non-HUD groups.

receiving HUD assistance are dual-enrolled in

Early PRA Tenancy Findings

Medicaid and Medicare, we were only able to

measure post-occupancy utilization using Medicaid

We found that after less than 1 year of being assisted

claims and managed care encounter data collected

by PRA, PRA residents tended to use inpatient hospital,

from the state Medicaid agencies. Medicare is the

emergency department, and medical transportation at

primary payer for hospitalizations, physician services,

lower rates than similar individuals in the PRAC, NED,

post-acute care services, hospice care, and prescription

and non-HUD groups, as well as to be less likely to have

drugs among dual-enrol ed individuals. Medicaid only

long-term institutional stays. The opposite pattern was

pays for specific services not covered by Medicare and

observed with respect to the other HUD group. Few of

sometimes covers the cost of premiums, deductibles,

the estimated differences were statistical y significant,

co-pays or co-insurance (benefits vary across states).

however. That is, some of the observed differences

While the Medicaid data do contain “crossover claims”

could be due to chance, including some relatively large

(a claim bil ed to Medicaid for the Medicare deductible

differences that may be reflecting chance variation in smal

or coinsurance), payment policies vary by state and

samples.

we cannot be certain that we captured services that

We also found that PRA residents were approximately 5

were entirely paid by Medicare. Thus, it is likely we have

to 10 percent more likely to use PCA or case management

underestimated healthcare utilization by PRA residents

services than similar individuals in each of the comparison

and the comparison groups. Moreover, about 20 percent

groups. PRA residents were 20 percent more likely than

of the PRA residents were dual-enrol ed in 2016 (they had

the non-HUD group to use case management services.

one or more crossover claims) while between 40 and 50

These results most likely reflect higher rates of use of

percent of PRAC, NED, and other-HUD groups were dual-

HCBS by PRA residents since, unlike the other HUD

enrolled. Although we matched the comparison groups

programs, they are required to be eligible for services, and

to PRA residents based on likely determinants of dual-

PRA grantees and their partners work to make sure that

enrol ment (disability status, prior utilization rates, and

the residents can access these services. The traditional

income levels), we still may have under or overestimated

PRAC program is similar to the PRA program in that it

the impact of PRA on healthcare utilization since being

provides stable, affordable housing with structured access

assisted by PRA, to some degree.

to services. In contrast, the NED voucher program and

other HUD housing programs provide rental assistance

7.3 Comparison of Healthcare

but with no formal provisions for coordinated access to

Utilization for PRA Residents and

services.

Each Comparison Group

This analysis of healthcare utilization rates of PRA

residents suggests that the PRA program could

To assess the impacts of PRA on residents’ healthcare

have a substantive long-term impact on healthcare

utilization, we compare PRA residents’ post-occupancy

utilization in a population with many unmet healthcare

healthcare utilization rates in 2016 to those of people in

needs. We cannot draw definitive conclusions, however,

the PRAC, NED, other HUD, and non-HUD groups after

because of the absence of statistical y significant

weighting the comparison groups to match the PRA

differences. The 1-year period we were able to observe

residents based on prior rates of healthcare utilization and

after PRA residents moved in is likely too short to detect

other characteristics.

or attribute changes in patterns of healthcare utilization

to the PRA program, particularly for rare outcomes like

52

H U D S e c t i o n 811 P R A P ro g r a m – P h a s e I I E v a l u a t i o n

Chapter 7. Healthcare Conditions and Utilization

PRA and PRAC Comparison

particularly when the visit did not lead to an inpatient

The traditional PRAC program is similar to the PRA

admission (0.19 more visits per quarter). On the other

program in that it provides stable, affordable housing with

hand, PRA residents had fewer 30-day readmissions to an

coordinated access to services. We found no statistical y

inpatient hospital (0.03 fewer readmissions per quarter),

significant differences between PRA and PRAC residents

and while they spent fewer days than PRAC residents

with respect to the number of inpatient admissions,

admitted to an inpatient hospital (0.81 versus 1.29 days

inpatient admissions for mental health conditions, or

per quarter), the impact was not statistical y significant.

medical transportation events. (Exhibit 7.4). Compared to

The proportion of PRA residents who used PCA services

similar PRAC residents, PRA residents were less likely to

was 5 percent higher than among PRAC residents (21.4

have a long-term inpatient stay (lasting 28 days or longer)

percent versus 16.5 percent), but similar proportions of

in a hospital, nursing facility, or other institutional setting,

the two groups used case management services (24.1

after they became a PRA resident. There were 3.8 percent

percent for PRA versus 20.5 percent for PRAC). Therefore,

fewer PRA residents admitted for a long-term stay, they

PRA residents may have somewhat higher rates of use of

had 0.02 fewer such admissions, and they spent about

HCBS than similar PRAC residents, if we assume PCA is a

1.2 fewer days per quarter. PRA residents visited the

proxy for a wider array of community-based supports.

emergency department more often than PRAC residents,

E xhibit 7.4 : Average Rates of Healthcare U tilization of P R A Resident s Relative to P R AC Resident s, 2016

Difference in Means (p-value)

Any case management services

Any PCA services

0.04

(0.21)

Any long-term impatient stay

0.05

(0.09)

Long-term inpatient stays (/qtr)

-0.04***

(0.01)

-0.02**

(0.04)

Long-term inpatient days (/qtr)

-1.22**

(0.03)

Medical transportation trips (/qtr)

0.00

(1.00)

ED visits/no inpatient admission (/qtr)

0.19**

(0.04)

ED visits (/qtr)

0.16

(0.14)

Inpatient MH admissons (/qtr)

-0.01

(0.27)

30-day readmissions (/qtr)

-0.03*

(0.06)

Inpatient admissions (/qtr)

-0.01

(0.65)

Inpatient days (/qtr)

-0.48

(0.15)

0.0

0.5

1.0

1.5

2.0

2.5

* for p < 0.10

** for p < 0.05

PRAC

PRA

*** for p < 0.01

Notes: After we adjusted for multiple comparisons, none of the estimated differences were statistically significant. PRA residents moved in during 2016 (or first two quarters of 2017 for Minnesota & Washington); PRAC residents were first assisted 2013 through 2016 (or first two quarters of 2017 for Minnesota & Washington). We could not identify PCA services using Maryland Medicaid fee-for-service claims or encounter data, thus Maryland was dropped from comparisons of PCA use.

Sources: Abt Associates analysis of TRACS and PIC household data from the quarter ending on March 31, 2018; and Medicaid claims and encounter data for 2015.

PRA and NED Comparison

PRA residents visited the emergency department less

Unlike PRA, NED voucher holders do not need to be

often than the NED group (0.82 versus 1.14 visits per

eligible for or necessarily have access to HCBS to

quarter), particularly for visits that did not result in an

receive housing assistance. On average, PRA residents

inpatient hospital stay (0.70 versus 1.04 visits per quarter)

tended to use somewhat less acute care and had fewer

(Exhibit 7.5). PRA residents and the NED group were as

inpatient stays longer than 28 days than did similar NED

likely to have a long-term inpatient stay but, on average,

voucher recipients. There were few statistical y significant

PRA residents had slightly shorter stays after they moved

differences in healthcare utilization rates, and none of the

into their unit (0.22 versus 0.51 days per quarter). PRA

differences were statistical y significant after adjusting for

residents also spent fewer days admitted to an inpatient

multiple comparisons.

hospital (0.81 versus 1.11 days per quarter) despite similar

53

H U D S e c t i o n 811 P R A P ro g r a m – P h a s e I I E v a l u a t i o n

Chapter 7. Healthcare Conditions and Utilization

rates of inpatient hospital admissions. PRA residents

PRA residents and the other HUD group had similar

also used medical transportation less often than the NED

rates of admission to an inpatient hospital or admission

group (1.65 versus 2.39 events per quarter).

for long-term inpatient care (Exhibit 7.6). On average,

The proportion of PRA residents who used PCA services

though, PRA residents spent more days admitted to an

was 10 percent higher than in the NED group (21.4 percent

inpatient hospital (0.81 versus 0.60 days per quarter) and

versus 11.6 percent), but similar proportions of the two

spent more days in a long-term inpatient stay (0.22 versus

groups used case management services (24.1 percent

0.07 days per quarter). While PRA residents visited an

for PRA versus 27.3 percent for NED). Therefore, PRA

emergency room more often than the other HUD group

residents may have somewhat higher rates of use of

(0.82 versus 0.68 visits per quarter), they used medical

HCBS than NED voucher recipients.

transportation less often than the other HUD group (1.65

versus 2.52 events per quarter).

PRA and Other HUD Comparison

About 5 percent more PRA residents used PCA services

Like the NED group, members of the other HUD group

(21.4 percent versus 16.8 percent), and about 6 percent

are not necessarily eligible for Medicaid HCBS, or other

more PRA residents used case management services

types of services. Compared to the other HUD group, PRA

(24.1 percent versus 18.1 percent) than individuals in

residents used healthcare services at somewhat higher

the other HUD group, suggesting there may have been

rates. Few of the differences estimated were statistical y

somewhat higher rates of post-occupancy HCBS use

significant, however, and none were statistical y significant

among PRA residents.

after adjusting for multiple comparisons.

E xhibit 7.5 : Average Rates of Healthcare U tilization by

P R A Resident s Relative to N ED Voucher Holder s, 2016

Difference in Means (p-value)

Any case management services

-0.03

(0.52)

Any PCA services

0.10*

(0.08)

Any long-term inpatient stay

-0.01

(0.34)

Long-term inpatient stays (/qtr)

-0.01

(0.35)

Long-term inpatient days (/qtr)

-0.29

(0.34)

Medical transportation trips (/qtr)

-0.75

(0.14)

ED visits/no inpatient admission (/qtr)

-0.34**

(0.04)

ED visits (/qtr)

-0.32*

(0.08)

Inpatient MH admissions (/qtr)

-0.02

(0.36)

30-day readmissions (/qtr)

0.02

(0.17)

0.03

(0.44)

Inpatient admissions (/qtr)

0.30

(0.39)

Inpatient days (/qtr)

0.0

0.5

1.0

1.5

2.0

2.5

* for p < 0.10

** for p < 0.05

NED

PRA

Notes: After we adjusted for multiple comparisons, none of the estimated differences were statistically significant. PRA residents moved in during 2016 (or first two quarters of 2017 for Minnesota & Washington); PRAC residents were first assisted 2013 through 2016 (or first two quarters of 2017 for Minnesota & Washington). We could not identify PCA services using Maryland Medicaid fee-for-service claims or encounter data, thus Maryland was dropped from comparisons of PCA use.

Sources: Abt Associates analysis of TRACS and PIC household data from the quarter ending on March 31, 2018; and Medicaid claims and encounter data for 2015.

54

H U D S e c t i o n 811 P R A P ro g r a m – P h a s e I I E v a l u a t i o n

Chapter 7. Healthcare Conditions and Utilization

E xhibit 7.6 : Average Rates of Healthcare U tilization by P R A

Resident s Relative to Recipient s of O ther H U D A ssist ance, 2016

Difference in Means (p-value)

Any case management services

0.06*

(0.07)

Any PCA services

0.05

(0.21)

Any long-term inpatient stay

0.01

(0.22)

Long-term inpatient stays (/qtr)

0.00

(0.18)

Long-term inpatient days (/qtr)

0.16

(0.16)

Medical transportation trips (/qtr)

-0.87**

(0.02)

ED visits/no inpatient admission (/qtr)

0.10

(0.20)

ED visits (/qtr)

0.15

(0.11)

0.01

(0.50)

Inpatient MH admissions (/qtr)

0.01

(0.23)

30-day readmissions (/qtr)

0.02

(0.35)

Inpatient admissions (/qtr)

0.21

(0.26)

Inpatient days (/qtr)

* for p < 0.10

0.0

0.5

1.0

1.5

2.0

2.5

** for p < 0.05

Other HUD

PRA

Notes: After we adjusted for multiple comparisons, none of the estimated differences were statistically significant. PRA residents moved in during 2016 (or first two quarters of 2017 for Minnesota & Washington); PRAC residents were first assisted 2013 through 2016 (or first two quarters of 2017 for Minnesota & Washington). We could not identify PCA services using Maryland Medicaid fee-for-service claims or encounter data, thus Maryland was dropped from comparisons of PCA use.

Sources: Abt Associates analysis of TRACS and PIC household data from the quarter ending on March 31, 2018; and Medicaid claims and encounter data for 2015.

E xhibit 7.7: Average Rates of Healthcare U tilization for P R A Resident s

Relative to Medicaid Enrollees Not Receiving H U D A ssist ance, 2016

Any case management services

Difference in Means (p-value)

Any PCA services

0.20***

(0.00)

Any long-term inpatient stay

0.05**

(0.03)

Long-term inpatient stays (/qtr)

-0.03**

(0.02)

Long-term inpatient days (/qtr)

-0.01**

(0.04)

-2.60*

(0.00)

Medical transportation trips (/qtr)

-0.29

(0.45)

ED visits/no inpatient admission (/qtr)

-0.05

(0.47)

ED visits (/qtr)

-0.04

(0.67)

Inpatient MH admissions (/qtr)

0.00

(0.67)

30-day readmissions (/qtr)

0.00

(0.87)

Inpatient admissions (/qtr)

-0.00

(0.89)

Inpatient days (/qtr)

-0.13

(0.50)

0.0

0.5

1.0

1.5

2.0

2.5

3.0

** for p < 0.05

Non-HUD

PRA

*** for p < 0.01

Notes: After we adjusted for multiple comparisons, none of the estimated differences were statistically significant. PRA residents moved in during 2016 (or first two quarters of 2017 for Minnesota & Washington); PRAC residents were first assisted 2013 through 2016 (or first two quarters of 2017 for Minnesota & Washington). We could not identify PCA services using Maryland Medicaid fee-for-service claims or encounter data, thus Maryland was dropped from comparisons of PCA use.

Sources: Abt Associates analysis of TRACS and PIC household data from the quarter ending on March 31, 2018; and Medicaid claims and encounter data for 2015.

55

H U D S e c t i o n 811 P R A P ro g r a m – P h a s e I I E v a l u a t i o n

Chapter 7. Healthcare Conditions and Utilization

PRA and Non-HUD Comparison

The only statistical y significant differences in healthcare

The non-HUD comparison group includes people who

utilization were between PRA and the group without HUD

were enrol ed in Medicaid in 2015 and 2016 but were not

assistance: lower use of long-term inpatient care services

simultaneously receiving any form of HUD assistance.

and higher use of case management services by PRA

PRA residents used long-term inpatient care services

residents. This could mean that whether a household

less often than the non-HUD group. There were 2.8

receives HUD housing assistance may have a positive

percent fewer PRA residents ever admitted for a long-term

effect on health outcomes regardless of the type of

inpatient stay, they had 0.01 fewer admissions for such

HUD assistance the household receives. This inference

stays per quarter, and they spent about 2.6 fewer days per

should be viewed with caution, however. Because of data

quarter (Exhibit 7.7).

limitations, we could only fol ow PRA residents’ use of

healthcare services for a relatively small sample in six

PRA residents had 0.29 fewer medical transportation

states and for less than 1 year after entering the program,

events per quarter and about 0.05 fewer emergency

and we cannot draw any conclusions about how the PRA

department visits per quarter, but there were no

program would impact the use of healthcare services over

substantive differences between PRA residents and the

a longer time period.

non-HUD group with respect to the number of inpatient

hospital admissions, 30-day readmission, or inpatient

We did find that PRA residents were more likely to use

admissions for mental healthcare.

personal care attendants or case management services

than similar individuals in other housing settings, and

Compared to the non-HUD group, a greater proportion

this pattern could be associated with greater use of a

of PRA residents used PCA services (21.6 percent

wide array of community-based supports. However,

versus 16.3 percent) and case management services

the only differences that are statistical y significant after

(24.3 percent versus 4.4 percent). Only the estimated

accounting for multiple comparisons are those comparing

differences in long-term inpatient days and the proportion

PRA residents to otherwise similar Medicaid enrol ees not

of residents using case management services remained

receiving HUD assistance. While we cannot determine

statistical y significant at the 5-percent level.

whether the PRA program is superior to other housing

programs serving low-income individuals with disabilities,

Summary

we can conclude that those receiving HUD assistance

Comparisons of the prevalence of chronic and disabling

exhibit a different pattern of medical care (higher case

conditions showed that the PRA program serves a

management and lower rates of long-term inpatient

potential y higher-need population than PRAC, NED, and

stays) than otherwise similar individuals not receiving

other HUD programs, with higher rates of healthcare

HUD assistance. We also might anticipate a longer-term

utilization before being assisted by PRA. There were few

impact of the PRA program on healthcare utilization to

statistical y significant differences when we compared

emerge once PRA residents have been in the program for

PRA residents’ rates of healthcare utilization during PRA

longer periods and more households are assisted through

tenancy to a matched comparison group of residents

the program national y. This study only analyzed post-

of other HUD-assisted housing and to a group who

occupancy healthcare utilization for PRA residents on

were enrol ed in Medicaid but not receiving any rental

average less than a year after initial occupancy and only

assistance through HUD programs. These findings

for a small percentage of households the PRA program

suggest that PRA residents did not have substantial y

will eventually assist.

different patterns of healthcare utilization after entering the

PRA program than they would have had in a different living

situation.

56

H U D S e c t i o n 811 P R A P ro g r a m – P h a s e I I E v a l u a t i o n

Chapter 8. Economic Study of the PRA Program

Economic Study of

$12,052 total annual average per-unit rental plus capital

subsidy provided by the PRAC program.48

the PRA Program

• Early evidence suggests that healthcare costs for PRA

residents, as compared to PRAC residents, may shift

away from inpatient hospitalization and long-term

inpatient care towards greater HCBS costs.

• PRA grantee and state agency partner average

The Section 811 PRA program is intended to be cost-

total per-unit annual administrative costs were

effective in providing supportive affordable housing to

approximately $5,380 in 2017 and consist primarily

people with disabilities and includes requirements related

of staffing costs. Averaging across the six study

to cost-effectiveness. The program leverages units in

states, grantees costs represent just less than half

high-quality affordable housing developments built with

(46 percent) of the administrative costs. These costs

other capital funding sources and program funds are

substantial y exceed the amount of administrative

allocated through a grant competition that encourages

funds drawn down, and several states report that

states to subsidize more units by negotiating rents that

staffing needs have been higher than anticipated.

are below the local area FMR.47 At the same time, state

housing agency grantees will want to select properties

• Taking the cost and impact findings around housing

that they expect will meet the needs of their target

together, we conclude that the PRA program provides

populations. As a result, the costs of the PRA program can

housing at a comparable or slightly lower cost than

vary, depending on how the grantees and their partners

the PRAC program, which produces housing quality

implement the program in their states.

and experience that is comparable to or slightly less

desirable than the PRAC program. For healthcare

To determine whether the PRA program is meeting

and supportive services, we conclude that, if short-

its cost-effectiveness goals, the team conducted an

term patterns persist, the PRA program has positive

economic study to describe how the PRA program

impacts that are approximately cost neutral. This

requirements drive costs and to discuss differences in

comparable cost-effectiveness of the PRA program

costs between PRA and the PRAC, NED, and other HUD

currently appears to come at a higher administrative

housing assistance programs in the six study states. The

cost. However, given key differences between the PRA

economic study consists of an analysis of what PRA and

and comparison programs, further research is needed

other housing subsidy programs cost and a discussion

to assess the cost-effectiveness of the administrative

of differences across the programs in how costs are

costs.

al ocated between HUD, other federal funding sources,

and state and local agencies. This chapter concludes

8.1 Analyzing Costs and Cost-

with a discussion of the cost-effectiveness of the PRA

program.

Effectiveness of PRA and

To summarize the findings of the economic study:

Comparison Programs

• Average PRA rental subsidies were $586 per unit, per

In this chapter, we report costs associated with assistance

month, which is $69 per month higher than the average

provided to PRA residents in our six study states. We

PRAC rental subsidy and lower than monthly subsidies

developed per-individual, annual cost estimates in four

in NED and other HUD housing assistance programs.

areas: housing rental subsidies, housing capital subsidies,

healthcare and supportive services, and program

• The LIHTC program funds, on average, 72 percent

administration. Where data allow, the study compares

of development costs for properties in which PRA

PRA costs in these four areas to the same costs for

program units are located. This represents a distinct

individuals in the PRAC, NED, and other HUD comparison

shift in the source of housing subsidy relative to the

groups. The study also compares the source(s) of funds or

PRAC program, for which HUD capital grants fund

resources for each type of estimated cost and describes

approximately 84 percent of development costs.

total estimated costs for the four areas combined.

• Total rental plus capital subsidies average $11,810

A detailed summary of our approach to the economic

annual y for each PRA unit. This is similar to the

study, including an overview of methods and data

sources, is in Appendix B.

47 The cost-effective use of PRA funds is a topic in the PRA Notices of Funding Availability: https://www.hudexchange.info/resources/documents/

Section-811-PRA-Demo-FY2012-NOFA.pdf.

48 This study did not gather capital subsidy data for properties housing residents in NED or other HUD programs.

57

H U D S e c t i o n 811 P R A P ro g r a m – P h a s e I I E v a l u a t i o n

Chapter 8. Economic Study of the PRA Program

Approach to Cost and Cost Allocation

Because this leveraging is explicit in the PRA program

Analyses of PRA and Comparison Programs

design, we consider the cost of these subsidies in the cost

The costs of the PRA and comparison group programs

of the PRA program.

depend on the design of each program, as well as how

The PRAC program includes a direct, development-level

programs are implemented. Thus, the role of different

capital subsidy from HUD to the nonprofit sponsor. PRAC

categories of costs differs from program to program.

properties also received additional capital subsidies from

Housing rental subsidies are the primary cost of the

other programs, but to a much lesser extent than PRA

PRA program and the primary or a key cost of each of

properties.50 For both programs, we convert the cost

our comparison programs. The rental subsidies are paid

of the capital subsidies incurred as a lump sum when

monthly to property owners in the PRA program and in

the property was developed to a per-unit, annualized

each of the programs serving our comparison groups.

cost in 2018 using a number of necessary and standard

Because we use household-level administrative data from

assumptions for allocating costs across units and

HUD, computing average per-individual rental subsidy

amortizing the value of grants and low-interest loans

costs is straightforward.

across time.51 We report the share of capital subsidies

49 In order to relate the costs of

different programs to serving comparable people, we

provided by HUD programs, the LIHTC, state and local

compute the average per-unit, per-month subsidy amount

programs, and other sources.

using analysis weights derived using the same propensity-

Healthcare and disability-related supportive services

score matching approach as we used in the impact

costs are not directly subsidized by PRA or any of our

analyses in the seventh chapter, as detailed in Appendix

comparison programs. Housing quality and location can

B. We report sample sizes for the rental subsidy analysis

affect both health and healthcare and supportive services

in Exhibit 8.1. For the cost al ocation analysis, we note

utilization patterns. Because of the resulting possibility

that HUD bears the costs of all rental subsidies that we

for cost-shifting across the housing and health domains,

observe.

we also estimate and compare healthcare and supportive

Average rental subsidies for similar people may differ

services costs for residents in PRA and the comparison

across the programs because of differences in the

programs. Our primary approach for analysis builds

housing itself—that is, the size, quality, or location of

directly on our analysis of healthcare utilization outcomes,

the units—or because of differences in how the unit

by incorporating state-specific average cost estimates into

is subsidized. The PRA and each of the comparison

the utilization analysis. We analyze Medicaid payments on

HUD programs all require households to contribute

fee-for-service Medicaid claims associated with a key set

approximately 30 percent of their income to rent, and

of measures used in the analyses of healthcare utilization:

the programs subsidize the difference between what

inpatient days, emergency department visits, medical

tenants pay and the unit’s rent. The total rent that can

transportation, days of facility-based long-term care, and

be subsidized is subject to a variety of HUD limitations,

case management services as a proxy for HCBS services.

depending on the program. Properties with larger up-front

The estimates of healthcare costs are supplemented by

capital subsidies have lower rents and so require lower

information learned from interviews with state Medicaid

ongoing rental subsidies.

agencies and service providers who work with PRA

applicants and residents. The staff provided information

Housing capital subsidies play an indirect but important

about sources of funding for healthcare and supportive

role in the costs of the PRA program, and are a direct

services utilization that may not be captured by state

expenditure in the PRAC program. PRA grantees use

Medicaid data.

program rental subsidies in affordable multifamily housing

developed with other federal, state, or private sources.

Program administrative costs are the final cost area of

A single development often receives subsidies from

our analysis. We estimate per-individual average annual

multiple programs at the federal and state and local levels.

costs of program administration for the PRA program and

49 Rental subsidy amounts for the PRA and PRAC programs are included in the Tenant Rental Assistance Certification System (TRACS) data, while amounts for NED and other HUD programs are elements of the PIH Information Center (PIC) data. The study reports average subsidy amounts for all households receiving assistance as of March 2018. In both PRA and PRAC, in some circumstances, property owners are eligible for payments when units contracted into the programs go vacant. We also discuss these potential payments in our analysis of rental subsidy costs.

50 Throughout this chapter we refer to “PRA properties” and “PRAC properties” as shorthand for properties that include units receiving subsidies through the respective Section 811 programs.

51 Complete details of our methodology are included in Appendix B. We apportion the initial subsidy among units that vary in size and amortize over the period for which the property provides low-income rental assistance, using a 3.5 percent discount rate (for example, 30 years for LIHTC

funding and 40 years for PRAC grants). We value the entire amount of grants and forgivable loans as a subsidy and estimate the value of assistance provided as low-interest, but repayable, loans as the implied lifetime savings relative to a market-rate loan provided by the lower rate.

58

H U D S e c t i o n 811 P R A P ro g r a m – P h a s e I I E v a l u a t i o n

Chapter 8. Economic Study of the PRA Program

draw comparisons to estimates for PRAC, NED, and other

by HUD and found capital subsidy information for

HUD programs. Our estimates of administrative costs

29 PRAC properties. We were able to review and

include both grantee costs and costs to HUD. To calculate

extract information from grant applications for only

these estimates, we use data col ected from PRA program

a fraction of PRAC properties. Because this review

grantees and partners through a cost and effort survey

provides the only available additional detail on the

and from consultation with HUD staff. Costs estimates for

prevalence of capital subsidy beyond the PRAC

the PRAC program are based on consultation with HUD

grant itself, we also include an analysis of housing

staff, and we additional y reviewed PRAC grantee annual

subsidies in this sample.52

financial report data and publicly available information for

- Combined Capital and Rent Subsidies Analysis:

a limited number of sponsoring organizations’ websites.

To assess the cost of the total combined capital and

Our estimates for NED and other HUD administrative costs

rent subsidy, we are limited to the capital subsidy

are based on prior research on administrative costs for the

analysis samples for which we have information

PHAs that administer these programs and consultation

about both housing subsidy types.

with experts that conducted this prior research (Turnham

et al., 2015). We also examine the extent to which HUD

• Our analysis of healthcare and disability-related

funds cover program administrative costs.

supportive services costs is subject to the same

data limitations as the analysis of tenancy, healthcare

Limitations of the Economic Study

utilization, and neighborhood outcomes. We analyze

The economic study has limitations related to data

a subset of healthcare and supportive services

availability and to the relative newness of the PRA program

utilization selected in part based on information that is

compared to PRAC, NED, and other HUD programs.

consistently coded in state-level Medicaid data across

our study states. In addition to these limitations, our

• Our analysis requires primary data col ection for capital

cost analyses require additional assumptions and

subsidy data, which we conducted before the last

estimates to estimate state-specific costs for our

extract of administrative data on rental subsidies. Also,

utilization measures. These assumptions, detailed in

the study only had access to capital subsidy data for

Appendix B, add some uncertainty to our findings.

a limited sample of PRAC properties. To make the

most of the available information on both rental and

• To estimate administrative costs of the PRA program

capital subsidies, we used multiple analysis samples

we rely on primary data col ected from grantees to

in our housing cost analyses. In each case, we use

directly estimate staffing and other costs. The study

the largest sample size for which the relevant data is

does not have primary data for PRAC or our other

available:

comparison programs for administrative staffing

at the grantee level. Rather, we rely on information

- Rental Subsidy Analysis: HUD administrative data

from HUD on costs associated with applying for and

provides information on rental subsidies, which we

administering grant funds, and published estimates

used to analyze data for all individuals in our analysis

of administrative costs associated with other HUD

groups as of March 2018.

programs.

- Capital Subsidy Analysis: No central database

documents capital subsidies for low-income

8.2 Costs of Rental Subsidies

housing. For PRA properties, the study team

The PRA program is primarily a rental subsidy program, in

col ected detail on capital subsidies from LIHTC

that program funding is devoted solely to rental subsidies

cost certification and applications for each of the 41

and program administration. In this section, we report the

properties that had any occupied PRA unit as of the

average cost of the ongoing rental subsidies that are paid

summer of 2017. Details on the PRAC capital grant

monthly to property owners in the PRA program and each

are available in HUD administrative data for all PRAC

of the programs serving our comparison groups. To make

properties, so we analyzed total PRAC program

the averages for PRA residents and those in the other

subsidies for this sample of 440 properties. Some

programs comparable, we compute the averages using

PRAC properties also received additional capital

analysis weights derived using the same propensity-score

subsidies from other sources. The study team

matching approach that we used in the impact analyses in

reviewed a number of PRAC applications provided

the fourth through seventh chapters.

52 We targeted PRAC properties for capital subsidy data research that had a high incidence of residents that our propensity score matching model indicate were similar to PRA residents. The 29 properties for which we recovered data may not be representative of the larger PRAC analysis sample.

59

H U D S e c t i o n 811 P R A P ro g r a m – P h a s e I I E v a l u a t i o n

Chapter 8. Economic Study of the PRA Program

Average Monthly Per-Unit Rental Subsidies

a 25th percentile of $443, a median of $548, and a 75th

in the PRA Program

percentile of $695. At $936 a month, Maryland has a

Average PRA rental subsidies were $586 per month

substantial y higher average monthly rental subsidy than

for the 540 PRA residents in HUD administrative

any of the other study states. Maryland’s PRA program

data as of March 2018, but subsidies vary among

focused on the Washington and Baltimore metropolitan

states (Exhibit 8.1). Monthly rental subsidies ranged

areas, which have relatively high rents among the areas

from a low of about $100 to a high of about $1,300, with

served by our sites.

E xhibit 8.1: Monthly Rent al Subsidy and C ontract Rent for Unit s in P R A Program

Estimated

Average

Average

Rental

Average

Subsidy

State

N

Subsidy

Contract Rent

(FY12)

Average FMR

Average SAFMR

California

83

$538

$798

$705

$1,157

$928

Delaware

52

$587

$766

$502

$971

$870

Louisiana

158

$538

$720

$597

$757

$649

Maryland

48

$936

$1,187

$1,055

$1,436

$1,755

Minnesota

115

$596

$761

$503

$793

$817

Washington

84

$506

$725

$354

$749

$729

Total

540

$586

$787

$889

$901

Sources: TRACS Household Data (rent statistics) as of March 2018; Section 811 PRA Cooperative Agreements for grantee projected average rental subsidy as of March 2018; published FMR and SAFMRs for 2017.

The PRA program includes a provision to al ow vacancy

unit’s rent for up to 60 days. The California state housing

payments—compensation to owners when a unit

agency does not provide for vacancy payments at al .

contracted into the program goes unfil ed for a prolonged

Our interviews with state agency staff indicate that only

period. Grantees can provide vacancy payments to

Delaware had made substantial vacancy payments at the

owners in the form of rental assistance payments for up

time of the interviews.53

to 80 percent of the unit’s rent for a maximum of 60 days.

Each state can determine whether or not to offer vacancy

Comparison of Actual and Estimated

payments, and owners must certify that they could not

Average Rental Subsidies

place anyone else in the unit for the vacancy period.

Average PRA rental subsidies are close to or below

Owners can request vacancy payments for units held

grantee expectations. HUD established incentives

open while grantees or partners are locating a program

in the FY12 and FY13 NOFAs to encourage applicants

participant that is eligible for a newly available unit or after

to propose lower per-unit subsidy costs than would

a PRA resident moves out.

be required if the PRA rents were based on FMR, the

Vacancy payments can result in additional costs to the

maximum rent allowed in the program. Several grantees

rental subsidies we report in Exhibit 8.1. In the six study

proposed maximum PRA rents below FMR. Given these

states, only two states (Delaware and Washington)

incentives, grantees estimated the expected average

adopted policies to allow owners to request the maximum

rental subsidies in their Cooperative Agreements with

vacancy payments allowed. In Maryland and Louisiana,

HUD.

owners cannot request vacancy payments until after a

Comparing actual averages to these expectations is one

unit has been vacant for at least 60 days, and owners

way of assessing the program goal of lowest feasible

in Minnesota can only receive up to 60 percent of the

53 In Delaware, we estimate that these payments added between $25 to $50 per month to the average monthly per-unit subsidy cost. The share of available units that were vacant in Delaware was high in the first year of the PRA program, but has fallen to below 10 percent in the most recent quarterly reports, suggesting that these payments may be unique to the start-up phase of the program in Delaware. Louisiana, Minnesota, and Washington reported only a few vacancy payments and have a small share (typically less than 5 percent) of available units reported as vacant in quarterly report data snapshots. California and Maryland reported they had not made any vacancy payments (and quarterly report data snapshots indicate few, if any, vacant units in these states at any given point in time).

60

H U D S e c t i o n 811 P R A P ro g r a m – P h a s e I I E v a l u a t i o n

Chapter 8. Economic Study of the PRA Program

costs.54 We caution that this comparison may also reflect

levels for assisted units in the neighborhoods in which the

whether grantees that serve multiple rental markets have

properties are located. Washington also has a gross rent

been more successful in contracting and leasing up units

that is close to both FMR and SAFMR. That average PRA

in higher-rent markets or lower-rent markets. Exhibit

rents are lower than average FMRs is expected given that

8.1 reports grantee’s expectations for average rental

PRA units are required to be in properties with capital

subsidies. California’s subsidy costs are well below their

subsidies that come with affordability restrictions and

estimate, while all other states’ actual average costs are

below-market rents.

reasonably close (within a standard deviation) to their

estimated costs.

Comparison of PRA, PRAC, NED, and Other

HUD Monthly Per-Unit Rental Subsidies

Comparison of PRA Contract Rent

In most states, average monthly rental subsidies are

to Fair Market Rent

similar to those supporting similar people (based on

The study also compared average monthly rental

multivariate propensity score modeling) in PRAC, NED,

subsidies plus total tenant payments to Fair Market Rents

and other HUD programs.55 Exhibit 8.2 reports average

and Small Area Fair Market Rents. Comparing the gross

subsidies for each of the comparison programs. Key

rent (rental subsidy plus the tenant rent portion) to FMR

findings shown in the exhibit are that:

and SAFMR provides an indication of how the rent for

• The average PRA monthly per-unit rental subsidy

the units compares to prevailing rents in the metro area

is $69 higher than the propensity-score weighted

(FMR) and neighborhood (SAFMR). FMRs or SAFMRs

average for our PRAC comparison sample, but the

are used to determine the rents that are affordable to

difference varies across states.

people assisted by NED and the Housing Choice Voucher

program. HUD calculates and publishes FMRs as

• The average monthly per-unit rental subsidy for NED

estimates of the 40th percentile of rent of recent movers in

is $70 higher than for PRA across the full sample,

each metropolitan area. SAFMRs adjust FMRs to account

with the difference driven largely by California and

for more and less expensive ZIP Codes.

Maryland.

Average gross rents for PRA units are, for the most

• Average monthly subsidies for the other HUD

part, well below the FMR and SAFMR. Exceptions are in

comparison group are higher than or similar to for PRA

Louisiana, where gross rents are actual y above SAFMRs,

across all of the states.

indicating that rents may be slightly above usual rent

E xhibit 8.2 : Monthly Rent al Subsidy for P R AC, N ED, and O ther H U D

PRAC

NED

Other HIUD

St.

St.

St.

State

N

Mean

Dev.

N

Mean

Dev.

N

Mean

Dev.

California

827

$596***

(260)

873

$809***

(506)

27,074

$771***

(471)

Delaware

163

$458

(281)

<10a

a

a

312

$666***

(183)

Louisiana

577

$405***

(140)

263

$517***

(114)

6,189

$494

(149)

Maryland

963

$557***

(305)

320

$934***

(302)

4,512

$837***

(398)

Minnesota

305

$438***

(216)

<10a

a

a

4,701

$472***

(161)

Washington

356

$550

(240)

946

$546

(237)

6,085

$523***

(192)

Total

3,191

$517***

(255)

2,523

$656

(329)

48,873

$636

(345)

a Values not reported due to small sample reporting restrictions.

*** Different from the same (overall or state) PRA average with p-value<0.001.

Note: Values are calculated using propensity-score weights based on demographic and health history characteristics.

Source: TRACs and PIC Household Data (rent statistics) as of March 2018.

54 All actual averages are statistically significantly different from the grantee estimates. This is not surprising because we would not expect grantee forecasts to be precise forecasts of subsequent rents.

55 The PRAC program also has provisions for vacancy payments should a unit go unfilled for a prolonged period of time. Further research focused on the PRAC program would be needed to determine whether such payments meaningfully affect program costs.

61

H U D S e c t i o n 811 P R A P ro g r a m – P h a s e I I E v a l u a t i o n

Chapter 8. Economic Study of the PRA Program

8.3 Capital Subsidy Costs

not research and analyze capital funding prevalence in

these other programs. For NED and the Housing Choice

Unlike the PRAC program, the PRA program does not

Voucher program, a property need not have received a

fund capital subsidies. Rather, units that are contracted

capital subsidy in order for the rental subsidy to be used.

into the program are in multifamily developments in which

other capital funding sources subsidize the development

Sources of Capital Financing for

costs. These sources include LIHTC, HOME funds, the

PRA and PRAC Properties

Community Development Block Grant (CDBG) program,

The primary source of capital financing for properties

and other federal, state, or private sources. Subsidies

with PRA units under contract in the study states

take the form of direct funds (grants or proceeds from tax

is the LIHTC program, which provides, on average,

credit sales), repayable loans with below-market interest

71 percent of development costs (Exhibit 8.3). State

rates, and forgivable loans with low or no interest rates.

and local low-income housing programs (housing trust

Funds can also be an unsubsidized private investment

funds, bonds, and grants) fund an average of 8 percent of

for which debt investors receive mortgage payments and

development costs. HUD programs (HOME and CDBG)

equity investors receive future rental income or maintain

make up 5 percent, and other federal agencies and private

an ownership interest in the property. The construction

grants make up 4 percent of costs. The remaining 12

or substantial rehabilitation of PRA properties are almost

percent of capital financing for PRA properties represents

always funded from a variety of sources. Properties

a combination of private investor funds and some

financed with subsidized capital subsidies have rent limits

unsubsidized loans.56

and affordability restrictions for some specified period.

About 60 percent of PRA units are in properties that

In contrast, the PRAC program included both rental and

received some capital funding from state or local low-

capital subsidies. PRAC capital subsidies were expected

income housing programs, although these other sources

to cover all or nearly all capital costs, although the

provide a smal er share of financing. Approximately 57

program did not prohibit owners from accessing other

percent of PRA units are in a development that had more

subsidies, grants, or private investment. Rental assistance

than a trivial amount of unsubsidized capital investment.

provided through NED and other HUD programs may

More than 40 percent of units are in developments that

also be used in properties that receive LIHTC, HOME,

used loans or grants from other HUD programs, and 37

CDBG, Project-based Section 8, Public Housing, or a

percent had some other form of grant or subsidized loan.

variety of other capital subsidies. In this evaluation, we do

E xhibit 8.3 : S ources of C apit al Subsidy C ost s for P R A and P R AC Proper ties

Unsubsidized, 3%

Unsubsidized, 12%

Other, 1%

LIHTC, 0%

State/local, 8%

Other, 4%

HUD-other, 3%

State/local, 8%

HUD-other, 5%

HUD-PRAC, 0%

LIHTC, 71%

HUD-PRAC, 84%

PRA

PRAC

Note: Percentages may not add to 100 percent due to rounding.

Source: Capital cost data collected for the cost analysis from LIHTC cost certifications and applications and PRAC grant applications in six study states.

56 The LIHTC program can include private investment that has an interest in rental revenue both during the period of affordability restrictions and beyond.

62

H U D S e c t i o n 811 P R A P ro g r a m – P h a s e I I E v a l u a t i o n

Chapter 8. Economic Study of the PRA Program

PRAC capital grants provide, on average, 84 percent

lives in units that have more than a trivial amount of

of development capital for PRAC developments

unsubsidized capital investment.

where our comparison sample lives. State and local

The per-unit value of development capital subsidies

grants or subsidized loan programs account for the next

for units in the PRA program is, on average,

largest share, at 8 percent, while other HUD programs

approximately $5,000 annually (Exhibit 8.4). The

provide 3 percent of PRAC development capital funding.

annualized value of PRAC program capital subsidies

Other sources (for example, other federal agency or

is $7,662 per unit for the sample for which we have

private grants) provided 1 percent of capital funding. We

ful information on capital funding sources. Since this

identified only one PRAC property that also had LIHTC

represents a fraction of PRAC properties (due to issues

funding, and the LIHTC share of the capital funding for

with availability of data), we also report the annualized,

this property was a modest 13 percent. The remaining 3

per-unit average PRAC capital grant from HUD for al

percent of capital development funding represents some

PRAC properties in our analysis. On average, the PRAC

combination of owner investment or unsubsidized loans,

grant provides nearly $6,000 per unit in annualized capital

and only 24 percent of our PRAC comparison sample

subsidy.

E xhibit 8.4 : A nnualized Per- Unit C apit al Subsidy for P R A and P R AC

PRA

PRAC—Complete Capital Subsidy

PRAC Grant Only (full sample)

Mean

St. Dev.

Properties

Mean

St. Dev.

Properties

Mean

St. Dev.

Properties

Total annual per-unit capital

$4,969

(3,379)

41

$7,662***

(2,187)

29

$5,851***

(2,151)

440

subsidies

By Category

HUD (not PRAC)

$255

(457)

19

$373

(698)

17

HUD PRAC

--

--

--

$6,346

(1,471)

29

$5,851***

(2,151)

440

LIHTC

$4,016

(2,957)

41

$27***

(177)

1

State/Local

$614

(897)

32

$776

(976)

19

Other

$83

(177)

19

$139

(537)

7

*** Different from the average with p-value <0.001.

Note: Values are calculated using propensity-score weights based on demographic and health history characteristics.

Source: Capital cost data collected for the cost analysis from LIHTC and PRAC grant applications in six study states.

Comparison of Combined Capital and Rent

different than the total annual housing subsidy for PRA

Subsidies in the PRA and PRAC Programs

units. The higher annualized capital subsidy provided by

PRA units receive, on average, $11,810 in total housing

the PRAC grant is offset substantial y by lower ongoing

subsidies annually. PRAC units for which we have

rental assistance. Because details on possible additional

complete capital subsidy detail receive, on average,

capital subsidies are not available for a larger number

$14,322 in total housing subsidies annually. The

of properties, it is difficult to draw a conclusion about

difference is the result mainly of the $2,693 larger average

what is driving differences in rental subsidies for the ful

annual per-unit capital subsidies in the PRAC program.

PRAC sample and the smal er sample with greater capital

For this sample, annual rental assistance differs by only

subsidy detail. However, in both samples, the average

$181, and the difference is not statistical y significant.

annualized value of the PRAC grant plus PRAC rental

subsidy is similar to the average annualized value of the

For the complete sample of PRAC properties, the

total subsidy (PRA rental subsidy plus value of LIHTC and

annualized value of the PRAC capital grant plus

other capital subsidies) for PRA units.

ongoing rental subsidies totals $12,043 (Exhibit

8.5). This combined sum is not statistical y significantly

63

H U D S e c t i o n 811 P R A P ro g r a m – P h a s e I I E v a l u a t i o n

Chapter 8. Economic Study of the PRA Program

E xhibit 8.5 : A nnualized Per- Unit Tot al Housing Subsidy for P R A and P R AC

$16,000

$14,322

$14,000

$12,043

$11,810

$12,000

$10,000

$6,720

$4,016

$5,851

$8,000

$6,000

$4,000

$6,841

$6,660

$6,192

$2,000

0

PRA

PRAC -

PRAC -

N = 41 properties

Complete Capital

Capital Subsity for

Subsity Detail

PRAC Grant Only

N = 29 properties

N = 440 properties

Rental HUD

Capital HUD

Capital LIHTC

Capital State/Local/Other

Note: Values are calculated using propensity-score weights based on demographic and health history characteristics.

Sources: Capital cost data collected for the cost analysis, for example, from LIHTC and PRAC grant applications; TRACS and PIC Household Data (rent statistics) as of March 2018 in six study states.

The available data suggests that the PRA program

on that analysis using estimates of the average fee-for-

provides slightly higher (about $600 annual y) rental

service costs of this utilization to the state Medicaid

assistance than the PRAC program in properties that

programs. The PRA program does not directly support

were built with lower annualized capital subsidies.

these costs. Rather, we include these costs in our analysis

Housing subsidies in the PRA program rely on the

to acknowledge that linkages between housing, health,

LIHTC program for capital subsidies, with HUD

and healthcare utilization may result in costs shifting

funds providing, on average, almost 60 percent of

across the housing and health domains. This linkage is

the total annual housing subsidy. In contrast, HUD

what motivates the partnerships with health and disability

funds provide more than 90 percent of the housing

services agencies that are required of PRA grantees.

subsidies in the PRAC program.

The study does not have access to complete

8.4 Costs of Healthcare and

documentation of healthcare and disability-related

services utilization or costs. Due to data limitations, our

Disability-related Services

analysis provides only a preliminary indication of how

Utilization

comprehensive costs for al healthcare and disability

services utilization likely differ for individuals in the

Successful tenancy in the PRA program could require

PRA and comparison programs. Before presenting the

different levels and types of support than in our

quantitative findings limited to state Medicaid data,

comparison programs. Housing quality and location

we include qualitative findings on the provision of and

may also affect both healthcare utilization and access

use of disability-related services. These findings are

to services. The seventh chapter reported findings on

drawn from study interviews, resident surveys, reviews

patterns of healthcare and disability services utilization

of program documents, and other primary research.

observed in state Medicaid data. This section builds

We discuss the sources of assistance for healthcare

64

H U D S e c t i o n 811 P R A P ro g r a m – P h a s e I I E v a l u a t i o n

Chapter 8. Economic Study of the PRA Program

and disability services for PRA program residents and

transportation, assistance with meals and housekeeping,

comparison group members. We specifical y highlight

and home modifications.57 PRA residents are eligible

costs associated with tenancy-related supports, because

for specific Medicaid HCBS based on their particular

they are most directly linked to successful use of the PRA

disabilities and associated needs. We examine differences

program.

in costs associated with HCBS utilization between PRA

residents and our comparison groups below. Additional y,

Sources of Assistance for Healthcare and

Medicaid’s MFP program pays for housing location and

Disability Services for PRA Residents

transition services for some Medicaid enrol ees who move

Like all low-income adults with disabilities, PRA program

from an institution to a community setting.

residents receive healthcare that may be funded by a

Service Provider Resources

variety of payers, but is primarily funded by Medicare,

Medicaid, or a combination of the two programs.

Several of the service providers we spoke with indicated

Disability-related services, including some tenancy

that their organizations, which rely on Medicaid

supports, are also primarily funded by Medicaid.

reimbursements to cover their costs, were not sufficiently

However, PRA residents and the similar individuals

reimbursed for time spent working with PRA applicants

in our comparison groups may receive services that

or residents. As a result, some providers say their

are not funded by Medicare or Medicaid, for example,

organizations lose money by taking on PRA residents

through other state and local programs or through private

as clients. These providers said their organizations

philanthropy.

had different reasons for accepting these losses and

continuing to serve PRA residents. Reasons include

Medicare

having underestimated the actual time that case managers

Medicare is a federal health insurance program that

would need to spend with PRA residents and having

insures seniors, as well as people younger than 65

related programs that also serve PRA-eligible residents

with disabilities who are approved for Social Security

and are adequately funded. If service intensity continues

Disability Insurance (SSDI). Medicare pays for inpatient

as the program matures, continuing to realize such losses

and outpatient care, prescription drugs, and limited

without additional funding may represent a risk to the

skil ed nursing facility stays. Medicare also pays for home

long-term sustainability of the PRA program.

health services under certain limited conditions. Medicare

Other Paid Sources of Assistance

requires patients to pay a share of their medical costs. In

the case of low-income individuals who also qualify for

Although Medicaid is the main source of funding for

Medicaid coverage, Medicaid will pay the patient’s share.

tenancy supports provided to PRA residents and similar

individuals in our comparison groups, Medicaid does not

Medicaid

cover all tenancy supports. Other sources of funding for

Medicaid is a health insurance program jointly run by

disability services provided to PRA residents are available,

states and the federal government that insures low-

but limited. Federal Projects for Assistance in Transition

income individuals and families. Like Medicare, Medicaid

from Homelessness (PATH) grants provide people with

pays for inpatient and outpatient care and prescription

serious mental il ness experiencing homelessness with

drugs. In addition, Medicaid pays for long-term skil ed

housing locator services.58 A variety of state and county

nursing facility stays, and home and community-based

social services agencies, as well as private philanthropic

services. Within a set of federal parameters, each state

entities, assist those leaving institutions or homelessness

determines the specific benefits available to Medicaid

with transitions to community-based living, by helping

enrol ees and the income limit for Medicaid eligibility. The

residents set up their new homes, establish utilities, and

PRA program requires applicants to be eligible for home

settle into their neighborhoods. PRA residents may receive

and community-based services funded by Medicaid or

tenancy supports not covered by Medicaid from state-

state plan services, but not necessarily to receive these

funded programs, county governments, and nonprofit

services.

organizations. Final y, a few PRA properties employ a

service coordinator who can help PRA and other residents

Importantly for the PRA population, Medicaid covers

gain access to community services.

HCBS, which include both health services and supportive

services. Examples of supportive services that can be

provided using Medicaid HCBS funds are personal care,

57 https://www.cms.gov/Outreach-and-Education/American-Indian-Alaska-Native/AIAN/LTSS-TA-Center/info/hcbs.html

58 https://www.samhsa.gov/homelessness-programs-resources/grant-programs-services/path

65

H U D S e c t i o n 811 P R A P ro g r a m – P h a s e I I E v a l u a t i o n

Chapter 8. Economic Study of the PRA Program

Informal Caregivers

In reality, there may be differences in utilization that we

The survey of Section 811 residents found similar rates

do not observe as a result of limitations of the available

of PRA and PRAC survey respondents receiving

data. Our findings more likely represent a limited view

regular help from family and friends. Many PRA and

into differences in average healthcare and supportive

PRAC residents have their own support networks—

services costs between PRA residents and individuals in

family or friends who may help with everyday activities or

our comparison groups. At a minimum, our approach and

otherwise provide assistance. We note that this interaction

findings provide a framework for comparing healthcare

is an important source of assistance and contributes to

costs for individuals in different assisted housing

PRA and PRAC program resident wel -being. We do not

situations for future analyses with more comprehensive

measure any difference between PRA and PRAC residents

data.

in these informal support networks, and this assistance

In the first year of PRA assistance, PRA residents have

does not represent a cost to the government. Therefore,

different patterns of healthcare and services costs

we do not assess a monetary value to the assistance that

than similar individuals in our comparison groups.

informal caregivers provide for our economic analysis.

Exhibit 8.6 reports our cost estimates for four healthcare

utilization measures. Compared to PRAC residents, PRA

Costs of Relevant Healthcare and Disability-

residents have lower costs of inpatient hospital stays,

Related Services Assistance through

and higher costs of medical transportation and HCBS

Medicaid

use, although none of these differences are statistical y

In this section, we provide evidence on the costs of

significant.

healthcare and disability-related service utilization that we

PRA residents have higher costs of emergency

observe in state Medicaid data for individuals in the PRA

department visits—$1,094, compared to $756 for

program and individuals in our comparison groups. Before

PRAC residents, a statistically significant difference.

presenting costs, we repeat that, because of limitations in

PRA residents have much lower costs related to long-

the availability of data on all utilization of healthcare and

term inpatient stays than PRAC residents—$299,

supportive services, this analysis includes only a subset

compared to $5,059 for PRAC residents, a difference

of the utilization observed in state Medicaid data. We

that is also statistically significant. The PRA and PRAC

focus our analysis on a subset of utilization that is likely

sums of the cost estimates for this subset of healthcare

to be affected by housing location and quality or access

and supportive services are within $1,000 of each other.

to services—the outcomes that may differ for individuals

Although we do not draw definitive conclusions with

assisted by PRA as opposed to PRAC or our other

regard to total healthcare utilization costs, this relatively

comparison groups—and for which we have consistently

small difference indicates that PRA is approximately

defined measures across state Medicaid data.

cost-neutral to Medicaid relative to PRAC in terms of this

The interpretation of our findings depends on whether

selected subset of healthcare and supportive services

differences in PRA residents’ healthcare utilization relative

costs.

to that of the people in our comparison groups occur

PRA residents in our analysis have somewhat lower

mainly in the measures of utilization and costs that we

utilization costs for each of our measures than their NED

are able to examine. Similarly, the interpretation also

counterparts, but none of the differences are statistical y

depends on whether differences in case management

significant. Costs for the other HUD comparison

utilization are a good proxy for differences in overall HCBS

group have the reverse relationship. PRA residents

utilization. If these assumptions are reasonable, then our

have somewhat higher utilization costs for each of our

estimates capture differences in overall healthcare and

measures than other HUD residents, but, again, none of

services utilization costs for PRA and PRAC estimates.

these differences are statistical y significant.

66

H U D S e c t i o n 811 P R A P ro g r a m – P h a s e I I E v a l u a t i o n

Chapter 8. Economic Study of the PRA Program

E xhibit 8.6 : Healthcare and Disabilit y- Related Ser vices C ost s

Exhibit 8.6: Healthcar

for P R A , P R AC, N ED, and O ther H U D Resident s

e and Disability-Related Services Costs for PRA, PRAC, NED, and Other HUD Residents

Inpatient admissions

Emergency room visits/no inpatient admission

PRA

PRAC

Medical transportation trips

NED

Long-term inpatient stays

Other HUD

Any case management

$0

$10,000

$20,000

$30,000

$40,000

$50,000

Note: PRA residents moved in during 2016 (or first two quarters of 2017 for Minnesota and Washington). PRAC, NED, and other HUD residents moved into their unit during 2013–2016 (or first two quarters of 2017 for Minnesota & Washington).

Sources: Abt Associates analysis of TRACS and PIC household data from the quarter ending on March 31, 2018; and Medicaid claims and encounter data for 2015.

8.5 Costs of Program

has costs associated with implementation of the PRA,

Administration

which we discuss along with estimates of administrative

costs associated with PRAC and the other comparison

The cost of labor to carry out the PRA program makes

programs.

up the vast majority of administrative costs. This section

PRA Program Staffing in State Housing and

describes how state agencies staff their programs and the

Health Agencies

level of effort that was reported as necessary to conduct

al activities outlined in their Interagency Partnership

The state-level partnerships in the study states typical y

Agreements. We then report total average administrative

consist of the state housing agency grantee and the state

cost estimates for PRA programs, which include direct

Medicaid agency. California and Maryland are exceptions

costs (travel and program-specific software) and

in that they include additional partner agencies. Exhibit 8.7

estimates of indirect costs (for example, space, materials,

shows the grantee and state housing and health agency

management) required to support these staff. HUD also

partners.

E xhibit 8.7: Section 811 P R A Program S t ate Agencies in Six S t ates

State

Grantee and State Housing Agency Partners

State Health Agency Partners

California Housing Finance Agency (Grantee)

Department of Health Care Services

California

Department of Housing and Community Development

Department of Developmental Services

Tax Credit Allocation Committee

Delaware

Delaware State Housing Authority

Department of Health and Social Services

Louisiana

Louisiana Housing Authority

Louisiana Department of Health and Hospitals

Maryland Department of Health and Mental Hygiene

Maryland

Maryland Department of Housing and Community Development

Maryland Department of Disabilities

Minnesota

Minnesota Housing Finance Agency

Minnesota Department of Human Services

Washington

Washington Department of Commerce

Department of Social and Health Services

Source: Abt analysis of Section 811 PRA Cooperative Agreements.

67

H U D S e c t i o n 811 P R A P ro g r a m – P h a s e I I E v a l u a t i o n

Chapter 8. Economic Study of the PRA Program

PRA programs are usually led by a PRA coordinator

On average, state agencies report an estimated

in the state housing agency, with a lead counterpart

combined 1.9 FTEs to administer the PRA program

in the lead state health agency. PRA programs are

in 2017 (Exhibit 8.8). The number of FTEs reflects the

typical y coordinated by one point person, the Section 811

number of partnering state agencies and whether some

PRA Coordinator, in both the state housing agency and

PRA activities are conducted by other state agencies

the partnering state health agency. The coordinators are

(or in one case, a contractor). On average, state housing

assisted by other parts of their organization or partnering

agency staff (the grantees) reported that it took 0.9 FTEs

agencies in awarding units, processing rental assistance

to administer the program in 2017, ranging between 0.4

payments, inspecting properties, and grant program

and 1.2 FTEs. PRA Coordinators spend between 30 and

monitoring. Those who lead or support the PRA program

100 percent of their time administering the program. Other

typical y do not work full time on PRA and have additional

state housing agency staff that spend smal er percentages

responsibilities within their agencies. In the discussion

of time on PRA include, by function, accounting, housing

below, we report PRA staffing as ful -time equivalents

transition and contracts, housing inspectors, asset

(FTEs).

management, and accounting and budgeting.

E xhibit 8.8 : S t ate Housing and Health Agency S t af f

Administering the P R A Program, in Full-Time Equivalent s

State

FTE Grantee

FTE State Partners

Direct Costs and Uses

Direct Costs and Uses Detail

California

0.9

2.3

$2,000

Travel

Delaware

1.1

0.4 (estimate)

Not reported

Louisiana

1.2

1.1

$3,600

Travel, software licenses

Travel, software and IT,

Maryland

1.0

3.1

$33,380

marketing, translation

Minnesota

0.4

0.8

$2,500

Travel

Washington

0.6

0.4

$2,500

Travel

Average

0.8

1.3

$7,700

Note: Average is weighted by the number of PRA residents in each state.

Source: Data collected directly from grantees and program partners as part of the evaluation’s data collection efforts.

Staffing for PRA in the state health agencies has a wider

spend on the program has decreased somewhat as the

range. State health agencies report an average of 1.3

work of identifying units and entering into contracts with

full-time employees to administer the PRA program,

owners has been ramping down. Other grantees say they

ranging between 0.4 and 3.1 FTEs per state. Staffing is

expect their PRA work will stabilize after all the units are

largely reflective of the number of divisions or departments

leased for the first time, but they expect the level of effort

within the state health agencies involved with PRA, which

needed will remain high until then. As states get more

itself reflects the populations targeted for PRA in each

units under contract, the work at the state housing agency

state.

is beginning to transition to ongoing program maintenance

Staff from state agencies on both the housing and health

through monitoring reviews and physical inspections.

sides report that the PRA program continues to be

Throughout, the state housing agencies stil have the

administratively burdensome and take more staff time

ongoing monthly work of processing rental assistance

than the agencies had anticipated when they applied for

payments from HUD to owners.

the PRA grant. State housing agency grantees report

In contrast, state health agency staff reported that they do

that staffing needs are leveling off as properties

not expect their level of effort to decrease that much as

become leased, but state health agencies and

the program matures. These staff members support the

program partner staff do not.

tenant selection process, which will continue as PRA units

In grantee interviews, staff from two of the state housing

turn over, as well as ongoing tenancy. Both housing and

agencies reported that they think the amount of time they

health agency staff report that the volume of PRA work

can fluctuate throughout the year, sometimes significantly.

68

H U D S e c t i o n 811 P R A P ro g r a m – P h a s e I I E v a l u a t i o n

Chapter 8. Economic Study of the PRA Program

Many property staff interviewed also reported that the

level costs for property owners, or service-provider

program was more administratively burdensome than

administrative costs, however. Rather, we assume

other rent subsidy programs they administer, such as

that these are costs of doing business that are partly

project-based Section 8 or LIHTC. Several factors may

reimbursed by the rental payments or Medicaid service

contribute to this perception. Unlike with other subsidy

payments that we include in previous sections of our cost

programs, PRA payments are not automatic; owners need

analysis. 59

to submit subsidy information to the state by a certain

Grantee and partner agency average total per-

date each month. The PRA may also have different income

unit annual administrative costs in 2017 were

requirements or calculations than other programs in which

approximately $5,350, with grantees’ share of these

owners participate. While the related verifications and

costs at approximately 46 percent. 60 This estimate

calculations may not necessarily be more difficult or time-

includes staffing, direct costs for software, travel, and

consuming for owners for the PRA program than they are

third party contractors, and estimates of indirect costs

for other programs, introducing the new and sometimes

for the additional resources that support the program

unique requirements of an additional program to owners

(for example, office space and agency management).

or inexperienced staff added a burdensome layer of

Variation in costs reflects variation in the makeup of

complexity.

the partnerships and in the target populations, which is

Some of the grantees anticipated that participating in

detailed in Chapter 2.

PRA would require additional work by property owners.

Maryland is an outlier in terms of per-unit annual

Owners in Minnesota and Delaware were required to

administrative costs, which reflects both the involvement

have experience with TRACS, the software system used

and coordination among multiple partner agencies and

to report tenant data to HUD in order to process subsidy

that the state had only 50 PRA units under lease in 2017.

payments. In Louisiana, the state housing authority enters

Maryland has the lowest grantee share of administrative

and processes data in TRACS on behalf of property

costs at 26 percent, reflecting the coordination of

owners who do not have access to the software.

multiple partner agencies. California has the second-

Estimated Annual PRA Program

highest administrative costs per unit, and similarly takes

Administrative Costs

a coordinated multi-agency approach reflected in the

second-lowest grantee cost share of 33 percent, but with

Exhibit 8.9 reports the costs associated with PRA

more units under lease at the time of data col ection on

program staffing and the additional direct and indirect

administrative costs. The lowest per-unit administrative

costs of administering the PRA program. Property

costs are in Washington, which has relatively lean staffing

owners and supportive services providers also incur

and 61-percent grantee share, and Louisiana, which has a

some administrative costs in housing PRA residents

large number of units under lease and an almost even split

(and individuals in our comparison groups) and providing

of grantee and partner administrative costs.

services to them. We do not assess any program-

59 Administrative cost overruns for specific PRA units or clients may be absorbed by rents for other units or claims for other clients. Alternatively, added costs could jeopardize owners’ participation in the program and providers’ ability to serve PRA residents. Both possibilities are left to future research. Most property owners we spoke with did not identify additional costs for complying with the PRA program. Owners and managers said that the time spent assisting PRA residents was about the same as time spent assisting other residents. There are a few areas where some managers did report spending additional time on the program relative to other assisted units, however.

60 We asked grantees and partners for current staffing levels and annual expenditures on direct costs in our data collection. Some grantees reported detail for the fiscal year that mostly overlapped with data for 2017.

69

H U D S e c t i o n 811 P R A P ro g r a m – P h a s e I I E v a l u a t i o n

Chapter 8. Economic Study of the PRA Program

E xhibit 8.9 : E stimated A nnual Grantee and S t ate Agency Par tner P R A Program Administrative C ost s California

Delaware

Louisiana

Maryland

Minnesota

Washington

All

Per-unit annual program administrative

costs (grantee and state agency

$7,909

$6,029

$2,860

$13,137

$3,214

$2,452

$5,345

partners)

Units under lease (September 2017)

60

35

98

50

76

56

375

Approximate grantee share of

33%

72%

53%

26%

36%

61%

46%

administrative costs

Projected units under contract by end

195

121

224

174

152

115

981

of September, 2019

Lower bound 2019 projected per-unit

under RAC or leased annual program

$2,433

$1,744

$1,251

$3,775

$1,607

$1,194

$2,043

administrative costs

Notes: Units under lease from grantee quarterly budget reports for quarter ending September 30, 2017, in six study states. Project units under contract by end of September 2019 from quarterly budget reports ending September 30, 2018. Lower bound projection assumes that administrative staffing remains at FY 2017 levels while units leased increase to projected levels.

Source: Data collected directly from grantees and program partners as part of the evaluation’s data collection efforts.

With a substantial share of rental assistance grant funds

Sources of Funding for PRA

remaining, PRA program grantees project substantial

Program Administration

increases in units in the program. Grantees FY 2018

HUD pays grant administration fees to grantees along

budget reports project the units under executed rental

with funding for unit rental assistance. While these fees

assistance contracts by the end of FY 2019 (September

offset much of the costs of grant administration, the

30, 2019). This projection provides an alternative base for

PRA program also relies on additional grantee agency

characterizing per-unit administrative costs. We use this

resources and partner agencies’ resources, including

alternative base to project a lower bound average cost per

general state funds, to administer the program.

unit under RAC for 2019, which we also report in Exhibit

8.9. This is a lower-bound estimate, because it assumes

PRA Grant Administrative Fees

administrative resources remain at the levels we measured

Grantees can use up to 8 percent of their PRA grant funds

(which were representative of FY 2017 costs) while units in

to pay for direct and indirect costs related to the PRA

the program increase to projected levels needed to utilize

program. PRA grant administrative funding is provided

the committed rental subsidy funds.

both in the initial funding award and in subsequent grant

In addition to staffing and indirect costs, grantees also

renewals. As reported in Exhibit 8.10, together, the study

incur direct costs for the PRA program, with details listed

grantees have drawn down 42 percent of their total

in Exhibit 8.8. On average, the state PRA programs

awarded grant administrative fees through September

(grantees and partners) report total direct costs of

2018, while drawing down 9 percent of total awarded

about $7,700 per year. Direct costs include training and

rental assistance funds. Grantees used 8 percent of al

travel to attend PRA training opportunities and meetings,

administrative fees (for both FY12 and FY13 grants) in

production and translation of program documents and

2015, while leasing a total of 16 units. As expected for a

marketing materials, software for tracking and submitting

new program, the number of units increased rapidly at

resident and rental assistance payment information,

first, and then more slowly, with a total of 480 units under

and infrastructure and supplies. In some larger states,

lease by the end of 2018.

agency staff incur costs to travel from the capital, where

A comparison of Exhibits 8.9 and 8.10 shows that the

the state agencies are typical y located, to other places

study’s estimated costs ($5,532 annually per leased

in the state to recruit owners and residents and to meet

unit) of administering the PRA program, based

and train service provider organizations. In Washington,

on reported state agency and partner staffing,

the state housing agency engages a third-party contractor

substantially exceed the amounts of administrative

to administer its rental subsidies. These costs are both

funds drawn (per leased unit). We estimate an ongoing

small on a per-unit basis and likely to be relatively fixed as

annual cost per unit of more than $5,000, while the

programs continue to add units.

approximate annual per-unit administrative fee drawn

70

H U D S e c t i o n 811 P R A P ro g r a m – P h a s e I I E v a l u a t i o n

Chapter 8. Economic Study of the PRA Program

E xhibit 8.10 : Section 811 P R A Grant Administrative Fees Awarded and Received, Program Year s 2015 -2018

2015

2016

2017

2018

Total

Total administrative fees drawn (six states)

$434,727

$611,548

$655,621

$429,914

$2,231,267

Approx. 1,000

# of units leased (cumulative)

16

145

375

480

unit years

Average administrative fee per units leased

$27,170

$4,218

$1,748

$896

$2,231

Percentage of total al ocated administrative

8%

11%

12%

8%

42%

fees received

Source: Analysis of grantee 2015–2018 budget reports through the quarter ending September 20, 2018, in six study states.

down is less than $2,231. In the study states, state health

through MFP demonstration funds wil be incorporated

agencies and other program partners do not receive any

into the agency operating budget.62

PRA administrative fee. Several states acknowledged

Because PRAC does not require formal partnerships

in their grant applications that they expected that their

between housing providers and the state agencies, we do

administrative costs would exceed the administrative fees

not consider any administrative costs for health agencies

(at the time set at 5 percent) and committed to providing

that direct supportive services that reach PRAC residents

the difference as in-kind leverage. These states report

to be costs of the PRAC program itself. In contrast, such

that they underestimated the actual staff time needed

partnerships are required in the PRA program, and we

to administer the PRA program and are covering more

include the cost of staff time and other resources from

staff time out of other programs than planned. Per-unit

these partners to administer and coordinate the PRA

administrative costs may, however, decrease as the

program in our administrative cost estimates.

number of leased units increases and the PRA program

matures, both at the federal level and within each state.

Costs of HUD’s Administration of the

State Housing Agency Resources

PRA Program and Administrative Costs in

Comparison Programs

While the entire administrative fee goes to the state

housing agencies (and in one case, their contractor),

Estimated costs of HUD’s administration of the PRA

most state housing agencies have found that, so far,

program are approximately $435 per unit annually

administrative fees received as part of the PRA grant have

for the FY12 and FY13 grants. This estimate includes

not covered their administrative costs associated with

the HUD staff that manage the program, interact with

implementing the grant.61 State housing agencies report

grantees, and manage administrative data flows for

that they supplement the administrative fees with other

the program. The cost estimate also includes technical

agency resources such as unrestricted operating funds,

assistance to PRA grantees provided by a contractor,

income-generating programs, and other HUD programs

as well as the costs to both applicants and HUD staff

such as CDBG and HOME.

associated with the estimated level of effort needed for the

grant solicitation and procurement process.

State Health Agency Resources

PRAC, NED, and other HUD programs also have

Although the state health agencies do not receive any of

administrative costs for HUD and for grantees.

the administrative fees, they usual y conduct the ongoing

Administrative cost estimates for these programs are

and often time-consuming work of tenant selection

expected to differ from estimates for the PRA program for

and matching of applicants to available units. State

three reasons:

health agencies report that the salary and benefits of

• The PRA program is relatively new and is still

PRA program staff are paid out of other state sources,

incurring start-up administrative costs, while

including Medicaid MFP, other Medicaid funds, and

other HUD programs are well established. Initial

unrestricted state funds. The state health agencies in our

years of HUD’s administration of the program included

study said that, going forward, positions previously funded

61 In one state, the grantee paid for some of the staffing costs of a PRA Coordinator in the state health agency until the position could be incorporated into the agency budget.

62 We have included staff positions funded by MFP and Medicaid only when these staff are engaged in PRA program administrative efforts, such as coordinating referrals of PRA residents, managing wait lists, selecting and contracting units, and coordinating the program across agencies. Case management functions were not included in administrative costs but instead are considered to be supportive services.

71

H U D S e c t i o n 811 P R A P ro g r a m – P h a s e I I E v a l u a t i o n

Chapter 8. Economic Study of the PRA Program

developing processes and policies to support

PRAC owners also play a role in administering the

grantees.63 In contrast, PRAC and NED and other

rental assistance that funds units in their developments.

HUD programs have well-established systems and

While we did not interview, or otherwise col ect primary

procedures after decades in operation.

data from, PRAC grantees for this study, our review of

• HUD programs are intentionally structured

grantee annual financial statements suggests that costs

differently. The PRA model is based on formalized

associated with such administration are funded by the

state agency partnerships that contract units and

ongoing rental subsidies for the units. These costs are

manage ongoing rental payments to places that

included in the costs of rental subsidies developed above.

house program residents while maintaining access to

We estimate that the costs to HUD of administering

services. Under the PRAC model, HUD provides funds

NED and other HUD programs are about $62 per

directly to housing providers, both for initial property

unit per year. This rough estimate is based on the total

development and as an ongoing rental subsidy.

salaries and expenses of the HUD office that administers

Because of HUD’s substantial capital investment, the

HUD’s public housing, tenant-based rental assistance, and

agency monitors property finances on an ongoing

Native American programs, divided by the total number

basis as well as administering monthly rental subsidy

of assisted units in the country that were administered

payments.64

in 2017. The estimate includes all staffing, non-personal

services, and al ocated agency overhead.

PRAC property owners are required to have a

Local PHAs are tasked with most of the administration

supportive services plan certified by a state or local

of HUD-assisted units. This administration includes

agency, and may have relationships with supportive

determining recipients’ eligibility, maintaining waiting lists,

service providers, but formal partnerships are not

inspecting units, interacting with landlords, managing

required among agencies. NED and other HUD

al ocated budgets, and a host of other functions. A

programs that assist adults with disabilities are

recent comprehensive study of administrative costs

administered by local PHAs that also administer HUD’s

in the Housing Choice Voucher program (the largest

mainstream housing programs. These programs

rental assistance program) found an average monthly

are normal y not required to have plans or formal

administrative cost per unit of $70.65 We use the estimated

relationships around providing supportive services to

average HCV administrative costs average to estimate an

program residents.

annual cost per month to PHAs of administering NED

The PRA program assists people who, on average,

and other HUD program units of $840.

have different health, disability, and sociodemographic

characteristics than individuals in PRAC, NED, and other

8.6 Total Estimated Costs

HUD programs. As presented in the seventh chapter,

and Cost Effectiveness

with the exception of PRAC, prior to being assisted by

PRA, PRA residents have a much higher prevalence of

To assess the overall cost-effectiveness of PRA compared

healthcare utilization and chronic and disabling conditions

to other HUD programs, the study calculated per-

than individuals in the comparison study groups. With

individual costs of housing assistance, healthcare and

these caveats stated, we estimate that the costs of

disability service utilization, and administrative costs for

HUD’s administration of the PRAC program are

people receiving assistance through the PRA program and

approximately $359 per unit per year. This estimate

each of our comparison programs.

includes the HUD staff that manage the program,

interacting with grantees, tracking property finances, and

Summary of PRA Program and

managing contracts associated with the program. It also

Comparison Group Costs

includes the activities funded through these contracts,

Exhibit 8.11 summarizes and totals our comparisons

which include property inspection, technical assistance to

of PRA program cost estimates and estimates for our

owners, and media and advertising to increase awareness

comparison programs.

of the program.

63 For example, HUD and its technical assistance contractor developed grantee and owner guides for participating in the program based in part on early grantees’ experience and feedback.

64 Administrative costs are associated with the LIHTC and other capital subsidies provided to properties that include PRA units. Except for where the agency administering the capital subsidy is an explicit partner to the PRA grantee and has staff dedicated to procuring units for the PRA program, we do not include costs of administering these capital subsidy programs as PRA program costs.

65 The study also found that administrative costs are lower for people with fixed incomes such as SSI and no earnings, which is likely to be the case for the NED residents that are compared with PRA here.

72

H U D S e c t i o n 811 P R A P ro g r a m – P h a s e I I E v a l u a t i o n

Chapter 8. Economic Study of the PRA Program

We estimate that the total annual cost per unit

Interviews with state agency staff uniformly report that the

per year of housing assistance and program

level of effort required to implement the PRA program has

administration in the PRA program is $17,402. Two-

exceeded their expectations and that the administrative

thirds of this cost is the combined value of capital and

fee is insufficient to cover the costs needed to support

rental subsidies. Per-unit administrative costs may decline

the PRA population. As one state housing agency staff

in coming years as more units come online and program

member states, “It’s very intensive, especially in the

processes continue to be refined and streamlined. We also

first part of the program—finding the properties,

include the more than $51,000 sum of Medicaid healthcare

working with the developers, for the partners it is also

and supportive services utilization costs that we estimate

extremely intense during the lease up period. There’s

in the exhibit. Because of the uncertainty associated

not enough money—even at 8 percent to cover our

with this cost (for example, due to data limitations and

costs, there’s no money to cover the partners’ cost—

assumptions made), we do not add it to our program total.

and then on the back end, the monthly payments that

Our analysis of the sample of PRAC properties, for

the contract administrators have to do to make sure

which we have complete capital subsidy detail, finds

the properties get their payments every month, it’s a

that PRAC housing assistance costs are about $1,200 a

lot.”

year greater in these properties than in PRA properties.

Another PRA staff member agrees that the staff time

This is roughly the amortized value amount of the capital

needed for the PRA program is substantial— “I think

subsidy provided by sources besides the PRAC grant.

it’s been implemented well. [Agency] and HUD at the

A comparison to the full PRAC sample finds lower rental

outset didn’t realize how much time it would take to

assistance costs in PRAC than in PRA, but higher capital

get a project from here to there and get people moved

subsidy amounts. Taken together, we conclude that

in. I don’t think that there was enough clear instruction

PRA and PRAC have similar total housing assistance

from HUD on how to do that and globally we didn’t

costs (within $100 a month). The main differences in

have a sense of how long it would take. The concept

the housing assistance costs of the program are that

of the program is wonderful but it takes the sun, the

capital subsidies for PRA come through the LIHTC

moon and the stars to actually get these people moved

program, in contrast to the HUD-funded PRAC capital

in from a nursing facility.”

grants.

NED and other HUD programs have administrative costs

Both NED and other HUD programs have rental

of about $900 per unit per year. It is not likely that the PRA

assistance costs that are approximately $1,000 per

program could attain this level of administrative costs

year higher than the same costs in the PRA program.

while reaching its goals. This is because the PRA program

However, these programs are sometimes in properties

includes additional interagency collaboration, and serves

that receive capital subsidies and sometimes not. Data

a population that has, on average, more challenges in

col ection for capital subsidies for the NED and other

obtaining and maintaining housing, and has a greater need

HUD comparison groups was not in the scope of the cost

for supportive services.

analysis, so we do not know total housing assistance

Final y, costs are fairly similar in magnitude (within

costs for these programs.

2 percent) between PRA and PRAC for the subset

Our estimates of administrative cost indicate that, as

of healthcare and supportive services costs we

of our data collection period (late 2017 to early 2018),

measure. Further research is needed to understand the

the PRA program had administrative costs that were

higher costs we observe for our NED comparison group

higher than those in our comparison programs. We

for HCBS use and inpatient care and the lower costs

identify three likely explanations. First, the PRA program

observed for the other HUD group, driven by much lower

is in a start-up phase, while the other programs are wel

estimated HCBS costs offset somewhat by much higher

established. Further, both HUD’s and grantees’ roles and

costs of long-term inpatient stays. We also reiterate the

responsibilities differ across the programs, with more

overall evaluation limitation that our program-level primary

inter-agency col aboration explicitly and formal y built into

data collection efforts were focused on understanding the

program administration in the PRA program. Finally, PRA

PRA program and did not include PRAC program data

program grantees target populations with different health

collection.66

and disability characteristics and housing needs than the

other programs.

66 It is possible, for example, that some PRAC programs have longstanding partnerships with organizations that provide supportive services to PRAC residents. Our estimate of program administrative costs may understate the administrative burden of these relationships. Our total cost estimates will not capture this burden if it is borne by sources other than HUD and average Medicaid reimbursement costs.

73

H U D S e c t i o n 811 P R A P ro g r a m – P h a s e I I E v a l u a t i o n

Chapter 8. Economic Study of the PRA Program

E xhibit 8.11: Per- Unit A nnual C ost E stimates for P R A and C omparison Programs

PRAC Complete

PRAC

PRA

Capital Subsidy

Grant Only

(41 properties,

(29 properties, 247

(435 properties,

NED

Other HUD

408 units)

units)

3,145 units)

(2,523 units)

(48,873 units)

Housing subsidy costs

Rental assistance

$6,841

$6,660

$6,199

$7,872

$7,632

Capital subsidy

$4,969

$6,346

$5,853

Unknown

Unknown

Total housing

$11,810

$13,007

$12,052

Unknown

Unknown

Program administrative costs

Partial y included in

Partial y included in

Grantee and partners

$5,345

$840 (PHA)

$840 (PHA)

rental subsidy

rental subsidy

HUD

$435

$359

$385

$62

$62

Total administrative

$5,780

$359

$385

$902

$902

Total program cost

$17,590

$13,392

$12,437

$8,774

$8,534

Healthcare and disability

$51,179

$50,321

$56,025

$34,204

services utilization costs

NA = not applicable.

Note: Research and data collection timelines required us to fix the sample for which we collected PRA capital subsidy data at an earlier time than the entire rental sample.

At that time, we sought information for 43 properties that include 420 of the 540 PRA residents in the final analysis sample.

Cost-Effectiveness of the PRA Program

than the PRAC program. In terms of HUD expenditures

This chapter finds that the PRA program houses adults

only, however, PRA is dramatical y more cost-effective, as

with disabilities at comparable to slightly lower cost than

HUD capital subsidies in the PRAC program are replaced

the PRAC program, with the capital subsidy portion of

by LIHTC (and other) subsidies in the PRA program.

assistance shifting from HUD to the LIHTC program.

Preliminary evidence indicates that the PRA program

As reported in earlier chapters on housing and

has some effect in shifting the type of healthcare and

neighborhood impacts, PRA units are in larger, newer

amount of supportive services that program residents

properties, with a substantial share of neighbors that are

obtain from inpatient care to home and community-based

not receiving housing assistance linked to a disability. PRA

services, as compared to PRAC and our other comparison

residents report that they like where they live at similar

programs. Our cost estimation approach translates these

rates as do PRAC residents, but they less frequently

shifts in healthcare utilization directly into costs. After

report knowing people in their buildings. Fewer PRA

moving into PRA units, PRA residents have lower inpatient

residents report their units are in excel ent or good

hospitalization costs and long-term inpatient stay costs,

condition. PRA residents’ neighborhoods have better

but these costs mostly offset higher costs of HCBS use.

walkability but worse environmental hazard scores and

If this short-term pattern continues and use of HCBS

have higher poverty levels and lower levels of education.

services displaces avoidable hospitalizations and long-

Our survey results also find that, while a majority of PRA

term institutional stays, we would conclude that the PRA

residents express satisfaction with their neighborhood, the

program has positive impacts on combined healthcare

share is lower than for PRAC residents, and PRA residents

and supportive services utilization that are cost neutral.

are less likely to feel safe in the neighborhoods.

This comparable cost-effectiveness of housing currently

Taking the cost and impact findings around housing

appears to come at a higher administrative cost. However,

together, we conclude that the PRA program provides

we note that the PRA program is relatively new and stil

housing at a comparable or slightly lower cost than the

incurring start-up level administrative costs, while our

PRAC program, for housing quality and experience that

comparison programs are wel established. Additional y,

is comparable to or slightly less desirable than the PRAC

while we measured administrative costs for the state

program. This conclusion suggests that the PRA program

agency partnership for PRA, we did not col ect data

is as cost-effective (but not more so) at providing housing

to assess the extent of any similar formal or informal

74

H U D S e c t i o n 811 P R A P ro g r a m – P h a s e I I E v a l u a t i o n

Chapter 8. Economic Study of the PRA Program

partnerships around the PRAC or NED programs.

Final y, the PRA program may more frequently target

individuals that require greater administrative resources

to be successful y housed than comparison programs,

notably including individuals with disability experiencing

homelessness. Given these caveats, further research is

needed to assess the cost-effectiveness of administrative

costs in the PRA program.

75

H U D S e c t i o n 811 P R A P ro g r a m – P h a s e I I E v a l u a t i o n

76

H U D S e c t i o n 811 P R A P ro g r a m – P h a s e I I E v a l u a t i o n

Chapter 9. PRA Program Practices that May Lead to Successful Results

PRA Program

The study does not try to determine the effect of

specific strategies, tools, or relationships on program

Practices that

outcomes or trends in data and does not attempt

to compare strategies to the program outcomes

on tenancy, healthcare utilization, or properties

May Lead to

and neighborhoods that are presented in the fourth

through seventh chapters. Instead, we offer these

Successful Results

observations to suggest areas of program modification

or further exploration and to provide information for state

PRA programs on practices and strategies that other state

agencies say work for them.

This chapter explores the relationship between strategies

HUD designed the PRA program so that state agencies

and results in six areas of the PRA program: selecting and

have the flexibility to tailor their PRA programs to meet

prioritizing state target populations; removing applicant

the needs of their states’ target populations, and in

barriers to eligibility; identifying quality, cost-effective units

response to state policy priorities. Grantees can make

for PRA; aligning tenancy supports with resident needs

a number of decisions that can affect their progress in

and preferences; and developing effective and sustainable

meeting the goals of their state PRA programs, how many

PRA partnerships.

people they ultimately assist with their PRA grants, and

the characteristics of these residents and where they live.

9.1 Strategies for Selecting and

These decisions include how the agencies select and

Prioritizing Target Populations

prioritize the state’s PRA target population, how the state

selects properties where PRA can be used to provide

States’ choices about whom to target and how to set

rental assistance for people with disabilities, and how

priorities among target populations affects who the PRA

PRA applicants are matched to affordable housing that

program ultimately assists. Comparing applicant and

meets their needs and preferences. All of these strategies

resident data among states and over time, we found that

can also affect the amount of time that PRA program

states that prioritized people exiting institutions over other

staff spend on administering the PRA program and how

applicants have assisted higher rates of these residents

much it costs the state and their partners to administer the

than states that do not prioritize these applicants.

program.

Similarly, states that identify or prioritize people

experiencing homelessness and actively conduct outreach

This chapter explores the relationships we observed

to those populations have higher rates of residents that

between elective strategies established by state

were experiencing homelessness directly prior to being

PRA programs and trends in program data. The

assisted by PRA. This section discusses strategies that

analysis uses state-level data from grantees’ quarterly

PRA program staff and partners in these states report

progress reports to assess whether there are differences

have been successful in reaching and referring eligible

among states or trends over time in certain key program

members of their states’ target populations.

measures: the number and characteristics of PRA

applicants and residents, ineligibility rates of applicants

Results of Selection and Prioritization of

and characteristics of those found ineligible, and the

PRA Target Populations

percentage of PRA units identified and under lease for the

As a result of state outreach strategies, 27 percent of PRA

PRA program compared to program projections.

residents assisted by September 2018 were previously

Where we identified differences among states or

living in an institution and 20 percent were homeless just

trends in program data over time, we examined

before being assisted by PRA (Exhibit 9.1). An additional

strategies that states adopted that may have

33 percent were reported to be at risk of becoming

influenced these differences and trends. We also

homeless or institutionalized unless they gained access to

asked staff from state agencies and local partners that

affordable housing. Another 6 percent had previously lived

administer the program what strategies and program

in a group home, adult care home, or other residential care

features they thought contributed to these trends, what

setting. The previous living situation is not known for the

challenges they have experienced while implementing

remaining 14 percent.

the program, and what they believe have been the tools,

strategies, and relationships that have contributed to them

meeting these challenges.

77

H U D S e c t i o n 811 P R A P ro g r a m – P h a s e I I E v a l u a t i o n

Chapter 9. PRA Program Practices that May Lead to Successful Results

E xhibit 9.1: Grantee- Repor ted Living Situation of P R A

A pplicant s and Resident s, September 2018

Applicants

Applicants on Waiting List

FY12 and FY13 Residents

2015-2018

as of September 2018

Number of applicants

6,781

3,237

725

Institutionalized

13%

8%

27%

At risk of institutionalization

11%

11%

13%

Homeless

23%

30%

20%

At risk of homelessness

18%

8%

20%

Living in a group home, adult care home, or other

7%

12%

6%

residential setting

Other/unknown

29%

31%

14%

Total

100%

100%

100%

Note: Percentages may not add to 100 percent due to rounding.

Source: Abt analysis of 2015-2018 Section 811 PRA Quarterly Reports through the quarter ending September 30, 2018, in six study states.

Selecting and Prioritizing People Exiting Institutions

provide a property owner with a ranked list of eligible and

Four states prioritize people residing in institutions over

appropriate applicants, rather than have owners lease

other applicants, and three of these states had the highest

units on a “first come, first served” basis.

percentage of PRA residents coming from institutional

Targeting People Experiencing or

settings: California (41 percent of all residents),

At Risk for Homelessness

Washington (38 percent), and Maryland (34 percent).

Homelessness raises distinct chal enges to applicant

All three states prioritized people living in institutions

matching. For example, caseworkers are not always able

over other applicants and built their applicant outreach

to locate applicants when their name comes to the top of

procedures off existing MFP procedures and systems.

the waiting list. People experiencing homelessness may

The PRA partnerships in Maryland and Washington also

lack a fixed address and could have moved several times

committed to using some of their MFP grant funds to help

between when they are put on the waiting list and when a

pay for PRA program administrative costs.

unit becomes available. Some potential applicants may be

Some chal enges are unique to individuals living in

hospitalized or in other institutional care when they reach

institutions. Service providers reported that, while

the top of the list. In general, state agency staff found that

individuals living in nursing facilities often expressed

the longer someone was on a waiting list, the more difficult

interest in living in independent housing, their actions

it was to find the person once the person’s name came to

show otherwise. Individuals may not be ready or wil ing to

the top of the list.

move when they say they will be. They may have current

While people experiencing homelessness are eligible

medical needs, may not have completed the required

for PRA units in all six states, four states (California,

paperwork, or have apprehension about living alone. As

Louisiana, Maryland, and Minnesota) have been

the Medicaid agency staff in one state noted “They want

specifical y targeting people who are homeless or at risk

to get out [of an institution] and will take anything to

for homelessness. States that had developed targeted

get out. [They] might be frightened to leave and go

outreach to people experiencing homelessness had higher

out on their own. They don’t want to live in a nursing

percentages of residents who were previously homeless

home, but they’re afraid. They decline and act like it’s

before being assisted by PRA.

a problem with the unit, but I think that it’s really a fear

of being alone in the community.”

In Minnesota, almost two-thirds of PRA residents (65

percent) had been homeless prior to moving into a PRA

In states with PRA waiting lists, possible residents are

unit, reflecting the extensive outreach to this target

identified based on unit size and accessibility, position on

population. State agencies work with organizations in

the waiting list determined by application date, readiness

Minnesota funded by Projects for Assistance in Transition

for independent living, and preferences for location

from Homelessness (PATH) to identify potential applicants

and property amenities. This al ows the state agency to

78

H U D S e c t i o n 811 P R A P ro g r a m – P h a s e I I E v a l u a t i o n

Chapter 9. PRA Program Practices that May Lead to Successful Results

without fixed addresses who may be living in shelters or

maintain regular contact. To locate such individuals, some

living on the street. The Minnesota Department of Human

states that required applications to come through referral

Services (DHS) initial y developed a state-wide waiting

organizations also connected with other organizations

list but found that approximately 40 percent of applicants

to expand their reach. In Delaware, the state housing

could not be located or were no longer in need of housing

agency sent PRA promotional materials to shelters and

when their name came to the top of the list. As a result,

other organizations that serve people who may not be

DHS changed to a more real-time matching process that

actively receiving Medicaid services. Those organizations

prioritizes potential residents based on current health

were then directed to a referral organization to apply on

status and readiness to move when a unit is available,

behalf of individuals. Service providers in Louisiana also

and reports this has reduced staff time in trying to locate

held meetings with people who were facing eviction or

applicants who may no longer be interested in PRA.

were using food banks, to let them know about the PRA

The state with the next highest proportion of residents

resource.

in PRA housing who were previously homeless is

Broader outreach can result in more applications from

Washington State, with 23 percent. State health agency

people who turn out to be ineligible. In Louisiana, a

staff in Washington State also work closely with local

significant percentage of applicants that applied did

homeless outreach providers to identify potential PRA

not have a qualifying disability or were not eligible for

applicants. In Louisiana, 9 percent of all residents were

Medicaid HCBS.

previously homeless but 54 percent were reported to have

been at risk for homelessness prior to being assisted by

9.2 Removing Barriers to Applicant

PRA. Louisiana’s PRA program addresses priorities in its

Eligibility for PRA

Ten-Year Plan to End Homelessness. To reach potential

PRA applicants, referral organizations in Louisiana

Between 2015 and 2018, about 19 percent of applicants

provided information about PRA at a monthly outreach

did not meet the PRA program’s requirement for age,

event that people experiencing homelessness often

income, or eligibility for services, or did not meet the

attend.

state’s target population definition. In addition, 14 percent

Fewer residents had previously been homeless in

of applicants did not meet the leasing requirements of the

California (1 percent of residents) and Delaware (6

property where they applied to live. The rates of ineligibility

percent) than in the other study states. Delaware did

varied among states and over time, and in some states

not identify people experiencing homelessness as a

the rate of ineligibility decreased over time. This section

target population for their PRA grant and prioritized

compares ineligibility rates among states and over time

people exiting institutions over other applicants. While

and identifies strategies that PRA program staff and their

California’s 2012 PRA grant did not specifical y identify

partners identified as contributing to improving the rate

people experiencing homelessness as a target population,

of acceptance of applicants to the PRA program and to

the state’s FY13 grant targets people experiencing

specific properties.

homelessness in the Los Angeles area. As of September

Applicant Eligibility Rates by State

2018, California had not yet started leasing units funded in

and Over Time

2013.

As long as residents meet PRA statutory program

Reaching Individuals Not Enrolled in Services

requirements, Section 811 PRA grantees are al owed

To be eligible for PRA, applicants must be eligible

flexibility in identifying potential PRA residents and

for Medicaid or state plan HCBS. Since services are

confirming their eligibility. In five of six study states, the

voluntary, PRA applicants and residents need only be

state departments of health or disability manage the

eligible for HCBS, and not enrol ed in these services.

tenant selection process, rather than the state housing

In order to reach individuals who are not enrol ed in

agency, and potential PRA residents apply to the PRA

HCBS but are eligible for PRA, some of the study states

program through approved referral organizations only.

developed additional outreach procedures to locate these

State health agency staff or contracted service provider

individuals, educate them on the PRA program, and

organizations typical y conduct direct outreach to the

assess their eligibility for housing assistance.

target population and determine applicant eligibility for

PRA. These state agencies had already been providing

People experiencing homelessness or at risk for

or coordinating services for the PRA target populations

homelessness may be less connected to services and less

through Medicaid waiver programs, MFP, and other

likely to have a Medicaid case manager with whom they

programs, creating a built-in pipeline for referrals to

79

H U D S e c t i o n 811 P R A P ro g r a m – P h a s e I I E v a l u a t i o n

Chapter 9. PRA Program Practices that May Lead to Successful Results

the PRA program. Referral organizations also include

requirements for age, income, or eligibility for home

organizations that help people experiencing homelessness

and community-based services, or did not meet the

find housing.

state’s target population definitions.

Exhibit 9.2 shows the number of applicants for the PRA

As of September 2018, approximately 2,000 households

program, the number of applicants referred to properties

had applied at properties where PRA units are under

to complete lease applications, and the number and

contract in the study states. Fourteen percent of

percent of applicants found ineligible at each stage. In the

referred applicants were found not to meet the

first three years of the program, 6,770 individuals applied

property’s leasing requirements. Exhibit 9.2 shows

to the PRA program in the six study states. PRA program

the percentage of applicants who were found ineligible

staff found that almost one in five of these applicants

for the PRA program and found to not meet property

(19 percent) either did not meet the PRA program

requirements by state.

E xhibit 9.2 : A pplicant s F ound Eligible for the P R A Program

in the Six S tudy S t ates, Program Year s 2015 –2018

Total

California

Delaware

Louisiana

Maryland

Minnesota

Washington

Applicants to PRA program

6,770

248

775

2,111

2,853

528

255

Applicants ineligible for PRA

1,255

55

program

<10a

1,078

66

35

>10a

% ineligible of applicants for

19%

22%

program

<2%a

51%

2%

7%

>4%a

Applicants referred to properties

2,009

183

678

455

173

279

241

# of referred applicants ineligible for

278

property

<10a

66

28

>10a

88

78

% of referred applicants ineligible

14%

for property

<3%a

10%

6%

>6%a

32%

32%

a Values not shown due to small sample reporting restrictions.

Source: Abt analysis of 2015-2018 Section 811 PRA Quarterly Reports through the quarter ending September 30, 2018.

Applicants Found Ineligible for the PRA

providers do not screen them, nor are they responsible for

Program

putting the potential tenants on the waiting list. As a result,

The percentage of applicants found ineligible for the

the most common reason that applicants in Louisiana

program (for age, income, eligibility for services, or

were found ineligible for the PRA program was not being

disability status) varies widely by state, reflecting the

eligible for HCBS (67 percent).

different outreach procedures of PRA programs in the

In states with closed referral procedures, the most

study states.

common reasons applicants were determined ineligible

Most (86 percent) of the ineligible applicants among

for the PRA program were being over age 62 or having

the six study states were in Louisiana, where the

household incomes that were too high (Exhibit 9.3).

PRA application is available to the public. More than

Applicants may age out of eligibility while they are waiting

half (51 percent) of PRA applicants in Louisiana were

for available PRA units, and changes in the local area

found ineligible for the PRA program between 2015 and

median income could affect an applicant’s eligibility

2018. Only in Louisiana can the public apply directly to

status. In Maryland, staff from the state housing agency

the PRA program, where the PRA application is publicly

commented that a change in AMI from one year to the

available on the Louisiana’s Department of Health’s

next was so significant that 10 percent of applicants on

website. Louisiana’s potential tenants may hear about the

the waiting list were deemed ineligible for the program.

waiting list through shelters or service providers, but those

80

H U D S e c t i o n 811 P R A P ro g r a m – P h a s e I I E v a l u a t i o n

Chapter 9. PRA Program Practices that May Lead to Successful Results

E xhibit 9.3 : Reasons A pplicant s Were F ound Ineligible for the P R A Program, Program Year s 2015 –2018

2015

2016

2017

2018

Total

Reason for Ineligibility

160

368

369

358

1,255

Not disabled

8%

0%

18%

<3%a

7%

Over income

11%

9%

11%

11%

10%

Not eligible for services

71%

74%

62%

62%

67%

Over age 62

<7%a

<3%a

<3%a

<3%a

1%

Criminal history

<7%a

8%

6%

4%

6%

Other

<7%a

<3%a

<3%a

7%

4%

Unknown

16%

10%

8%

19%

13%

a Exact percentages not shown due to small sample reporting restrictions.

Note: Applicants could be determined ineligible for more than one reason, so totals do not add up to 100.

Source: Abt analysis of 2015-2018 Section 811 PRA Quarterly Reports through the quarter ending September 30, 2018, in six study states.

The percentage of applicants found ineligible for the PRA

for homelessness. In 2018, California began outreach to

program has fluctuated since the program began but has

people experiencing homelessness, a target population

been higher in recent years. Ineligible program applicants

for the state’s 2013 PRA grant. Leasing under this grant

have increased in California and Maryland, reaching

just began in 2018. The PRA eligibility trends are shown in

60 and 41 percent respectively in 2018. A

Exhibit 9.4: lmo

Perst all of

Exhibit 9.4.

centage of Applicants That Did Not Meet PRA

California’s ineligible applicants wer

Pr e reported to be a

ogram Requir t risk

ements, Program Years 2015-2018

E xhibit 9.4 : Percent age of A pplicant s T hat Did Not Meet

P R A Program Requirement s, Program Year s 2015 –2018

25%

22%

20%

20%

19%

18%

15%

Percent of Total PRA

12%

Program Applicants

10%

Found Ineligble

5%

0%

Total

2015

2016

2017

2018

Source: Abt analysis of 2015–2018 Section 811 PRA Quarterly Reports through the quarter ending September 30, 2018, in six study states.

81

H U D S e c t i o n 811 P R A P ro g r a m – P h a s e I I E v a l u a t i o n

Chapter 9. PRA Program Practices that May Lead to Successful Results

Applicants Found Ineligible for the

E xhibit 9.5 : Reasons A pplicant Lease A pplications

Properties

Were Not Accepted, Program Year s 2015 –2018

Between 2015 and 2018, 14 percent of applicants

referred to specific properties did not meet the

leasing requirements of the property. Property owners

Reason Application Was Not Accepted

Total

require that Section 811 PRA residents meet the same

N

278

criteria as other residents at the property, including

Criminal history

56%

undergoing screening for criminal history, credit history,

Poor credit

20%

rental history, and income.

Poor rental history

14%

More than half of the applicants who were denied

Unable to submit required documentation

6%

units (56 percent) were denied because of criminal

history. Others were denied because of poor credit

Other

15%

histories (20 percent) and poor rental histories (14

Note: Applicants could be denied units for more than one reason, so total does

percent). Fewer (6 percent) were rejected for not

not add to 100 percent.

submitting the required documentation. (See Exhibit 9.5.)

Source: Abt analysis of 2015-2018 Section 811 PRA Quarterly Reports through

the quarter ending September 30, 2018, in six study states.

State agency staff and service providers we interviewed

attributed this change to improved practices, as case

managers have learned more about each property’s

Automating Program Application and Eligibility Procedures

eligibility requirements and what applicants are required to

Automated, web-based application tools can help case

provide when they apply. Staff also attributed the change

managers determine eligibility before referring applicants

to improved prescreening of applicants to determine

to properties to complete applications. The tools also

whether they meet the property’s eligibility requirements

helped case managers quickly learn about the eligibility

before referring applicants to the property to complete a

status of the people they assist. In Delaware, applicant

lease application.

eligibility is determined immediately upon submittal of the

Ineligibility decisions can also stem from the

person’s birth certificate, license or other identification,

misunderstanding of PRA eligibility requirements,

Social Security award letter, and proof of Social Security

particularly around calculating income and the tenant’s

number. In Louisiana, the state health agency accepted

portion of rent. Sometimes housing providers must apply

applications electronical y and added drop-down

stricter guidelines for housing eligibility for PRA applicants

menus when they realized that service providers were

than for other housing subsidy programs in their building.

not submitting uniform information. In Washington and

For example, tax credit properties may have higher

Maryland, state health agency staff use an online tool to

maximum income thresholds and calculate the tenant’s

enter applicant information and to determine eligibility for

portion of the rent payment differently from PRA. Service

Medicaid and other PRA eligibility requirements.

providers also reported that owners did not always

Support from Service Providers and Property Managers

take a person’s medical costs into consideration when

calculating household income and total tenant payment.

Both service providers and property managers may help

They noted that some applicants may have been denied at

PRA applicants with the complicated program-level and

properties based on this calculation.

property-level processes for eligibility determination.

Service providers report that working with PRA applicants

as early in the process as possible helps improve

Strategies Used to Remove

the applicant’s likelihood of meeting PRA eligibility

Barriers to Eligibility

requirements. Service providers often help residents

Strategies that state agencies and local partners

obtain proper identification and documentation of

developed to reduce eligibility barriers have caused

income prior to applying to PRA. This can include helping

the percentage of applicants who did not meet the

applicants obtain documentation for their SSDI, SSI, or

property lease requirements to decrease by half

other benefits, correct inaccurate information on credit or

between 2016 and 2018 (Exhibit 9.6). Three states,

rental histories, and resolve outstanding utility or rental

California, Louisiana and Maryland, had consistently high

debt.

acceptance rates by property owners between 2015 and

Property owners are also often wil ing to assist applicants,

2018.

and some properties have created pre-application

packages to help move the process along quickly. Some

82

H U D S e c t i o n 811 P R A P ro g r a m – P h a s e I I E v a l u a t i o n

Chapter 9. PRA Program Practices that May Lead to Successful Results

Exhibit 9.6: Percentage of PRA Applicants Whose Lease Applications

Were Not Accepted by Owners, Program Years 2015-2018

E xhibit 9.6 : Percent age of P R A A pplicant s W hose Lease A pplications

Were Not Accepted by Owner s, Program Year s 2015 –2018

20%

18%

18%

16%

14%

14%

14%

12%

10%

11%

Percent of Total

10%

Applicants Found

8%

Ineligble for

PRA Properties

6%

4%

2%

0%

Total

2015

2016

2017

2018

Source: Abt analysis of 2015–2018 Section 811 PRA Quarterly Reports through the quarter ending September 30, 2018, in six study states.

property managers allow applicants to provide personal

Case managers report that they also attempt to anticipate

statements explaining any extenuating circumstances that

these denials by obtaining these criminal records and

resulted in a poor credit history, rental history, or criminal

credit reports themselves early in the application process.

background that would make them ineligible to live at

Some case managers will even include an appeal with

the property. Some PRA programs offer extra incentives

the original lease application when they believe the client

to property owners participating in the program to help

will automatical y be denied a unit. Educating property

mitigate the risk associated with accepting tenants with

owners on reasonable accommodation requests and laws

chal enges to successful tenancy. As an example, in

protecting the rights of people with disabilities has also

Washington, the state health agency will offer a larger

been effective in reducing ineligibility rates. In Minnesota,

security deposit for individuals with poor credit or a

one property owner denied more than half of applications.

criminal background. Overal , case property managers

State health agency staff met with the property staff, the

and case managers who work with PRA applicants felt

state housing agency, and a Legal Aid to address the

these strategies helped address issues early and speed

issue.

up the process of leasing the unit.

9.3 Identifying and Securing

Use of Reasonable Accommodation Requests

to Appeal Lease Rejections

Quality, Cost-Effective PRA Units

Service providers have sometimes succeeded in getting

The six states have had varying rates of success at getting

reasonable accommodation from property owners when

PRA units under contract and under lease. States that tied

applicants had first been denied based on the results

the al ocation of PRA units to LIHTC awards, switched

of a criminal or credit background check that could

their focus from placing units in existing multifamily

be attributed to a person’s disability. Under the Fair

housing to new construction properties, and received

Housing Act, property owners in HUD programs must

approval from HUD to increase contract rents in high-cost

provide “reasonable accommodations in rules, policies,

areas were able to identify all or most of their planned

practices, or services when such accommodations may

units. However, these strategies also come with tradeoffs

be necessary to afford persons with disabilities an equal

such as reducing the number of PRA units a state will be

opportunity” to access and receive housing assistance.67

able to support as a result of increasing contract rents.

67 https://www.hud.gov/program_offices/fair_housing_equal_opp/reasonable_accommodations_and_modifications

83

H U D S e c t i o n 811 P R A P ro g r a m – P h a s e I I E v a l u a t i o n

Chapter 9. PRA Program Practices that May Lead to Successful Results

Percent of Estimated PRA Units Under

• Percentage of estimated units identified but not

Contract and Under Lease

yet under contract. While not al planned units were

State housing agencies enter into Cooperative

under contract as of September 2018, grantees in five

Agreements with HUD that include the estimated number

of the study states had identified all of the properties

of PRA units that their PRA grant funds are expected to

where they planned to use PRA subsidies for the 2012

assist. Through September 2018, study grantees had

grants. Two states, Minnesota and Louisiana, had

contracts in place with owners for more than half of

identified more units than the estimated number of

planned units and had identified properties for most of

units in their Cooperative Agreements with HUD.

the rest of their planned units. We measured progress in

Five of the six study states have identified nearly all of

identifying units for PRA using three metrics:

their units estimated to be funded under their 2012 grants.

• Percentage of estimated units under lease by PRA

Units that are identified but not under contract yet are in

residents. The states that had the highest PRA leasing

properties that are under construction and will be placed

rates were Minnesota (91 percent) and Louisiana (62

under contract for PRA closer to when the property will be

percent), fol owed by Washington (52 percent) and

available for lease. Grantees may sign written agreements

Maryland (49 percent).

to signal their commitment to enter into a contract with the

owner at a later date.68

• Percentage of estimated units under contract with

owners for specific properties. As of September

Only in Delaware had the state housing agency not

2018, grantees in the study states had entered into

identified all of its estimated PRA units. The agency

contracts with property owners for 78 percent of

planned to award the remaining units through future tax

planned FY12 units. Louisiana and Minnesota had al

credit al ocations. Grantees expect to eventual y enter into

of their planned PRA units under contract with owners.

contracts with owners for identified units once the initial

Delaware had 78 percent of their planned units under

leasing date is near; however, owners are not contractual y

contract, Washington had 65 percent, California had

bound to make units available to the PRA program until

55 percent, and Maryland had 49 percent.

they sign the contract with the state housing agency

grantee. Exhibit 9.7 shows unit status by grant year and

study state.

E xhibit 9.7: S t atus of Planned Section 811 P R A Unit s by S t ate, September 2018

FY12

FY13

% Units

% Units

# of Planned

% of Units

under

% Units

# of Planned

% of Units

Under

% Units

State

Units

Identified

Contract

Leased

Units

Identified

Contract

Leased

California

233

96%

55%

36%

200

17%

0%

0%

Delaware

148

82%

78%

42%

Louisiana

199

128%

128%

62%

Maryland

150

115%

49%

49%

150

86%

5%

4%

Minnesota

85

99%

99%

91%

75

104%

37%

13%

Washington

133

100%

65%

52%

Total

948

104%

78%

51%

425

56%

7%

3%

Source: Abt analysis of 2015-2018 Section 811 PRA Quarterly Reports through the quarter ending September 30, 2018.

68 Written agreements can include memoranda of understanding (signed by the grantee and the owner); tax credit or other funding award letters signed by the funders (which may or may not be the grantees); or a letter of agreement or commitment signed solely by the grantee.

84

H U D S e c t i o n 811 P R A P ro g r a m – P h a s e I I E v a l u a t i o n

Chapter 9. PRA Program Practices that May Lead to Successful Results

States are not as far along with their 2013 grants, typical y

another, and how vacant units are fil ed between PRA

leasing their 2012 units first since their grant period wil

applicants and other applicants.

end first. Of the study states, only California had not yet

As of September 2018, vacancy rates of PRA units under

identified the majority of their estimated 2013 units. In

contract with owners in the six states ranged from 0 to

Minnesota and Maryland, grantees have identified their

7 percent. Although vacancy rates were low, three of the

2013 units in new construction properties.

study states had high percentages of PRA units under

Units under contract in existing properties become

contract that were unavailable for PRA residents because

available for PRA applicants once there are vacancies

they were still being occupied by other, non-PRA residents

in the properties and according to the property’s tenant

or because the units were not yet available for lease

selection plan. Grantees approve owners’ tenant selection

(Exhibit 9.8). For new construction and properties being

plans that include policies for how applicants are matched

rehabilitated, grantees typical y enter into contracts with

to units. The plans address how many applicants are

owners about six months prior to the unit being available

referred to complete applications for each available unit,

for PRA residents. More than half of PRA units under

how owners should prioritize one PRA applicant over

contract (52 percent) were unavailable for PRA residents

in Louisiana, 43 percent in Delaware, and 34 percent in

California.

E xhibit 9.8 : Unit s Identified, Under C ontract, and Under Lease for P R A , September 2018

FY12 PRA Grants

FY13 PRA Grants

% of Units

% of Units

# of Units

Under

% Units

% Units

# of Units

Under

% Units

% Units

State

Under RAC

Lease

Vacant

Unavailable

Under RAC

Lease

Vacant

Unavailable

California

128

66%

0%

34%

0

0%

0%

0%

Delaware

115

54%

3%

43%

Louisiana

255

48%

0%

52%

Maryland

73

100%

0%

0%

8

75%

25%

0%

Minnesota

84

92%

7%

1%

Washington

87

79%

3%

17%

Total

742

66%

2%

32%

36

44%

6%

50%

Source: Abt analysis of 2015-2018 Section 811 PRA Quarterly Reports through the quarter ending September 30, 2018.

Study State Adaptations to

housing markets. Increasing rents wil , however, reduce

Secure More PRA Units

the number of PRA units the grant will support.

At first, the states that were most successful in placing

In California, the grantee had committed only one-quarter

units under contract were those that awarded at

of the grant funds after 16 months. After increasing rents

least some of their PRA units to properties that were

to FMR, the grantee was able to identify properties for the

already in operation as affordable housing. Over

remaining three-quarters of its PRA funds within a year.

time, grantees have tried to attract properties under

In Washington, the state housing agency had difficulty

development—for example, tying the award of PRA units

attracting owners in the high-cost Seattle area. As in

to LITHC al ocations. They have also leveraged existing

California, the grantee reported that increasing contract

relationships with owners of affordable multifamily housing

rents helped them attract owners and also helped the

and increased PRA rents offered to owners in high-cost

program secure higher-quality units in locations where

areas.

many prospective PRA residents want to live.

Increasing PRA Contract Rents in High-Cost Areas

Exhibit 9.9 shows the effect of increasing rents on the

number of units the grants can support. Combined,

State housing agencies found increasing contract rents to

grantees estimate they will assist 280 fewer PRA units, an

the highest al owable by the program to be a very effective

overall reduction of 17 percent of estimated PRA units for

strategy for obtaining owner participation in competitive

the six study states.

85

H U D S e c t i o n 811 P R A P ro g r a m – P h a s e I I E v a l u a t i o n

Chapter 9. PRA Program Practices that May Lead to Successful Results

E xhibit 9.9 : Change in E stimated P R A Unit s as a Result of Average E xpected Increases in C ontract Rent Estimated Units

State

in Cooperative

Current

Change

% Change

Agreement

California

618

433

-185

-30%

Delaware

148

148

0

0%

Louisiana

199

199

0

0%

Maryland

303

300

-3

-1%

Minnesota

170

160

-10

-6%

Washington

215

133

-82

-38%

Total

1,653

1,373

-280

-17%

Source: Abt analysis of 2015-2018 Section 811 PRA Quarterly Reports through the quarter ending September 30, 2018, in six study states.

PRA partnerships in Maryland and Louisiana initial y set

9.4 Ensuring PRA Residents

their rents at FMR, the maximum al owed by HUD. These

grantees reported this approach made it relatively easy

Receive Tenancy Supports They

to recruit owners. As of September 2018, Louisiana had

Need

placed more than its planned units under contract with

owners. The grantee reported that having more units

Tenancy supports are available to PRA residents from

under contract than the total they expect to use will create

pre-move through ongoing tenancy to help them live

greater flexibility in matching applicants to units.

independently in the community. The study found that not

all residents receive the supports they need (see the sixth

Providing Vacancy Payments to Owners

chapter). The study team interviewed property owners

Another measure of how well states match applicants

where PRA residents live, service providers who work with

to PRA units is the percentage of units under contract

PRA residents, and state agency staff who administer the

for PRA that have been leased with PRA residents.

PRA program about the chal enges of ensuring that PRA

As of September 2018, grantees report vacancy rates of

residents receive the services and support they need to

PRA units for the six states range from 0 to 7 percent.

remain independent in their homes.

In California and Delaware, all available PRA units under

While most ongoing tenancy supports are funded through

contract are leased.

Medicaid, funding for transition services is limited and

While vacancy rates are low, when they occur, PRA

varies widely by state and even by the individual PRA

grants can be used to compensate owners for lost rent

resident. In addition to limited resources for tenancy

if an owner needs to hold a unit until a PRA applicant is

supports, service providers spoke about chal enges

ready to move in or if a PRA resident moves out without

posed by the voluntary nature of supports, the differences

appropriate notice. HUD al ows grantees to set vacancy

in support needs among residents, and the high turnover

payments at up to 80 percent of the unit’s rent for up to

and large caseloads of their profession.

60 days. (See discussion of vacancy payment costs in the

Challenges in Coordinating Tenancy

second section in the eighth chapter.)

Supports for PRA Residents

Grantees can also establish policies that offer less

compensation to owners, and some chose to do so. Even

Enrollment and Participation in Services is Voluntary

in states that provide the maximum al owable vacancy

Enrol ment in Medicaid HCBS and other tenancy support

payments, several owners reported that they do not

services and continued participation in these services are

always get this compensation. Some owners reported not

voluntary, and not all participants elect to receive them.

requesting payments because they would only cover a few

PRA residents must be eligible for Medicaid services but

days and were not worth the effort. Other owners reported

cannot be required to accept these or other services and

they automatically submit requests for vacancy payments

may discontinue participation in services at any time while

to the state housing agency for all days PRA units stayed

retaining their eligibility for housing. Property managers

vacant each month.

and service providers both noted that the voluntary

86

H U D S e c t i o n 811 P R A P ro g r a m – P h a s e I I E v a l u a t i o n

Chapter 9. PRA Program Practices that May Lead to Successful Results

nature of services can be problematic, particularly when

sometimes leave gaps. For example, several days may

residents with mental health needs dis-enroll from case

elapse between a resident’s final meeting with the

management, and property managers no longer have a

transition services coordinator who moves them into a

case manager counterpart with whom to coordinate a

PRA unit and their first meeting with the case manager

response to a crisis.

who will provide ongoing support. A new resident’s needs

Some property managers raised concerned about

for physical assistance, emotional support, or help with

residents with behavioral or mental health issues who do

tenancy issues may be particularly high at this time of

not agree to receive services, which results in negative

change. Case managers suggested that it would be

social behaviors and potential disruption or property

helpful if they could spend more time with clients after

damage.

they move into a PRA unit to help acclimate residents to

their new homes and lifestyles.

Providing Adequate Transition and Ongoing Supports

Strategies for Coordinating Tenancy

Property managers told us that some individuals need

Supports for PRA Residents

more help than they receive when moving into a PRA

unit. In most states, resources to assist residents with the

To assess how well tenancy supports are being

physical move to a PRA residence are scarce. Transition

coordinated for PRA residents, we interviewed service

providers were also frustrated by their inability to fil

providers who work with PRA residents and property

certain needs, such as assisting with food stamp eligibility

managers where PRA residents live. These respondents

or procuring household cleaning products. To address

described approaches to coordinating tenancy

these challenges, one service provider recommended

supports and how they addressed challenges they have

a flexible funding pool to be used as needed during the

encountered.

move-in process.

Strong Lines of Communication Between Property

Once PRA residents have moved in, additional needs

Management and Service Providers

may surface. While the PRA program subsidizes monthly

Property managers stated that strong lines of

rent, many PRA residents lack adequate funds for food,

communication between residents, case managers, and

utility payments, and cleaning supplies. PRA residents

property managers are crucial for PRA residents to have

may be new to managing a household budget and could

successful tenancies. Strong relationships can identify

benefit from personal finance training. Property managers

issues early and more quickly eliminate barriers facing a

and service providers reported that some PRA residents

resident. One property manager suggested that monthly

may also need help understanding community rules and

check-in cal s between the property management and

standards such as guest policies and noise issues.

case management help to foster these open lines of

Staffing Challenges and Large Caseloads

communication. Property owners and service providers

for Service Providers

also report that maintaining a central point of contact for

each resident—for example, a caseworker or a family

Staffing chal enges and large caseloads affect all three

member—is important for addressing tenant issues and

phases of tenancy supports: housing locator services,

ensuring successful tenancy.

transition services, and ongoing supports. Service

provider organizations told us they experience significant

Tenant releases of information have been an

turnover in staffing, making it difficult at times to maintain

especially useful tool for tenancy support. In al

consistent relationships with both clients and property

states, property owners report that PRA residents sign

managers. As a result, property managers at times are

information release forms that al ow them to provide

unsure whom to contact when issues arise with PRA

information about themselves to third parties as

residents. Shifts in staffing also complicate handoffs

consented to in the form. In Delaware, new tenants sign

between the phases of supports.

a resident services release form that al ows property

managers to refer tenants to a resident services

In some cases, one case manager or organization works

coordinator if needed and allows the resident to check

with an individual throughout the process of locating

off exactly whom they want to be contacted and under

housing, moving in, and remaining stably housed. More

which circumstances. In Maryland, tenants sign a release

commonly, different agencies may assist in each phase.

of information as a standard part of their application to live

An individual case manager may deliver all services

in a property and complete emergency contact forms that

directly or may coordinate with additional providers who

property managers keep in the resident’s file in case it is

provide services to meet a resident’s specific needs.

needed.

Handoffs between tenancy support organizations

87

H U D S e c t i o n 811 P R A P ro g r a m – P h a s e I I E v a l u a t i o n

Chapter 9. PRA Program Practices that May Lead to Successful Results

The first few days in a new residence can be particularly

A goal of the PRA program is to encourage col aboration

challenging for people newly living in the community.

between state housing and state health and human

Property managers found that having service

service agencies that results in long-term strategies for

providers accompany tenants to milestone events,

providing supportive housing for people with disabilities

such as lease signing and move-in, facilitated a

in their states. Whether the PRA partnership is ultimately

smooth transition. Both service providers and property

successful and sustainable depends on a number of

managers told us that the transition process for PRA

factors. Factors that affect the PRA partnership include

residents has become smoother over time. They report

how well state partners communicate and col aborate

that the process has improved as service providers and

and whether they have adequate and knowledgeable

property managers became familiar with the program and

resources to staff and implement the program.

built communications channels with one another.

In this section we summarize input from PRA program

Leveraging Other Sources of Tenancy Supports

staff at state housing and Medicaid agencies on how they

work together to administer the PRA program and the

In Minnesota, the PATH program conducts outreach to

success of their col aboration.

homeless individuals and works with them through the

early stages of transition to housing. However, PATH is

How State Agencies Define

not designed to provide ongoing tenancy supports, and

Successful PRA Partnerships

some formerly homeless individuals, particularly those

with mental health conditions, struggled to remain housed.

Most state housing and health agencies staff we

In response, Minnesota is using state funds to support

interviewed spoke about collaborative partnerships

transitions in resident support from PATH to a nonprofit

built on mutual goals and respect. These interviews

organization that offers community-based mental health

revealed common themes of successful PRA partnerships.

and substance use disorder services to adults with a

Staff spoke about building off of existing relationships,

history of serious mental il ness.

regular communication between agencies, sharing

aggregate resident and program data, and col aborating

Final y, a few of the PRA properties in the study states

on key areas of joint concern but relying on one another’s

employ a service coordinator to address residents’

expertise. State agency staff also recognized the value

health and social service needs. These services

and contribution of dedicated staff and local partners.

are available to all residents, not just PRA tenants. For

example, several properties in Minnesota have a resident

PRA Program Partnerships Continue To

connections coordinator who provides referrals to social

Benefit from Pre-Existing Relationships

services agencies that assist residents with education,

In four of the six study states, the partnership between

employment, and physical and mental health services and

the state health and the state housing agencies was a

works directly with residents to provide tenancy supports.

continuation of a longstanding relationship, with a shared

The property manager at a property in Maryland provides

commitment to supportive housing. In these states, the

al new residents with a handbook that outlines guidelines

PRA program continues to benefit greatly from pre-

for independent living, such as taking out the trash,

existing state relationships and programs, particularly the

cleaning one’s apartment, and general guidelines on being

MFP program and supportive housing programs. Program

a respectful neighbor.

staff we interviewed from these states underscored the

long-lasting effects of these relationships. In Maryland, a

9.5 Developing Effective and

staff member noted that “It’s a balanced partnership—I

Sustainable PRA Partnerships

don’t know if other states could say that but because

of our long history and the trust that we have all been

At the core of the PRA program is the partnership between

able to establish. We have established clear roles and

the state housing agency and state health agency or

everyone knows what the other people do—everyone

agencies. The Melvil e Act that authorized PRA requires

plays their roles really well.”

the state-level partnership as a condition of receiving PRA

Staff from state agencies without pre-existing

funding. The state housing agency receives the PRA grant

relationships also noted the difference. “This program is

funds and administers the PRA subsidies. The state health

nothing without its partnerships and it takes time and

agency commits to providing services and, in most states,

energy to develop those—especially for states that

to identify and select target populations to be served by

didn’t have a pre-existing program.”

the PRA program. An Interagency Partnership Agreement

describes how state housing and health agencies

delineate grant responsibilities.

88

H U D S e c t i o n 811 P R A P ro g r a m – P h a s e I I E v a l u a t i o n

Chapter 9. PRA Program Practices that May Lead to Successful Results

Relying Upon Each Other’s Expertise

result when staff is stretched too thin to maintain frequent

in Health and Housing

and regular communication, due to work on other housing

Helping residents obtain and maintain tenancy services

and/or services programs and competing priorities for

and supports involves col aboration by all PRA program

their time. These gaps can result in missed opportunities

partners: the state health and housing agencies, service

for applicants to obtain housing or delays in getting

providers, and property managers. Staff from many of the

residents the tenancy supports they may need. To avoid

state agencies and partners spoke about the importance

such gaps, the PRA agencies in most of the study states

of relying upon each other’s expertise and knowledge in

have established standing meetings varying in frequency

making successful programmatic decisions. As one PRA

from weekly to monthly. Some of these are part of

program staff member noted, “It’s all about trusting the

regular, standing meetings that discuss the PRA program

agency to do what they do well.”

among other programs. States also report more frequent

communication around lease-ups and resolving tenancy

State housing agencies note that state health agencies’

issues. For example, in Maryland, the state housing

input on where PRA units should be located is an

agency hosts weekly meetings with property owners in the

example of successful col aboration. In all of the study

months leading up to initial leasing of the property.

states, although the state housing agency makes the final

decision, the state health agencies have provided critical

State agency staff report that a central point of contact

information on the needs of the target population, their

in each agency helps streamline communications. There

current location, and availability of services in particular

can also be downsides to this approach, however, if the

areas. This helps determine whether there wil be enough

success of the program relies too much on one person’s

demand for units in specific geographic locations and

availability and knowledge. Some property owners also

whether adequate services will be available for residents.

reported that having clear lines of communication open

between them and the state agencies is particularly critical

In Minnesota and Maryland, state health agencies

for questions or reporting tenant issues. Owners report

specifical y review property applications and have rejected

that they did not necessarily understand when to contact

some properties that the state housing agency selected

the state housing agency and when to contact the state

because of lack of services or transportation in the area.

health agency for questions or concerns with the program

In Louisiana and Washington, health agency staff used

or regarding specific tenant issues.

their existing relationships with owners to help identify

units for PRA. State health agencies also played a role by

Sharing Program Data and Program

providing input on unit sizes, accessibility, and amenities

Monitoring Between Partners

that will meet the needs of the PRA target population in

In all of the study states, state housing and health

their states.

agencies share property and aggregate resident data to

Regular, Ongoing Communication Keeps

monitor program progress and outcomes. In Louisiana,

Partners Up to Date on Program Progress

staff from both the state housing and state Medicaid

agency have access to the waiting list system to review

Where the state housing and health agencies have

resident information and make updates. Agency staff in

longstanding relationships, PRA may be one part of a

a few states specifical y noted that they did not share

broader state initiative for affordable housing. In new

individual applicant or resident data in order to protect

partnerships, staff had to develop ways to collaborate

the privacy of residents and noted the practices that

and make decisions about the PRA program. State

they use to ensure applicant and resident data is always

agency staff reported that it takes time and effort

protected. Furthermore, most state agencies did not

to develop a PRA partnership and effective ways to

report collecting additional data on applicants, residents,

collaborate. Staff need to learn each other’s language and

or units beyond what is required for their tenant selection

determine which mechanisms for communication and

needs or for grant reporting to HUD. None of the state

making joint decisions work best for them. Successful

agency staff interviewed spoke of developing additional

communication strategies include standing meetings,

internal performance measures, although many noted

ad hoc communication around lease-ups and resolving

that they were considering them as part of ongoing grant

tenancy issues, and consistent points of contact between

monitoring procedures.

state health and housing agencies and between property

owners and service providers.

Sustaining PRA Partnerships

Some PRA program partners reported gaps in

State agency staff spoke about factors that may affect

communication among program partners. These can

sustaining and scaling up their PRA program partnerships.

89

H U D S e c t i o n 811 P R A P ro g r a m – P h a s e I I E v a l u a t i o n

Chapter 9. PRA Program Practices that May Lead to Successful Results

A central theme was having adequate staffing and

In addition, state health agencies and service

financial resources to implement the program. While

providers in the study states are increasingly

state housing agencies report that the time they spend

including staff with affordable-housing knowledge. In

on the program has decreased somewhat, this is not the

Washington, the state housing agency benefits from the

case for the state health agencies and other program

housing knowledge of the Regional Housing Coordinators,

partners. Grantees and their partners are concerned that

who help coordinate both services and housing

they may not be able to maintain the level of commitment

placement. The coordinators have connections with

to the program going forward without adequate funding.

property owners and have helped identify properties for

Those that are committing their own financial resources

PRA units and place tenants quickly. State-funded service

to supplement administrative grant fees may not be able

provider organizations called Community Choice Guides

to do so indefinitely. Several state agency staff noted that,

also have knowledge of both available health resources

while they understood that it would require additional

and affordable housing resources. In Minnesota, the state

support and resources to get the program up and running,

health agency hired a PRA Coordinator with previous

they had hoped the necessary level of effort would lessen

extensive experience in state and local housing agencies.

over time. The staff was concerned about maintaining

initial high levels of investment over the long haul.

Expanding Beyond the PRA

Program Partnership

Because staff turnover is a concern for the sustainability

of the program, state agencies have automated

State agencies reported that new collaborations

procedures as much as possible to reduce having to

serving people with disabilities have emerged as a

rely on the actions or knowledge of one person. PRA is

result of participation in the PRA program. The five

often administered by single individuals at state agencies

state housing and health agencies that administer the PRA

with institutional knowledge or unique backgrounds

program in California did not previously work together but

that may not be easily replaced or transferred to new

report that the partnership developed for PRA has led to

staff. Staff turnover has also been an issue for property

other state-level collaborations. For example, state agency

management and service providers. High staff turnover

staff in California are assessing how Medicaid service

means that agencies have to continual y train and educate

dollars can flow more effectively into affordable housing

new staff on the requirements of PRA. State agency and

projects. State housing agency staff also noted that,

partnering staff also need to continual y develop new

beyond PRA, there has been an overall effort between

relationships and contacts.

two of the state housing agencies to coordinate on

similar housing efforts to avoid duplicating work, such as

To address this, the PRA partnerships in the study states

inspections for properties funded by both agencies.

have automated some tenant selection and eligibility

procedures, as well as the notification of unit availability.

Several service providers and property managers

Other states developed written training materials and

also spoke about developing relationships between

procedural manuals so that new staff can work from

their organizations that had not existed before PRA.

detailed instructions. State agencies also report that

Service providers who helped applicants move into PRA

cross-training staff helps to bridge the knowledge gap

reported learning about new housing resources for their

between program partners and facilitates easier staff

other clients, in addition to learning about overall local

transitions.

affordable housing resources. In turn, property managers

report learning about community tenancy supports that

Agencies that have successfully managed staff

may be helpful for some of their other residents.

transitions have engaged multiple staff in certain

functions or have cross-trained with other agencies.

In Louisiana, the state health and housing authorities

For example in Delaware, the state housing agency

report that they have started working closely with the

developed training for service providers who refer

Department of Corrections to figure out how to better

applicants to the PRA program. The agency created a

serve people with disabilities who are re-entering the

PRA training program for service providers and property

community from the state correctional system. “There

managers that included fact sheets and how-to guides,

is definitely a permanent supportive housing 69

a supportive housing website that listed state resources,

population within the corrections system. Louisiana

information on fair housing, and the use of reasonable

has the highest rate of incarceration in the world. We

accommodation requests in appealing denials of eligibility.

are all on board and we are trying to figure that out.

69 Permanent Supportive Housing (PSH) is a housing model that combines affordable housing and supportive services, with admission limited to people who formerly were homeless.

90

H U D S e c t i o n 811 P R A P ro g r a m – P h a s e I I E v a l u a t i o n

Chapter 9. PRA Program Practices that May Lead to Successful Results

The Section 811 program has made it easier to expand

managing waiting lists, working with property owners on

into other programs and identify new opportunities for

reasonable accommodations, and requesting waivers

us.”

for higher contract rents than the grantees had initial y

In Maryland, state agencies have had a longstanding

established in their grant applications. We anticipate that

partnership, and report that they continue to find new

grantees will continue to adapt their approaches as they

ways of col aborating to serve people with disabilities.

gain more experience with the program. Some chal enges,

Like staff in Louisiana, staff at the state housing agency in

especial y around cost and available services, are likely to

Maryland also report replicating some of their partnership

persist nonetheless.

characteristics with other state agencies, and point to

The ultimate goal of the PRA program is to create

new col aborations with the Governor’s Office of Crime

institutional knowledge and capacity within states to

Patrol and Prevention to provide assistance to victims of

further expand the availability of permanent supportive

domestic violence.

housing for people with disabilities. At the core of

this are sustainable partnerships between health and

9.6 Summary

housing agencies that can bring together their respective

resources and expertise. These partnerships grow

Grantees’ effective implementation of the PRA program

over time, and many have their antecedents in the

requires successfully navigating and coordinating several

MFP program. The grantees we evaluated see their

processes. Eligible applicants and units must be identified

partnerships as successful and offer insight into strategies

and matched, and services must be adequate and aligned

to form and deepen them. These include regular meetings

with needs and preferences. Multiple stakeholders must

and communication, recognizing and valuing the expertise

collaborate: state housing and health agencies, property

of each partner, and automating or documenting key

owners, service providers, and potential tenants. The

knowledge and functions so they are not lost when

study found both structural chal enges to successful

individual staff move on to new positions.

PRA implementation, such as limited housing stock,

staffing turnover, and low rents, and a variety of strategies

that had been developed to address these challenges.

Many of these evolved over time, as grantees applied

lessons learned to refine their approaches. These include

91

H U D S e c t i o n 811 P R A P ro g r a m – P h a s e I I E v a l u a t i o n

92

H U D S e c t i o n 811 P R A P ro g r a m – P h a s e I I E v a l u a t i o n

Chapter 10. Conclusion on the Overall Effectiveness of the PRA Program

Conclusion on the

programs, but that program administrative costs are

higher. The study found that costs for healthcare- and

Overall Effectiveness

disability-related services were similar for PRA and PRAC

residents, but dissimilar to such costs for residents in

other HUD programs.

of the PRA Program

The PRA program was designed to respond to a number

of policy priorities:

• Increase the supply of affordable housing for

people with disabilities in a cost-effective way, while

This Final Report on the Evaluation of the Department of

continuing to serve households with extremely low

Housing and Urban Development’s Section 811 Project

incomes.

Rental Assistance program assesses the effectiveness of

• Provide affordable, community-based housing options

PRA compared to other housing options for people with

for people who might otherwise be, or be at risk of

disabilities. The evaluation focused on PRA programs

becoming, homeless or unnecessarily institutionalized.

in six states that had early success in developing state-

• Offer integrated housing settings where people with

level PRA partnerships (California, Delaware, Louisiana,

disabilities live in multifamily housing that assists

Maryland, Minnesota, and Washington). In this chapter, we

people both with and without disabilities.

review the PRA program’s progress toward its goals and

the evidence of its effectiveness in these six states. We

• Encourage collaboration between state housing and

discuss implications of the study’s findings for the entire

health agencies that results in long-term strategies

PRA program going forward and the opportunities for

for providing permanent, affordable housing options

future research.

for people with disabilities, and providing coordinated

access to services.

10.1 How Effective Have States

Here we review key results for each of these priorities.

Been in Meeting the Goals

of the PRA Program?

Increasing the Supply of Affordable Housing

for People with Disabilities: PRA grantees

Overal , the study found that the PRA program assists

have secured units for the program, but at a

people who differ from other people with disabilities in

slower pace than HUD had planned

HUD’s housing programs. The areas of difference include

This study found that PRA grantees are securing units

demographic characteristics, healthcare conditions, the

under agreement and housing eligible households but

types and sizes of properties the people live in, and the

more slowly than had been planned. National y, 27 state

characteristics of the neighborhoods where they live. In

housing agencies are administering PRA grant programs

our survey of PRA and PRAC residents, we found that they

and expect to provide rental assistance for an estimated

report receiving HCBS at similar rates and rate the quality

6,000 households. By September 2018, roughly four years

of their services similarly but that PRA residents rate the

after the first PRA grantees had launched their programs,

quality of their properties and their neighborhoods lower

about three-quarters of planned PRA units were under

than PRAC residents.

agreement, and some 1,200 households had moved into

In our analysis of Medicaid claims data for PRA residents

units. In the six states in this study, grantees had contracts

about a year after they moved in, we found that PRA

in place with owners for more than half of planned units

residents use healthcare services differently than

and had identified properties for most of their awarded

members of all of the comparison groups being assisted

rental subsidies. In the study states, about three-quarters

by PRA. However, most of these differences were not

of planned units were under agreement and half units were

statistical y significant. The only statistical y significant

occupied by PRA residents.

difference we found was the significantly lower rates of

As described in the Phase I evaluation of PRA, two factors

healthcare use by PRA residents relative to similar people

slowed the pace of leasing units. First, owner recruitment

not assisted by HUD programs.

was chal enging, because some eligible owners were

The component of this study that assesses the cost-

reluctant to commit to long-term contracts, under a new

effectiveness of PRA in relation to other HUD programs

and unfamiliar program, at rents at or under FMR. Second,

that assist people with disabilities found that PRA rental

on the household side, identifying eligible applicants and

subsidy costs are similar to or lower than for other HUD

matching them with available and suitable units takes time

93

H U D S e c t i o n 811 P R A P ro g r a m – P h a s e I I E v a l u a t i o n

Chapter 10. Conclusion on the Overall Effectiveness of the PRA Program

and administrative effort. Many local partners praised

point to the need for greater coordination between

the program for providing opportunities for persons who

state agencies, property owners, and service providers

wanted to live independently, but they also noted that not

to ensure PRA residents have the support they need

enough PRA units are available, which limits choices for

immediately after move-in and going forward throughout

applicants.

their tenancy.

The pace of leasing seems to be picking up. At the

The PRA program’s requirement that housing and health

same time, some PRA residents are expressing concern

agencies formal y collaborate helps state housing and

about the housing quality of their units, properties, and

health agencies engage with and learn from each other

neighborhoods. This raises questions about the balance

and effectively serve the high-needs populations the

between pressures to lease units within the grant period

program hopes to reach. Grantees uniformly report that

and ensuring the housing meets PRA residents’ needs

the administrative cost is considerable for them and for

and is in high-quality properties and neighborhoods.

many of their partners. States with established structures

The study’s cost-effectiveness analysis tentatively

for CMS’s MFP program often had a head start on this

concludes that the PRA program is as cost-effective as

col aboration, and most hope to continue to support the

the PRAC program but at a considerable administrative

staffing and systems put in place for MFP after those

cost. Administrative costs may level off as the grantees

grants end. This requires that states commit resources for

gain more experience and develop a larger pipeline of

doing so.

available units. On the other hand, some grantees that set

The study found that, once housed, PRA residents report

rents below FMR had trouble recruiting owners in higher-

for the most part that they receive the services they want

cost areas and received approval from HUD to increase

and need. PRA residents surveyed report getting more

rents. This will increase costs and reduce the number of

pre-move and transition services to help them move

households the fixed grant amount can assist.

to these settings than do their counterparts in PRAC

Expanding Housing Opportunities for People

properties. More than two-thirds of both PRA and PRAC

with Significant Healthcare and Service

residents report receiving some assistance in moving

Needs: The PRA program is housing its

into their new home. However, the study identified some

intended target populations

gaps in the coordination of services and in the types of

transition and ongoing services available to residents.

PRA grantees succeeded in reaching eligible people

Less than half of surveyed PRA and PRAC residents

who were living in institutions, homeless, or at risk of

reported receiving information on how to live on their own

homelessness or institutional care. As of September 2018,

when moving into their unit. About one-fourth to one-third

grantees reported that more than a quarter (27 percent)

of residents surveyed reported going without a needed

of the 725 households assisted with PRA in the study

service at least sometimes. In addition, our interviews

states to date had previously lived in an institution, and

with state agency staff, property managers, and service

about 20 percent had been homeless. About one-third

providers highlighted some areas where services are

of the remainder were at risk of institutionalization or

unavailable or insufficient to meet all the needs of PRA

homelessness. Healthcare utilization data confirm that the

residents.

PRA program appears to serve a higher-need population

relative to other HUD programs, with a higher prevalence

Service needs and gaps vary by state and by target

of developmental, mental health, and other disabling

population and affect residents both before and after

conditions before entering the PRA program.

moving into a PRA property. Areas with gaps in services

that service providers and property managers reported

Grantees continue to report high rates of ineligibility,

during our interviews include mental health support,

however. In particular, applicants experiencing

medication management, employment search and

homelessness often have poor credit or housing histories

readiness, mental health and substance use disorder

or have criminal backgrounds that make them ineligible

treatment, and transportation.

for housing at particular properties. Grantees are working

with owners and service providers to reduce barriers to

Expanding Housing Opportunities in

property eligibility, and the denial rate is going down. Exits

Integrated Settings: The evidence is mixed

from the program are high in the study states, as about

Grantees are securing units in properties that meet the

a fifth of PRA residents exit the program each year in the

25 percent cap on set-asides for people with disabilities.

first three years after move-in. The study also found that

In fact, the average percentage of PRA units per

PRA residents have higher rates of exits due to tenant

development is 10, well below the cap. Anecdotal y, we

nonpayment of rent than PRAC does. These findings

94

H U D S e c t i o n 811 P R A P ro g r a m – P h a s e I I E v a l u a t i o n

Chapter 10. Conclusion on the Overall Effectiveness of the PRA Program

also heard that some potential PRA applicants would

Building Institutional Capacity at the

prefer to live in properties even smal er than the five-unit

State Level: PRA grantees established

minimum that PRA requires. PRA residents also report

partnerships at the state and local levels,

high levels of autonomy and personal choice in some

often building on existing relationships from

of their daily activities. Almost all PRA residents report

previous supportive housing efforts, but at

being able to be alone when they want to be and to eat

considerable administrative cost

when they want. Not all PRA residents report being able

PRA partnerships are beneficial in many ways but can

to exercise their personal choices at all times, however.

be time-consuming and resource-intensive. Even if the

More than a quarter of PRA residents report not always

effort and administrative cost were to level off, grantees

being able to see their friends and family when they want

would still be concerned about securing resources to

to see them, significantly more than the 12 percent of

sustain program staffing. Staff turnover is a concern, as

PRAC residents who say the same. Additional y, a fifth of

state-level staff in particular often have unique institutional

PRA residents said they have difficulty getting around their

knowledge of the program that is not easily replaced.

neighborhood, at least sometimes. Less than half of PRA

residents report viewing more than one unit before moving

Grantees and state health agency staff are addressing

into their current home.

this by cross-training staff within and across agencies,

developing written manuals, and automating procedures

We conclude that, at its current scale, the PRA program

to both increase program efficiency and mitigate the

offers housing in properties where people with and

effects of turnover, but they report that more technical

without disabilities live, but residents may not always be

assistance is needed on how to integrate housing

satisfied with the specific units or neighborhoods that are

assistance and healthcare and supportive services. Topics

available to them. With the possible exception of PRAC, al

for such assistance include guidance on procedures,

of the comparison programs offer options that, in theory,

on what is and is not permitted under the program,

are in integrated settings. The comparison programs

and on what program changes require HUD approval.

are also well established and have more units, and so

Program staff also noted that some owners still have

may offer more choices just by virtue of their size relative

misconceptions about the PRA program and its target

to PRA. We learned that providers that offer transition

populations, suggesting a need for training on contracting

services often screen for PRA eligibility while also signing

and the administration of rental assistance payments

people up for Housing Choice Vouchers and public

and for education about the nature and needs of the

housing waiting lists. Providers report anecdotal y that

target populations. Training on tenant selection and unit

many of their clients would eagerly accept a voucher that

matching emerged as a specific training need for service

could be used anywhere owners accept them and may be

providers.

receiving Medicaid-funded supportive services anyway.

As a practical matter, however, the alternative for many in

10.2 Policy Implications and Areas

the PRA’s target populations is not necessarily a voucher

for Future Research

program or public or other HUD-assisted multifamily

housing. It is living in institutions or being served by the

The results of our evaluation have several policy

homeless assistance system (potential y with high use

implications for HUD and HHS to consider, and several

of costly crisis care and other services). Some states

areas for possible future research.

specifical y target people they see as the hardest to serve

and who might not be successful with a traditional housing

Program Monitoring and Data Sharing

voucher if the voucher has no connection to services. The

The Phase II evaluation provides evidence on short-term

PRA program’s coordinated access to services could play

outcomes for residents who have been receiving PRA

a key role in successful tenancy. The evidence from our

assistance for about a year on average. Further, the

impact analyses points in that direction, but the follow-up

study had access to data for only about 10 percent of the

period is too short to conclude definitively that that is the

households that states expect to assist national y. We

case.

recommend that HUD continue to monitor outcomes for

PRA residents over time and for all of the 27 states that

received PRA grants in FY12 and FY13.

Among the six study states, only one (Washington) had

a formal agreement between the housing and health

agencies to share data. In the remaining states, the

95

H U D S e c t i o n 811 P R A P ro g r a m – P h a s e I I E v a l u a t i o n

Chapter 10. Conclusion on the Overall Effectiveness of the PRA Program

agencies work together to complete required quarterly

While state housing agencies have flexibility in selecting

progress reports to HUD but do not share person-level

properties and locations where they can use PRA, they

data. State housing agencies should consider pursuing

are also constrained by some of the PRA program’s

opportunities with their partnering agencies to share data

requirements for integration, and for the specific types of

to assess outcomes for their state’s PRA residents and

units that PRA target populations typical y need (mostly

potential y to compare them to outcomes for other similar

one-bedrooms, and in some cases, accessible or ground-

populations in the state. These populations could include

floor units). Most PRA residents live in properties financed

people served by other supportive housing programs

by LIHTC. As a result, each state’s LIHTC requirements

in the state or people served under specific Medicaid

and incentives affect the types of properties and

waivers. Such agreements should include policies and

neighborhoods where PRA can be used. These locations

practices to protect people’s privacy. Similarly, we

and properties may not always align with the needs and

encourage HUD and CMS to pursue similar opportunities

preferences of the state’s PRA target population.

to share data at the national level to better understand the

It is difficult to identify studio or one-bedroom units for

populations they assist and to evaluate joint initiatives.

PRA in properties that are funded by tax credits or other

With respect to monitoring PRA implementation, several of

federal or state subsidies. Some developers may restrict

the grantees we spoke with noted that quarterly reporting

the number of smaller-sized units in affordable housing

requirements for PRA can be burdensome and that the

developments because they bring in less average rent

state agencies have difficulty collecting and calculating

than units with more bedrooms. The studios and one-

their data in the manner requested by HUD. Further, it is

bedrooms they do construct are often in response to

not clear to grantees whether the frequent reporting and

capital subsidy financing incentives for setting aside a

the level of detail requested are necessary for monitoring

minimum number of units for seniors or for people with

PRA’s progress, given the expected long-term nature of

disabilities, and these incentives may be inconsistent

the program. HUD should consider streamlining reporting

with the PRA program’s 25 percent cap on set-asides for

requirements for grantees and should seek grantees’ input

people with disabilities.

on specific areas where data is chal enging to col ect or to

In some cases, these constraints have led grantees to

report in the manner requested.

contract for PRA units in older properties that some PRA

In order to streamline data col ection for the quarterly

residents rate as having poor quality or that have unit sizes

report, grantees suggest that HUD request only the

or accessibility features that do not necessarily match

data they need for monitoring or reporting purposes

what applicants need. While properties still need to meet

and not request data that can be obtained from other

federal standards for safe, sanitary, and decent housing,

HUD sources or are submitted to HUD in a different

we learned from staff interviews and from surveys with

manner. Specific questions that grantees and their state

PRA residents that some units are in older properties with

agency partners reported as chal enging to answer were

persistent maintenance needs or in geographic areas and

those that asked about applicants’ prior living situation

neighborhoods where residents may not necessarily want

or applicants’ or residents’ type of disabling condition,

to live.

since this information was not always reported or easily

Grantees are required to inspect PRA properties at least

obtained. Grantees also reported that some confusion

every three years and make inspection results available as

resulted from the fact that some questions were point-

part of regular monitoring reviews. To improve the quality

in-time, cumulative counts and others were quarter-only

of PRA units under contract, HUD could require that

counts. They suggest being consistent on this throughout

properties be inspected according to UPCS standards

the report. HUD may also want to consider reducing the

before units are placed under contract for PRA, and

frequency of reports to semiannual or annual.

that individual units are inspected before they can be

Greater Flexibility in Identifying PRA Units

leased by PRA residents. Although grantees in most of

That Meet Resident Needs and Preferences

the study states conduct inspections of PRA units more

frequently than every three years, HUD could make this a

PRA residents have diverse needs and differing

requirement. HUD could request inspections information

preferences for properties and neighborhoods. Unlike

as part of regular grant progress reports (for example,

in the tenant-based programs such as HCV and NED,

require grantees to report when a unit fails inspection),

applicants for property-based rental assistance programs

through resident surveys, or through random audits of

are limited in the number of choices they have in where

properties outside of their regular monitoring review.

they can live affordably.

96

H U D S e c t i o n 811 P R A P ro g r a m – P h a s e I I E v a l u a t i o n

Chapter 10. Conclusion on the Overall Effectiveness of the PRA Program

State agency and partner staff we spoke with suggested

and their partners have adequate staffing and financial

areas for greater flexibility in where and how PRA can be

resources to maintain the level of PRA staffing needed

used. Their recommendations include:

to successful y implement PRA over the length of the

• HUD should continue to consider waivers for grantees

PRA grant and the 20-year contract period. Grantees

to increase the maximum al owable rent for PRA-

are finding ways to mitigate the consequences of relying

assisted units above the level established in their

on a few people with institutional knowledge to manage

Cooperative Agreements, especial y in high-cost areas.

the PRA program, and also the consequences of staff

This will al ow grantees to provide more PRA options

turnover, which is particularly high for case managers

in high-cost areas close to services and transportation

who work with PRA residents and for property managers.

and to improve the likelihood that owners will want

HUD and CMS should continue to support the efforts of

to administer the PRA program in their properties. A

grantees and their state and local partners by providing

downside to this approach is that as average subsidies

technical assistance, tools, and templates that agencies

increase, grantees will serve fewer households with

can modify for their own use.

their PRA grant funds.

While many state agency staff spoke favorably of

• In states where studios or one-bedroom units are

the assistance provided by HUD and their technical

particularly hard to place under contract for PRA,

assistance contractor, property owners and service

HUD should consider waivers to allow households that

providers do not have access to this guidance and

qualify for one-bedroom units to occupy and receive

training. HUD may want to consider expanding their

subsidies for two-bedroom units.

technical assistance to property owners and service

providers or providing additional support to grantees to

• HUD should encourage state housing agencies to

provide this assistance to their partners. Areas where

place more units under contract with owners than

owners report needing additional information or training

are estimated in Cooperative Agreements between

include the rental assistance contracting process, entering

grantees and HUD. In addition to providing applicants

tenant data for payment, tenant eligibility including

more choices in property type and neighborhood, this

calculation of income and rent, and use of reasonable

will al ow grantees to better manage uncertainties in

accommodation requests.

resident turnover in existing properties and potential

delays in development schedules for new construction

Owners also noted that they would like additional

properties.

guidance from the state partnerships about the potential

needs of the state’s target population and clear guidance

• HUD should provide additional guidance to grantees

on how to prioritize one applicant over another and on

on how they can pursue using PRA in more scattered-

how to put written tenant support plans into practice.

site properties. HUD should al ow greater flexibility in

Service providers report wanting additional guidance

the types of properties where PRA can be used. We

on PRA eligibility requirements, requesting reasonable

heard anecdotal y that PRA applicants may prefer to

accommodation requests for their clients, and on other

live in smal er properties, including those with less than

affordable housing options in their communities.

the minimum five units required for the PRA program.

Placing more PRA units in scattered-site housing may

Another anticipated impact of the PRA program was to

help grantees address this preference. Although the

promote effective use of healthcare services. Housing is

PRA statute al ows PRA to be used in “scattered sites”

considered an important social determinant of health, and

where units for one contract are in more than one

stable housing can contribute to improved health status

property or location, only a small percentage of PRA

and self-care, as well as reducing a person’s incidence of

residents are assisted in these types of properties.

unplanned and emergency care. Furthermore, community-

based supports, such as PCA, are general y preferred by

• HUD could consider creating incentives in future PRA

people with disabilities and can be less costly on average

funding rounds for grantees to locate PRA properties

than institutional care. The short observation period for

in low poverty areas, areas with more racial integration,

PRA-supported residency limits our ability to observe

or other indicators of opportunity.

definitive policy implications in the area of healthcare

Sustaining and Expanding the

impacts. We did observe some differences in service

PRA Partnerships

utilization over the short term that could translate into

long-term trends. This may hinge on continued funding

Whether the PRA program is sustainable and ultimately

through Medicaid or other sources of the full complement

successful depends in part on whether state agencies

of community supports, including tenancy services.

97

H U D S e c t i o n 811 P R A P ro g r a m – P h a s e I I E v a l u a t i o n

Chapter 10. Conclusion on the Overall Effectiveness of the PRA Program

Is PRA an Effective Approach?

of various data sources, and the small number of PRA

The Phase II evaluation assessed whether PRA had

residents in our sample compared to the total number

achieved its goals for integration, cost-effectiveness,

of households that all 27 PRA grantees expect to assist

and coordinated access to services for residents, and

with their FY12 and FY13 grants. As such, the study team

whether PRA is an effective alternative to the PRAC,

identified several areas for future research:

NED, or other HUD programs that also provide housing

• HUD may want to expand the research of the PRA

assistance to non-elderly people with disabilities. Overal ,

program to all states where the PRA program is being

PRA and PRAC residents rate their properties and home

implemented. Using HUD administrative data, HUD

and community-based services similarly. The study found

can analyze and compare household, property, and

significantly lower rates of use of long-term inpatient care

neighborhood characteristics for a larger sample

services and higher use of case management services

of PRA residents, and assess how applicable the

for PRA residents compared to similar people enrol ed in

study’s results are to the entire PRA program.

Medicaid and those not served by HUD programs. While

Alternatively, HUD could use the administrative data

some of the early findings on neighborhoods and resident-

already col ected for this study to generalize a subset

reported health status suggest that PRA residents may be

of findings for the national PRA population. This

worse off than they would be if assisted by PRAC or other

analysis would only include HUD and neighborhood

HUD programs, these findings represent outcomes from

administrative data for households and properties and

only a subset of the PRA residents assisted national y and

not healthcare data due to the data use restrictions of

a fraction of the residents whom PRA will eventual y serve,

the study.

and these findings are for people with about a year of PRA

• HUD administrative data does not provide complete

assistance on average.

information on reasons for program exits. HUD may

The scale of the PRA program is still modest compared

want to consider adding additional elements to regular

to among other HUD programs that assist non-elderly

grantee progress reports to learn more about why

people with disabilities. The PRAC program has about

residents leave PRA units, whether they are for tenant-

34,000 units for non-elderly people with disabilities, and

initiated or owner-initiated reasons, and whether there

the NED program assists roughly 55,000 households.

are differences in characteristics of residents who

In addition, for the first time since 2005, in 2018 HUD

remain in the PRA program versus those who exit.

awarded $98 mil ion in subsidies to 286 PHAs to expand

• Future evaluations of the Section 811 PRA program

the Mainstream Voucher program, a tenant-based voucher

should look at longer-term outcomes after the program

program for people with disabilities.70 Another large

has had time to mature and after enough time has

funding competition for these vouchers was announced in

passed to more accurately assess the effect of the

2019. While this was not a requirement, HUD encouraged

PRA on residents.

PHAs to target these vouchers to non-elderly people

with disabilities who are transitioning from institutions or

- The study team recommends revisiting the status of

homelessness or are at risk. The 2018 and 2019 funding

PRA grant implementation presented in this report in

notices also encouraged PHAs to partner with state

another two years. A potential milestone for fol ow up

Medicaid agencies or other entities that could provide

could be when all PRA units that are funded in the

resources for services. These tenant-based vouchers

first two funding rounds are expected to be under

wil further expand housing choices for people with

contract with owners: September 2020 for FY12

disabilities, perhaps with more flexibility and with less

grantees and September 2021 for FY13 grantees.

administrative burden than is the case with the project-

- The timeline for revisiting healthcare utilization data

based PRA program. HUD should continue to monitor the

needs to consider the minimum 18-month lag for

results of the PRA program relative to the range of other

Medicare and Medicaid claims data. A three or

options.

more year post-occupancy period would be ideal for

finding stronger evidence of changes in patterns of

Areas for Future Research

healthcare utilization and spending.

This evaluation assessed early outcomes of PRA residents

- Given a substantive portion of PRA residents are

relative to similar individuals in four comparison groups.

likely to be dual enrol ed in Medicare and Medicaid it

As noted throughout this report, the study had several

would be important for a future evaluation to col ect

limitations related to the timing of the study, the availability

Medicare and Medicaid administrative data covering

70 https://www.hud.gov/program_offices/public_indian_housing/programs/hcv/mainstream

98

H U D S e c t i o n 811 P R A P ro g r a m – P h a s e I I E v a l u a t i o n

Chapter 10. Conclusion on the Overall Effectiveness of the PRA Program

the full pre/post periods in order to accurately

measure healthcare utilization and spending.

• This evaluation compared outcomes of the PRA

program to outcomes for PRAC, NED, and other

HUD-assisted housing programs. Given that PRA

residents were found to have different demographic

and socioeconomic characteristics and to have higher

rates of chronic and disabling conditions, HUD may

want to consider whether these comparison groups are

the right ones. Future impact studies could compare

outcomes between other groups of individuals similar

to those assisted by PRA, such as people experiencing

homelessness, people residing in institutional facilities,

and people living in the community and receiving

services under HCBS waivers.

• HUD should explore further how the 25 percent cap

on set-asides for people with disabilities may limit the

number of properties where PRA can be offered. In

some states, the cap is inconsistent with incentives

set by state tax credit programs or other funding

sources. While properties may not have specific set-

asides for people with disabilities that exceed PRA’s

requirements, it is likely that some percentage of

people living in the non-PRA units include someone

with a disability in the household.

• The study reviewed capital financing data for a

limited sample of PRAC properties. Through analysis

of existing administrative data, HUD may want to

consider expanding this sample in the six study states

or in all states with PRA programs.

• HUD may wish to col ect more nuanced data on why

units under contract for PRA are unavailable for PRA.

HUD could request that grantees report the reasons

for unavailable units in their grant progress reports, or

HUD could obtain qualitative information from grantees

during ongoing technical assistance webinars. HUD

may also wish to develop monitoring tools to help HUD

and grantees monitor the average time it takes units

under contract to be leased and average vacancy rates

in units under contract for PRA.

• Any future research on the PRA program should

continue to take into consideration the perspective of

the people who the program assists – the residents.

HUD may want to expand the implementation of the

Section 811 resident survey to residents in all grantee

states or make it available for grantees to administer in

their own states. HUD could also consider developing

a hotline for residents so that they can report any

questions or concerns they have with their property or

neighborhood.

99

H U D S e c t i o n 811 P R A P ro g r a m – P h a s e I I E v a l u a t i o n

100

H U D S e c t i o n 811 P R A P ro g r a m – P h a s e I I E v a l u a t i o n

Appendix A: Status of National PRA Program Implementation as of September 2018

Appendix A: Status of North Carolina (FY12), the District of Columbia (FY13), and

Kentucky (FY13). This section provides an overview of the

National PRA Program 27 ongoing PRA grant awards.

Grantees entered into Cooperative Agreements with HUD

to establish targets for the number of units expected to be

Implementation as

assisted by the PRA grants and to govern other activities

under the grants. Each grant funds a maximum of 60

of September 2018

months of rental assistance per unit. Grantees can use up

to 8 percent of their grant awards toward administrative

costs. As state grantees have implemented their PRA

programs, some worked with HUD to update their

Cooperative Agreements with changes in the number of

This appendix summarizes the implementation status of

units they anticipate funding based on changes to planned

27 Section 811 Project Rental Assistance (PRA) programs

PRA property locations and the cost of rental units in

through September 2018. The appendix is based on data

those locations. Some states have also received minor

from Section 811 PRA Program Quarterly Reports and

changes in funding.

administrative data on tenants and properties from HUD’s

On average, HUD awarded $7.2 mil ion per FY12 grantee

Tenant Rental Assistance Certification System (TRACS)

and $6.2 mil ion per FY13 grantee. Funding amounts and

and Integrated Real Estate Management System (iREMS).

the number of units that grantees expect to assist with

The appendix provides an overview of the status of the

their grants vary widely:

first two PRA grant funding rounds. We present an update

• Across the two funding rounds, the grant amounts

on grantees’ efforts to establish contracts with owners

ranged from $2 mil ion to $24 mil ion.

for PRA rental subsidies, and data on units occupied by

PRA residents as of September 30, 2018. The appendix

• Five states expect to provide PRA subsidies for fewer

presents information on the number of bedrooms and

than 100 rental units. At the higher end of the range,

accessibility of units under contract and occupied by

Il inois and Georgia both expect to provide PRA

PRA residents. Final y, the appendix presents aggregate

subsidies for more than 500 units over the 2 funding

data on applicants to the PRA program, including

years.

eligibility determination and referral sources, and limited

• FY12 grantees expect their grants to fund an average

characteristics of households who have moved into PRA

of 190 units per state, whereas FY13 grantees expect

units.

their grants to fund an average of 169 units.

A.1 Overview of Section 811

As of September 2018, 27 grantees had entered into

Cooperative Agreements with HUD for $88 mil ion in FY12

PRA Grant Awards

grant funds for an estimated 2,283 units, and $142 mil ion

in FY13 grant funds for an estimated 3,772 units. Between

Congress approved funding for two rounds of HUD

the 2 grant years, PRA funding is expected to provide

grants for the Section 811 PRA program. HUD awarded a

rental subsidies for an estimated 6,055 units. Exhibit A.1

demonstration round of grants for Fiscal Year 2012 (FY12)

shows each state’s PRA grant amount and the estimated

in February 2013, and a second round of grants for Fiscal

number of units the grantee expects its PRA subsidy to

Years 2013 and 2014 in March 2015 (FY13). Across the two

fund per its Cooperative Agreement.

funding rounds, HUD awarded funding to 30 state housing

agencies: 13 in FY12 and 25 in FY13 (eight states received

funding in both rounds). Three grantees left the program:

101

H U D S e c t i o n 811 P R A P ro g r a m – P h a s e I I E v a l u a t i o n

Appendix A: Status of National PRA Program Implementation as of September 2018

E xhibit A .1: F Y12 and F Y13 P R A Grant Funding Awards and Planned P R A Unit s

FY12

FY13

State

Grant Amount

Planned Units

Grant Amount

Planned Units

Alaska

-

-

$7,722,343

160

Arizona

-

-

$2,950,000

54

California

$12,208,558

233

$11,985,436

200

Colorado

-

-

$7,610,719

157

Connecticut

-

-

$4,112,906

150

Delaware

$5,100,753

148

-

-

Georgia

$4,279,650

233

$10,174,407

350

Illinois

$12,324,352

369

$6,420,000

200

Louisiana

$8,489,928

199

-

-

Massachusetts

$5,427,208

90

$6,803,050

107

Maryland

$11,229,308

150

$9,808,054

150

Maine

-

-

$2,000,000

59

Michigan

-

-

$5,516,950

174

Minnesota

$3,085,500

85

$3,000,000

75

Montana

$2,057,000

81

-

-

New Hampshire

-

-

$8,634,824

191

New Jersey

-

-

$5,099,229

206

New Mexico

-

-

$2,278,447

50

Nevada

-

-

$2,000,000

44

Ohio

-

-

$11,991,399

485

Oregon

-

-

$2,335,000

75

Pennsylvania

$5,870,880

200

$8,557,014

205

Rhode Island

-

-

$5,627,829

150

South Dakota

-

-

$2,797,972

135

Texas

$12,342,000

362

$12,000,000

293

Washington

$5,739,717

133

-

-

Wisconsin

-

-

$2,532,090

102

Total

$88,300,377

2,283

$141,957,669

3,772

Source: Abt Associates analysis of Section 811 PRA Quarterly Reports for quarter ending September 2018.

A.2 Status of Units under

affordable rental housing and to developments under

Agreement for the PRA Program

construction (or substantial rehabilitation) or planned to be

developed.

This section provides information on PRA units that are

In order to move residents into the selected properties,

under contract or otherwise committed by owners as of

a grantee must first reach an agreement with property

September 2018. Grantees select properties to receive

owners that they will lease units to residents. Rental

PRA subsidies through a Notice of Funding Availability

Assistance Contracts (RACs) are 20-year agreements

(NOFA) or through their state’s Low Income Housing

between the grantee and the owner of the eligible

Tax Credit (LIHTC) program or other capital funding

multifamily property. The agreement identifies the

mechanism. Grantees award units both to existing

number of units the property owner agrees to commit

developments already constructed and operating as

to the program, the bedroom sizes of the units, and the

102

H U D S e c t i o n 811 P R A P ro g r a m – P h a s e I I E v a l u a t i o n

Appendix A: Status of National PRA Program Implementation as of September 2018

maximum al owable rent the owner may charge for a unit

Status of Rental Assistance Contracts

by bedroom size. Owners must also agree to a 30-year

between Grantees and Owners

affordability restriction.

By September 2018, FY12 grantees had executed

Both FY12 and FY13 grants have a deadline for

210 RACs with property owners for an estimated

disbursement of rental assistance funds. FY12 grants

1,718 PRA units, representing 75 percent of units in

must be disbursed by September 30, 2025, and FY13

grantees’ Cooperative Agreements (Exhibit A.2). The

grants must be disbursed by September 30, 2026. After

average number of units per executed RAC for FY12

that time, the funds are canceled. The grant provides

grantees is eight, with a range between five and thirteen.

five years of rental assistance. If owners want to benefit

Massachusetts had the lowest average number of units

from the full 5 years of rental assistance before the funds

per RAC at 2.8 (9 RACs for 25 units total), and Montana

expire, leasing must start by September 30, 2020, for FY12

had the highest average number of units per RAC, with

grants and by September 30, 2021, for FY13 grants.

12.5 (6 RACs for 75 units).

E xhibit A .2 : R AC s E xecuted with F Y12 Grantees through September 2018, by S t ate

# of PRA Units

# of PRA Units in

Average # of PRA

in Cooperative

State

# of RACs Executed

Executed RACs

Units per RAC

Agreement

% Units under RAC

California

11

128

11.6

233

55%

Delaware

22

115

5.2

148

78%

Georgia

23

208

9.0

233

89%

Illinois

18

164

9.1

369

44%

Louisiana

21

255

12.1

199

128%

Massachusetts

9

25

2.8

90

28%

Maryland

8

73

9.1

150

49%

Minnesota

9

84

9.3

85

99%

Montana

6

75

12.5

81

93%

Pennsylvania

41

202

4.9

200

101%

Texas

27

302

11.2

362

83%

Washington

15

87

58.0

133

65%

Total

210

1,718

8.2

2,283

75%

Source: Abt Associates analysis of Section 811 PRA Quarterly Reports for quarter ending September 2018.

FY13 grantees executed 118 RACs for an estimated

grantees entered into RACs with owners for an average of

516 PRA units, or 14 percent of their planned PRA units

4.3 units per RAC, about one-half of the average number

(Exhibit A.3). Eighteen of the 23 grantees awarded in FY13

of units per RAC for FY12 grantees. The average number

had units under RAC. Of the remaining five FY13 grantees

of units per RAC ranged from 2.3 in New Hampshire (36

without any RACs executed as of September 2018, three

RACS for 81 units) to 28 in Wisconsin (2 RACS for 57

(California, Il inois, and Massachusetts) were also awarded

units).

FY12 grant funds and had units under RAC for FY12. FY13

103

H U D S e c t i o n 811 P R A P ro g r a m – P h a s e I I E v a l u a t i o n

Appendix A: Status of National PRA Program Implementation as of September 2018

E xhibit A .3 : C ontract s E xecuted with F Y13 Grantees through September 2018, by S t ate

# of PRA Units

% Cooperative

# of PRA Units in

Average # of PRA

in Cooperative

Agreement Units

State

# of RACs Executed

Executed RACs

Units per RAC

Agreement

under RAC

Alaska

1

5

5.0

160

3%

Arizona

7

41

5.9

54

76%

California

0

0

NA

200

0%

Colorado

1

20

20.0

157

13%

Connecticut

3

31

10.3

150

21%

Georgia

6

36

6.0

350

10%

Illinois

0

0

NA

200

0%

Massachusetts

0

0

NA

107

0%

Maryland

2

8

4.0

150

5%

Maine

1

4

4.0

59

7%

Michigan

2

12

6.0

174

7%

Minnesota

5

28

5.6

75

37%

New Hampshire

36

81

2.3

191

42%

New Jersey

26

69

2.7

206

33%

New Mexico

0

0

NA

50

0%

Nevada

0

0

NA

44

0%

Ohio

4

14

3.5

485

3%

Oregon

3

8

2.7

75

11%

Pennsylvania

9

43

4.8

205

21%

Rhode Island

5

19

3.8

150

13%

South Dakota

2

16

8.0

135

12%

Texas

4

24

6.0

293

8%

Wisconsin

2

57

28.3

102

56%

Total

119

516

4.3

3,772

14%

Source: Abt Associates analysis of Section 811 PRA Quarterly Reports for quarter ending September 2018.

Status of Other Agreements with Owners for

Grantees expect owners who enter into ARACs to

PRA Units

eventual y enter into RACs once the initial leasing date is

Prior to executing an RAC, grantees and property owners

near; however, ARACs do not contractual y bind owners

can also sign Agreements to Enter into RAC (ARACs) or

to make units available to the PRA program. Although

other written agreements that indicate a firm commitment

ARACs are supposed to represent firm commitments by

to eventual y enter into an RAC for an assisted unit.

the owner, the number of units under ARACs reported by

Other written agreements can include memorandums of

the grantee might not always reflect units that will later be

understanding (signed by the grantee and the owner);

funded.

LIHTC or other funding award letter signed by the funder

Exhibit A.4 presents the status of FY12 grantees’ entering

(which may or may not be the grantee); or a letter of

into ARACs or other agreements with owners through

agreement or commitment signed solely by the grantee.

September 2018. FY12 grantees entered into 127 ARACs

Many state grantees signed ARACs or other agreements

or other agreements for a total of 1,238 units, an average

with property owners constructing new properties after

of 9.7 units per ARAC. Combining units under RAC (1,718

being awarded PRA subsidies through the state’s LIHTC

units) and units under ARAC (1,238 units), FY12 grantees

program or through other multifamily housing funding

identified 2,956 units, or 129 percent of the PRA units in

programs.

their Cooperative Agreements.

104

H U D S e c t i o n 811 P R A P ro g r a m – P h a s e I I E v a l u a t i o n

Appendix A: Status of National PRA Program Implementation as of September 2018

E xhibit A .4 : S t atus of A R AC s with F Y12 Grantees through September 2018, by S t ate

# of ARACs

# of Units in

Average # of

PRA 5-Year

% Units under

# of RAC and

% of PRA Unit

State

Executed

ARACs

Units per ARAC

Unit Goal

ARAC

ARAC Units

Goal Identified

California

8

96

12.0

233

41%

224

96%

Delaware

1

6

6.0

148

4%

121

82%

Georgia

0

0

NA

233

0%

208

89%

Illinois

1

10

10.0

369

3%

174

47%

Louisiana

0

0

NA

199

0%

255

128%

Massachusetts

9

29

3.2

90

32%

54

60%

Maryland

11

100

9.1

150

67%

173

115%

Minnesota

0

0

NA

85

0%

84

99%

Montana

1

2

2.0

81

2%

77

95%

Pennsylvania

1

3

3.0

200

2%

205

103%

Texas

90

946

10.5

362

261%

1,248

345%

Washington

5

46

9.2

133

35%

133

100%

Total

127

1,238

9.7

2,283

54%

2,956

129%

NA = not applicable.

Source: Abt Associates analysis of Section 811 PRA Quarterly Reports for quarter ending September 2018.

As shown in Exhibit A.5, FY13 grantees have entered

FY13 grantees. When the number of units under RAC and

into 118 ARACs for 746 units as of September 2018. The

ARAC are combined, FY13 grantees had identified 1,262

average number of units per ARAC for FY13 grantees is

units, or about 33 percent of their Cooperative Agreement

6.3, nearly double the average of 3.7 units per RAC for

units.

E xhibit A .5 : S t atus of A R AC s with F Y13 Grantees through September 2018, by S t ate

# of ARACs

# of Units in

Average # of

PRA 5-Year

% Units under

# of RAC and

% of PRA Unit

State

Executed

ARACs

Units per ARAC

Unit Goal

ARAC

ARAC Units

Goal Identified

Alaska

0

0

NA

160

0%

5

3%

Arizona

0

0

NA

54

0%

41

76%

California

3

33

11.0

200

17%

33

17%

Colorado

5

66

13.2

157

42%

86

55%

Connecticut

2

16

8.0

150

11%

47

31%

Georgia

0

0

NA

350

0%

36

10%

Illinois

0

0

NA

200

0%

0

0%

Massachusetts

0

0

NA

107

0%

0

0%

Maryland

20

121

6.1

150

13%

129

86%

Maine

2

6

3.0

59

10%

10

17%

Michigan

9

83

9.2

174

48%

95

55%

Minnesota

12

50

4.2

75

67%

78

104%

New Hampshire

7

28

4.0

191

15%

109

57%

New Jersey

3

12

4.0

206

6%

81

39%

New Mexico

0

0

NA

50

0%

0

0%

Nevada

0

0

NA

44

0%

0

0%

( cont )

105

H U D S e c t i o n 811 P R A P ro g r a m – P h a s e I I E v a l u a t i o n

Appendix A: Status of National PRA Program Implementation as of September 2018

E xhibit A .5 : S t atus of A R AC s with F Y13 Grantees through September 2018, by S t ate ( cont )

# of ARACs

# of Units in

Average # of

PRA 5-Year

% Units under

# of RAC and

% of PRA Unit

State

Executed

ARACs

Units per ARAC

Unit Goal

ARAC

ARAC Units

Goal Identified

Ohio

50

300

6.0

485

62%

314

65%

Oregon

0

0

NA

75

0%

8

11%

Pennsylvania

1

3

3.0

205

1%

46

22%

Rhode Island

0

0

NA

150

0%

19

13%

South Dakota

4

28

7.0

135

21%

44

33%

Texas

0

0

NA

293

0%

24

8%

Wisconsin

0

0

NA

102

0%

57

56%

Total

118

746

6.3

3,772

20%

1,262

33%

NA = not applicable.

Source: Abt Associates analysis of Section 811 PRA Quarterly Reports for quarter ending September 2018.

Occupancy Status of Units under RAC

could be not available for PRA residents because they are

The RAC identifies the total number of units that an owner

being leased to other, non-PRA residents, or because the

agrees to commit to the PRA program, but not all units are

units are still under construction. HUD advises grantees

vacant and immediately available for a PRA resident when

to execute RACs with owners within 6 months of the

the RAC is executed. Units in existing properties that

expected lease dates.

are already operating as affordable housing are typical y

Exhibit A.6 shows the occupancy and availability status of

available for PRA residents upon resident turnover, which

the 2,799 units under RAC as of September 2018 (2,283

can vary considerably among properties. All PRA units

units for FY12 grantees and 516 for FY13 grantees). Of

under RAC in new construction properties typical y

units under RAC, close to half were not available in FY12

become available for lease when the building comes

(52 percent) and FY13 (43 percent). As of September

online.

2018, some 45 percent of FY12 units under RAC and 33

For units under RAC, we distinguish between occupied

percent of FY13 units under RAC were occupied by PRA

units, vacant units, and units that are not available. Units

residents.

E xhibit A .6 : Occup

Exhibit A.6: ancy Status of Units under RAC as of September 2018, by Gr

Occupancy Status of Units under RAC as of September 2018, by Grant Y an

eart Year

100%

90%

80%

70%

60%

52%

50%

45%

43%

40%

33%

30%

25%

20%

10%

3%

0%

Leased units

Units under RAC

Units not available

FY12 (n=2,283)

FY13 (n=516)

Source: Abt Associates analysis of Section 811 PRA Quarterly Reports for quarter ending September 2018.

106

H U D S e c t i o n 811 P R A P ro g r a m – P h a s e I I E v a l u a t i o n

Appendix A: Status of National PRA Program Implementation as of September 2018

About 3 percent of FY12 units and 16 percent of FY13

the number of units occupied by PRA residents as of

units were vacant as of September 2018. Units may be

September 2018.

vacant for several reasons. The PRA units under RAC may

not match applicant preferences for locations, specific

Implementation Status of FY12 PRA Grants

properties, unit size, or accessibility features. In addition,

All states that were awarded FY12 grants had units

units reported as vacant in one quarter might have PRA

occupied by PRA residents as of September 2018 (Exhibit

residents moving into them in the fol owing quarter.71

A.7). Overal , 34 percent of FY12 Cooperative Agreement

Owners also might be holding units vacant specifical y for

units were occupied, and states had between 12 and 91

the PRA program.72

percent of their Cooperative Agreement units occupied

by this time. On average, FY12 grantees had units under

A.3 Status of PRA Grants by

RAC for three-quarters of the units they had planned.

State and by Grant Year

Three states had 99 percent or more of their Cooperative

Agreement units under RAC, and three states had fewer

This section presents the implementation status of the 2

than half under RAC. Combining units under RAC and

PRA funding years by state, represented by the number

ARAC, most FY12 grantees had identified over 80

of units under executed RACs, the number of units under

percent of units they expected to fund with their PRA

ARACs and other agreements with owners, and

grants, and five had reached or exceeded their unit

goals.

E xhibit A .7: I

Exhibit A.7: mplementation Status of FY12 Grantees as of September 2018, by State

Implementation Status of FY12 Grantees as of September 2018, by State

1400

1200

1000

800

600

400

200

0

CA

DE

GA

IL

LA

MA

MD

MN

MT

PA

TX

WA

Occupied Units

Units under RAC but not Occupied

Units under ARAC or other Formal Agreement

PRA 5-Year Unit Goal

Source: Abt Associates analysis of Section 811 PRA Quarterly Reports for quarter ending September 2018.

Implementation Status of FY13 PRA Grants

Four FY13 state grantees did not report any units under

FY13 grantees had far fewer ARACs, RACs, and occupied

RAC, ARAC, or otherwise committed to the PRA program

units than FY12 grantees did (Exhibit A.8). Of the 23

through September 2018—including two states that

FY13 grantees, nearly two-thirds had leased PRA units

were funded in both years. Since FY12 grants must be

as of September 2018 (15 out of 23). Six FY13 grantees

disbursed by September 30, 2025, a year earlier than the

had leased more than 10 percent of their Cooperative

FY13 disbursement deadline, grantees funded in both

Agreement units.

years are likely to prioritize leasing of FY12 units.

71 Grantees report occupancy and vacancy rates quarterly.

72 The PRA program allows grantees to provide vacancy payments to owners for up to 60 days before or after a unit becomes vacant for a PRA applicant or resident.

107

H U D S e c t i o n 811 P R A P ro g r a m – P h a s e I I E v a l u a t i o n

Appendix A: Status of National PRA Program Implementation as of September 2018

E xhibit A .8 : I

Exhibit A.8: mplementation Status of F Y13 Grantees as of September 2018, b

Implementation Status of FY13 Grantees as of September 2018, by State y State

600

500

400

300

200

100

0

AL AZ CA CO CT GA IL MA MD ME MI MN NH NJ NM NV OH OR PA RI SD TX WI

Occupied Units

Units under RAC but not Occupied

Units under ARAC or other Formal Agreement

PRA 5-Year Unit Goal

Source: Abt Associates analysis of Section 811 PRA Quarterly Reports for quarter ending September 2018.

A.4 Characteristics of PRA

sizes of units under RAC, suggesting that the units

Units under RAC

for which grantees have obtained agreements align

somewhat with the demand for these units by applicants.

This section presents data on the characteristics of PRA

More than half of occupied units (55 percent) were

units, both for units under contract for the PRA program

one-bedroom units, about one fifth (18 percent) were

and for specific units under lease by PRA residents. We

efficiencies/SROs, and a quarter (24 percent) were two-

show the distribution of bedroom sizes, the accessibility

bedroom units. Very few (3 percent) were three-bedroom

of units, and amounts of contract rents and actual rental

units.

subsidy amounts by bedroom size.

All twenty states with occupied PRA units as of

Bedroom Size of Units

September 2018 had some combination of one- and two-

bedroom units under lease.

Grantees enter into agreements for a specified number of

units and bedroom sizes. This al ows the PRA program

Accessibility of Units under Contract and

to serve households of varying sizes, including single

under Lease

individuals and individuals living with other family

Grantees report the number of units under contract that

members, caregivers, or roommates. Stil , grantees

are accessible for people with mobility, vision, and hearing

expect the PRA program to mostly serve single-person

impairments to HUD in quarterly grant progress reports,

households.

and they report the accessibility of units occupied by PRA

As of September 2018, the majority of units under RAC

residents in TRACS. Exhibit A.10 shows the accessibility

and occupied by PRA residents were designed for smal

of all units under RAC and reported under lease through

households of one or two people, as shown in Exhibit A.9.

September 2018.

The majority of units under RAC were for one person or a

Property owners cannot typical y identify PRA units as

couple: 60 percent of units under RAC were one-bedroom

accessible or not until a particular unit is vacant and

units, and 12 percent were single-room occupancy units

available for lease to a PRA resident. PRA units float

(SROs) or efficiencies.73 Approximately a quarter of units

within a housing development, rather than specific units

(25 percent) under RAC were two-bedroom units, and 3

being designated as such. Therefore we do not know the

percent were three-bedroom units.

accessibility status of the majority of units under RAC

The distribution of bedroom sizes for units occupied by

(68 percent). We also don’t know the accessibility of 43

PRA residents is similar to the distribution of bedroom

percent of units occupied by PRA residents, and the

accuracy of the other data reported is unclear.

73 SROs are single room dwelling units that may share a bathroom or kitchen. Efficiencies are units with their own bathrooms and kitchens but no separate bedrooms.

108

H U D S e c t i o n 811 P R A P ro g r a m – P h a s e I I E v a l u a t i o n

Appendix A: Status of National PRA Program Implementation as of September 2018

E xhibit A .9 : Section 811 P R A Unit s under R AC and under Lease as of September 2018, by Bedroom Size

# of PRA Units under RAC a

% of PRA Units under RAC

# of PRA Units Leased

% of PRA Units Leased

SROs/efficiencies

262

12%

116

18%

One-bedroom units

1,330

60%

358

55%

Two-bedroom units

567

25%

156

24%

Three-bedroom units

74

3%

20

3%

Total

2,233

100%

650

100%

a Grantees reported 2,234 units under RAC, with bedroom sizes missing for 1 unit under RAC.

Sources: Abt Associates analysis of Section 811 PRA Quarterly Reports for quarter ending September 2018; Abt Associates analysis of TRACS Household data for period ending September 30, 2018.

With those caveats, grantees reported that 18 percent of

reported that only 44 units, or 7 percent of leased units,

units under contract are accessible (378 units): 15 percent

were accessible. Of these, all 44 were accessible for people

of units were accessible to people with mobility impairments

with mobility impairments or multiple impairments. Grantees

(326 units), 2 percent accessible to people with hearing

reported an additional 50 percent of occupied units were not

impairments (36 units), and 1 percent of units accessible to

accessible.

people with visual impairments (16 units). Grantees reported

The uncertainty of the data makes it unclear whether the

that 15 percent of units under contract were not accessible.

supply of accessible units matches the demand. Grantees

Of the 650 units occupied by PRA residents with available

do not report to HUD how many applicants need accessible

HUD administrative data as of September 2018, grantees

units or are on the waiting list.

E xhibit A .10 : Section 811 P R A Unit s under R AC as of September 2018, by Accessibilit y

Type of Accessibility

# of PRA Units under RAC

% of PRA Units under RAC

# of PRA Units Leased

% of PRA Units Leased

Hearing Impaired

36

2%

0

0%

Visually Impaired

16

1%

0

0%

Mobility Impaired / Multi-access

326

15%

44

7%

Not Accessible

327

15%

327

50%

Unknown

1,529

68%

279

43%

Total

2,234

650

Sources: Abt Associates analysis of Section 811 PRA Quarterly Reports for quarter ending September 2018; Abt Associates analysis of TRACS Household data for period ending September 30, 2018.

Rent and Rental Subsidy Levels

Grantees determine the maximum amount of rent that

One goal of the PRA program is to provide affordable

property owners can charge PRA residents within certain

housing more cost-effectively than other affordable

parameters set by HUD. Rents cannot exceed the area’s

housing programs, while continuing to serve households

applicable Fair Market Rent (FMR) or Small Area FMR, which

with extremely low incomes. The grant NOFA encouraged

is used as a standard for the HCV and other affordable

grantees to maximize its subsidies by targeting units that

housing programs and is determined annual y by HUD.

would produce the lowest possible per-unit costs. Eligible

Rents must also be affordable to residents earning up to 50

properties are those with capital costs financed by other

percent of the Area Median Income (AMI). HUD established

affordable housing programs such as LIHTC, HOME, or the

incentives in the FY12 and FY13 NOFAs to encourage

National Housing Trust Fund.

applicant states to propose lower per-unit subsidy costs

than would be required if the PRA rents were based on FMR.

Several grantees responded by proposing maximum PRA

109

H U D S e c t i o n 811 P R A P ro g r a m – P h a s e I I E v a l u a t i o n

Appendix A: Status of National PRA Program Implementation as of September 2018

rents in their grant applications that were lower than FMR,

set rent levels based FMR, based on being affordable

and were set to be affordable to households earning

to households earning a certain percentage of AMI, or

between 20 and 50 percent of AMI.

based on a combination of approaches that might differ

HUD expects that matching PRA funds with other

by location.

affordable housing programs will result in PRA subsidy

Of FY12 grantees, six established rent levels at FMR, five

costs that will be significantly less than if the units were

to be affordable to households earning up to 50 percent

offered at FMR. The lower a grantee’s per-unit subsidy

AMI, and one (Georgia) to be affordable to households

costs, the more units the grantee can make available to

earning up to 60 percent of AMI. Of FY13 grantees, 10

PRA households. Rental subsidies provided to a unit are

established rent levels at FMR, and the remaining to be

based on the difference between the unit rent and what

affordable to households earning a percentage of AMI

tenants pay toward rent based on their income. Thus, the

or a combination of FMR and AMI. Of them, three states

final number of units that a PRA grant can support cannot

(Connecticut, New Jersey, and Wisconsin) set rents to be

be determined until all the PRA units are leased.

affordable to households earning up to 30 percent of AMI,

Exhibit A.11 shows the grantees’ commitments to

and one (Georgia) to be affordable to households earning

establish rents. The exhibit shows whether the grantees

up to 60 percent of AMI.

E xhibit A .11: Grantee C ommitment for Unit s Rent Levels as of September 2018, by Grant Year and S t ate FY12

FY13

Grantee Commitment for

Grantee Commitment for

State

Unit Rent Levels

State

Unit Rent Levels

California

FMR

Alaska

47% AMI

Delaware

At or below 50% AMI

Arizona

FMR (20% AMI)

Georgia

60% AMI

California

FMR

Illinois

FMR

Colorado

FMR

Louisiana

FMR

Connecticut

30% AMI

Massachusetts

50% AMI

Georgia

60% AMI

Maryland

50% AMI

Illinois

FMR

Minnesota

50% AMI

Maine

50% AMI

Montana

FMR

Maryland

50% AMI

Pennsylvania

50% AMI

Massachusetts

50% AMI

Texas

FMR

Michigan

50% AMI

Washington

FMR

Minnesota

50% AMI

New Hampshire

FMR

New Jersey

30% AMI

Nevada

FMR

Ohio

30% AMI (50% AMI)

Oregon

FMR

Pennsylvania

50% AMI

South Dakota

FMR

Texas

FMR

Washington

FMR

Wisconsin

30% AMI

Source: Abt Associates analysis of Section 811 PRA Quarterly Reports for quarter ending September 2018.

110

H U D S e c t i o n 811 P R A P ro g r a m – P h a s e I I E v a l u a t i o n

Appendix A: Status of National PRA Program Implementation as of September 2018

Contract Rent for Units under RAC

the program. The average monthly contract rent for al

The contract rent is the maximum amount of rent that

occupied PRA units through September 2018 was $741,

owners can charge PRA residents. The executed RAC

ranging from $331 to $1,761. The median contract rent

between the grantee and owner identifies PRA contract

was $704. As expected, contract rents increased with

rents by unit size. Exhibit A.12 shows the contract rent by

bedroom size. The average contract rent was $694 for

bedroom size for all units leased since the PRA program

a studio, $693 for a one-bedroom unit, $856 for a two-

began, including for households that have since exited

bedroom unit, and $981 for a three-bedroom unit.

E xhibit A .12 : C ontract Rent and Rent al A ssist ance Payment for Unit s

under Lease as of September 2018, by Bedroom Size

Contract Rent (Monthly)

Studio / 0 Bedroom

1 Bedroom

2 Bedrooms

3 Bedrooms

All Units

N

116

358

156

20

650

Minimum contract rent

$476

$360

$331

$590

$331

Average contract rent

$694

$693

$856

$981

$741

Median contract rent

$660

$675

$816

$990

$704

Maximum contract rent

$1,009

$1,301

$1,761

$1,274

$1,761

Rental Assistance Payment (Monthly)

Studio / 0 Bedroom

1 Bedroom

2 Bedrooms

3 Bedrooms

All Units

N

112

320

125

14

571

Minimum rental assistance payment

$236

$123

$35

$261

$35

Average rental assistance payment

$494

$517

$676

$738

$553

Median rental assistance payment

$471

$481

$662

$803

$493

Maximum rental assistance payment

$900

$1,114

$1,653

$1,033

$1,653

Note: Rental assistance payments were only included for 571 of 650 units in the TRACS data because 79 rental assistance payment amounts appeared to be errors and were removed from the data or were not reported.

Source: Abt Associates analysis of TRACS Household data for period ending September 30, 2018.

Rental Assistance Payments of Units

Cooperative Agreements and the actual average monthly

under Lease

rental assistance paid to property owners through

Rental assistance payments are the amount of subsidy

September 2018. Among states where rental assistance

that HUD pays grantees to pay property owners for units

payment data is available for residents, the estimated

under lease for the Section 811 PRA program. Exhibit A.13

average monthly rental assistance amount assumed in

also shows the minimum, maximum, median, and average

grantees’ Cooperative Agreement budgets was $508,

rental assistance payment by bedroom size for units under

ranging from $226 in Il inois to $1,055 in Maryland. On

lease. For all bedroom sizes, the average rental assistance

average, the actual monthly cost for assistance payments

payment was $553 per month, ranging from $35 to

was $554, ranging from $262 in New Jersey to $1,214 in

$1,653. The median rental assistance payment was $493 a

Connecticut. The actual average amounts shown in the

month. Average monthly assistance payments were $494

exhibit are only for less than 10 percent of total planned

for studio apartments, $517 for one-bedroom apartments,

PRA units, and may not be representative of future trends.

$676 for two-bedroom apartments, and $738 for three-

For all reported rental assistance payments, the average

bedroom apartments.

monthly subsidy amount was $46 higher on average

than assumed in the Cooperative Agreements ($554 for

Difference between Estimated and Actual Rental

Assistance Payments

states with estimated average rental payments available

compared to an estimated average of $508), but variance

Exhibit A.13 shows the difference between the estimated

was larger than the average difference across states.

average monthly rental assistance payment in their

111

H U D S e c t i o n 811 P R A P ro g r a m – P h a s e I I E v a l u a t i o n

Appendix A: Status of National PRA Program Implementation as of September 2018

The actual monthly costs for assistance payments were

monthly subsidies were substantial y higher than

closest to estimated costs in Delaware ($534 actual costs

estimates were Montana ($192), Texas ($175), and

compared to an estimated monthly payment of $502).

Washington ($137). Four states, California, Georgia,

In six states, including Delaware, Minnesota, Montana,

Louisiana, and Maryland, overestimated the rental subsidy

Pennsylvania, Texas, and Washington, actual rental

amounts; that is, actual average subsidy amounts have

subsidy costs were higher than grantees had estimated

been lower than expected. The differences range from $73

in their Cooperative Agreements. States where actual

to $195 per month.

E xhibit A .13 : E stimated and Actual Rent al A ssist ance A mount s through September 2018, by S t ate Estimated Average

Difference between

# Units with Reported

Actual Average Monthly

State

Monthly PRA Rental

Estimated and Actual

Payments (N=598)

PRA Rental Assistance

Assistance

Monthly Payments

Arizona

5

a

b

a

California

71

$705

$534

($171)

Colorado

14

a

$871

a

Connecticut

14

a

$1,214

a

Delaware

37

$502

$534

$32

Georgia

13

$441

$368

($73)

Illinois

4

$226

b

N/A

Louisiana

66

$597

$522

($75)

Massachusetts

8

a

b

a

Maryland

21

$1,055

$860

($195)

Minnesota

82

$503

$587

$84

Montana

16

$351

$543

$192

New Hampshire

4

a

b

a

New Jersey

41

a

$262

a

Ohio

2

a

b

a

Pennsylvania

63

$378

$452

$74

South Dakota

12

a

$363

a

Texas

44

$478

$653

$175

Washington

51

$354

$491

$137

Average

30

$508

$554

$46

a = Not available.

b Actual average monthly PRA rental assistance not shown due to small sample reporting restrictions.

Notes: Contract rent and rental assistance payment only available in TRACS for 457 of 474 PRA residents. Some rental assistance payment amounts appeared to be errors and were removed from the data. Units under lease in states with FY12 and FY13 grants are compared to FY12 grant estimates because the majority of units are reported as units under FY12 grantees.

Source: Abt Associates analysis of TRACS Household data for period ending September 30, 2018.

112

H U D S e c t i o n 811 P R A P ro g r a m – P h a s e I I E v a l u a t i o n

Appendix A: Status of National PRA Program Implementation as of September 2018

A.5 Status of Applicant Referrals

Target Populations of PRA Grants

and Lease-Ups

As part of their PRA grant applications, grantees specified

specific vulnerable populations they planned to target

PRA funds may only be provided for households

as part of their PRA program. They often defined those

with extremely low household incomes and with at

populations by living situation, such as living in an

least one person with disabilities who is at least 18

institution; experiencing homelessness; residing in a group

and no older than 61 years old at the time of initial

home, adult care housing, or other residential group home;

occupancy of PRA units. The eligible person must be

or transitioning from foster care. Grantees also targeted

eligible for Medicaid-funded community-based long-

individuals at risk of institutionalization or homelessness

term services and supports or similar services from

without access to affordable, community-based housing.

state programs. PRA applicants must meet these PRA

Several grantees also chose to target people with specific

program eligibility requirements in addition to meeting

types of disabilities, such as serious mental il ness,

the owner’s requirements for the specific property where

developmental disabilities, and physical disabilities.

PRA applicants want to live. In this section, we present

Exhibit A.14 shows the intended target populations for the

characteristics of applicants and referrals to the PRA

FY12 grantees, and Exhibit A.15 shows the intended target

program, applicants on the waiting list for available PRA

populations for FY13 grantees.

units, and applicants who did not meet PRA program or

owner eligibility criteria for the FY12 and FY13 grantees.

E xhibit A .14 : Section 811 P R A Grant s Target Populations ( F Y12 Grantees )

Leaving Group Home,

At Risk of

Experiencing Homelessness

State

Institutionalized

Adult Care Home, or

Institutionalization

or at Risk of Homelessness

Residential Home

California

√

√

Delaware

√

√

√

Georgia

√

√

Illinois

√

√

Louisiana

√

√

√

Maryland

√

√

√

√

Massachusetts

√

Minnesota

√

√

Montana

√

√

√

√

Pennsylvania

√

√

√

√

Texas

√

√

√

Washington

√

√

√

√ means that the state’s PRA program targets this population.

Source: “Section 811 Project Rental Assistance: Bringing Supportive Rental Housing to Scale. Status Report to Congress,” US Department of Housing and Urban Development, Office of Multifamily Housing, January 2014.

113

H U D S e c t i o n 811 P R A P ro g r a m – P h a s e I I E v a l u a t i o n

Appendix A: Status of National PRA Program Implementation as of September 2018

E xhibit A .15 : Section 811 P R A Grant s Target Populations ( F Y13 Grantees )

Living Situation

Disability

n

e,

e

the

s

al

io

am

om

e, or

s

ic

f

s

from

zed

f

om

ys

are

H

om

ing

es

es

rogr

eriou

ith

ng

ness

ental

C

sn

isk o

sn

ollows

ties

ties

isk o

nalizat

roup

tial H

Ill

ter

re H

t R

y F

ith S

le w

eles

eles

lopm

ith Ph

sitioni

isabili

isabili

Fos

At R

iden

or a

one

le w

Experienc

ental

D

le w

D

Hom

Hom

Peop

M

Person P

M

eveD

Tran

Institutionali

Res

State

Institutio

Leaving G

Adult Ca

Peop

Peop

Alaska

√

√

√

Arizona

√

California

√

√

√

√

√

√

√

Colorado

√

√

√

√

Connecticut

√

√

√

√

√

√

√

Georgia

√

√

√

√

√

Illinois

√

√

√

√

√

√

√

Massachusetts

√

Maryland

√

√

√

√

√

Maine

√

√

√

√

√

√

Michigan

√

√

√

√

√

Minnesota

√

√

√

√

New Hampshire

√

√

New Jersey

√

√

√

√

New Mexico

√

√

√

√

Nevada

√

Ohio

√

√

√

√

Oregon

√

√

√

√

√

Pennsylvania

√

√

√

Rhode Island

√

√

√

South Dakota

√

Texas

√

√

√

√

√

Wisconsin

√

√

√ means that the state’s PRA program targets this population.

Source: Analysis by the Technical Assistance Collaborative, 2017.

All twelve FY12 grantees targeted people institutionalized,

developmental disabilities, and six states targeted people

and all but three FY12 grantees targeted people at

with physical disabilities. Three grantees did not choose

risk of institutionalization. Nine states targeted people

any target populations based on living situation, selecting

experiencing or at risk of homelessness.

populations based only on a single disability type.

Of the 23 FY13 grantees, 18 states targeted people

PRA Applicant Referral Sources

living in institutions or at risk of institutionalization, and

11 states targeted people experiencing homelessness

Between 2015 and September 2018, grantees reported

or at risk of homelessness. Five states targeted people

12,506 applicants to the PRA program. Applicants

leaving group homes, adult care homes, or other

are households that were referred to or completed

residential settings. Almost two-thirds of FY13 grantees

applications for the PRA program during the quarter. We

also targeted people with serious mental il ness or with

note that some applicants for programs in states that

114

H U D S e c t i o n 811 P R A P ro g r a m – P h a s e I I E v a l u a t i o n

Appendix A: Status of National PRA Program Implementation as of September 2018

received both FY12 and FY13 grants might be counted

A.16. Service providers referred 50 percent of applicants.

under both grant years. Because of this potential for

The most common type of service provider referrals

double-counting, we do not report combined totals for

was from a mental health service provider (28 percent),

FY12 and FY13 applicants in this section.

fol owed by independent living facilities (7 percent) and

Almost all applicants were referred by either a service

service providers for intellectual/developmental disabilities

provider or a government agency, as shown in Exhibit

(7 percent).

E xhibit A .16 : Referral S ource of A pplicant s through September 2018, by Grant Year

Referral Source

N

12,506

Service Provider

Service Provider–Mental health

28%

Service Provider–Intellectual / developmental disabilities

7%

Service Provider–Centers for Independent Living

7%

Service Provider–Other

8%

Total

50%

State/Local Human Service Agency or Authority

State/local mental health agency or authority

22%

State/local intellectual / development disability agency or authority

2%

State/local aging / adult services agency or authority

6%

State/local child/family agency or authority

2%

Other state/local human service agency or authority

7%

Total

38%

Other

12%

Source: Abt Associates analysis of Section 811 PRA Quarterly Reports for quarters from December 2014 through September 2018.

State and local human service agencies or authorities

Applicants by Living Situation

referred an additional 38 percent of applicants. Of such

Grantees reported that more than one-fourth of applicants

referrals, mental health agencies were the most common

were homeless (26 percent), and about a fifth were

referral source, referring 22 percent of applicants. Area

institutionalized (21 percent) when they applied to the

aging and other adult services agencies referred 6

PRA program. Some 15 percent of applicants were

percent of applicants, development disability or other

reported to be at risk for being institutionalized and 12

adult services agencies or authorities and state or local

percent were reported to be at risk becoming homeless

child and family agencies or authorities each respectively

(Exhibit A.17). An additional 17 percent of applicants were

referred 2 percent of applicants. Seven percent were

reported as having “other” living situations. About one in

referred by other state or local human service agencies.

ten applicants were living in a group home, an adult care

Grantees selected “Other” as the referral source for 12

home, or other residential settings.

percent of applicants.

115

H U D S e c t i o n 811 P R A P ro g r a m – P h a s e I I E v a l u a t i o n

Appendix A: Status of National PRA Program Implementation as of September 2018

E xhibit A .17: Living Situation of A pplicant s through September 2018, by Grant Year

Living Situation

FY12

N

12,432

Institutionalized

21%

At Risk of Institutionalization

15%

Homeless

26%

At Risk of Homelessness

12%

Living in a Group Home, Adult Care Home, or Other Residential Setting

8%

Other

17%

Note: Grantees reported 12,506 applicants to the PRA program since the program started in 2015, with living situation missing for 68 applicants.

Source: Abt Associates analysis of Section 811 PRA Quarterly Reports for quarters from December 2014 through September 2018.

Living Situation of Applicants on Waiting List

Exhibit A.18 below shows the living situation at the time

Many grantees developed waiting lists for their PRA

of application for applicants on the waiting lists by grant

program, but they organize and use them in different

year. Over one-third of FY12 applicants on the waiting list

ways. Grantees and their partners may maintain waiting

were experiencing homelessness (36 percent), and 19

lists at the state, region, contracted service providers, or

percent were institutionalized. An additional 8 percent of

property level. HUD asks grantees to report the number

applicants resided in group homes, adult care homes, or

of applicants on their waiting lists for PRA units that have

other residential settings.

been determined eligible for the PRA program. As of

For FY13, one-third of applicants on the waitlist were

September 30, 2018, grantees reported 5,991 applicants

experiencing homelessness (33 percent), 15 percent were

on their waiting lists for units available through FY12

living in a group home, adult care home or other residential

grants and 3,302 through the FY13 grants.74

setting, and 12 percent were living in institutions. One-

fourth of applicants on the waitlist (25 percent) had other

or unknown living situations.

E xhibit A .18 : Living Situation of A pplicant s on the Waitlist through September 2018, by Grant Year Living Situation

FY12

FY13

N

5,991

3,302

Institutionalized

19%

12%

At Risk of Institutionalization

14%

14%

Homeless

36%

33%

At Risk of Homelessness

5%

1%

Living in a Group Home, Adult Care Home, or Other Residential Setting

8%

15%

Other/Unknown

17%

25%

Source: Abt Associates analysis of Section 811 PRA Quarterly Reports for quarter ending September 2018.

74 Maryland has a combined waiting list that includes applicants for other housing programs. The state’s total of 2,277 applicants represents a broader population than those on the waiting list for PRA assistance. Because of the bias this adds to the data, we removed Maryland from the sample presented above. In addition, three states, Georgia, Minnesota, and Pennsylvania, listed the same number of applicants for both FY12 and FY13; we suspect those grantees might have reported the same applicants in both grant years, resulting in double counting.

116

H U D S e c t i o n 811 P R A P ro g r a m – P h a s e I I E v a l u a t i o n

Appendix A: Status of National PRA Program Implementation as of September 2018

Applicants Found Ineligible for the PRA

PRA program, by grant year. Of households determined

Program and Properties

to be ineligible for FY12 units, 35 percent were at risk

To move into PRA-subsidized units, applicants must be

for homelessness, and 15 percent were experiencing

determined eligible for the PRA program by age, income,

homelessness. Only 7 percent of ineligible applicants

and disability, and for home and community-based

were living in an institution, and 10 percent were at risk for

services. Since the start of the PRA program, 2,007

institutionalization. An additional 3 percent of applicants

households have been determined ineligible based on

were living in a group home, adult care home, or other

program eligibility requirements—1,777 households by

residential setting. More than a quarter (30 percent) had

FY12 grantees and 300 households by FY13 grantees.

“other” living situations.

Typical y, the grantee state housing agency or partnering

Of FY13 applicants that were found ineligible for the

health or other agencies determine eligibility before

program, 21 percent were institutionalized, and 20

applicants apply to specific properties with PRA units.

percent were experiencing homelessness. About one-

Some PRA programs, however, do not determine program

tenth are living in a group home, an adult care home,

eligibility until applicants apply to specific properties.

or other residential settings (10 percent), and similar to

Exhibit A.19 shows the living situation at the time of

FY12 grantees, 30 percent of applicants had other living

application of households reported ineligible for the

situations (30 percent).

E xhibit A .19 : Living Situation of Ineligible A pplicant s through September 2018, by Grant Year Living Situation

FY12

FY13

N

1,777

300

Institutionalized

7%

21%

At risk of institutionalization

10%

16%

Homeless

15%

20%

At risk of homelessness

35%

<4%a

Living in a group home, adult care home, or other residential setting

3%

10%

Transitioning from foster care

0%

0%

Other

30%

30%

a Exact percentages not shown due to small sample reporting restrictions.

Source: Abt Associates analysis of Section 811 PRA Quarterly Reports for quarters from December 2014 through September 2018.

Since 2015, 4,289 households have been referred to

credit histories, 13 percent for poor rental histories, and

specific properties with PRA units to complete lease

5 percent because they were unable to submit required

applications. To move into PRA-subsidized units,

documents. Grantees reported that 26 percent of

applicants also must meet the application requirements

households were denied for “other” reasons. Reasons

set forth by the property owner, such as income, rental

the grantees selected the “other” category included

history, credit history, and criminal background. Since

applicants withdrawing their application after submitting it

2015, owners determined that 747 households that applied

or being over the age limit at the time of application.

to live at their properties did not meet their requirements,

representing 17 percent of the households referred to PRA

A.6 Characteristics of Households

units.

That Moved into PRA Units

Owners screened out 8 percent of applicant households

referred to PRA units in 2015, 19 percent in 2016, 12

This section provides selected data on the prior living

percent in 2017, and 19 percent of households in 2018.

situation and duration of tenancy of households that

Of households that owners screened out, nearly half

moved into PRA units as of September 2018.

were determined ineligible because of criminal history

(46 percent), 16 percent were rejected for having poor

117

H U D S e c t i o n 811 P R A P ro g r a m – P h a s e I I E v a l u a t i o n

Appendix A: Status of National PRA Program Implementation as of September 2018

Previous Living Situation of Households that

group homes, adult care homes, or other residential

Moved into PRA Housing

settings, and six percent moved from “other” housing.

Grantees report that 1,229 households moved into PRA

Of households that moved into FY13 grantee units, 24

units since the beginning of the PRA program in 2015

percent had been previously institutionalized, 32 were at

through September 2018 (Exhibit A.20). Of households

risk for institutionalization, 21 percent had been previously

that moved into FY12 grantee units, nearly one third

experiencing homelessness, and 13 percent were living

(32 percent) had been institutionalized, and 23 percent

in a group home, adult care home, or other residential

were experiencing homelessness directly prior to being

setting.

assisted by PRA. Six percent of residents moved from

E xhibit A .20 : Previous Living Situation of P R A Resident s through September 2018, by Grant Year Living Situation

FY12

FY13

N

1,021

208

Institutionalized

32%

24%

At risk of institutionalization

16%

32%

Homeless

23%

21%

At risk of homelessness

16%

8%

Living in a group home, adult care home, or other residential setting

6%

13%

Transitioning from foster care

<5%a

0%

Other

6%

<5%a

a Exact percentages not shown due to small sample reporting restrictions.

Source: Abt Associates analysis of Section 811 PRA Quarterly Reports for quarters from December 2015 through September 2018.

Length of Tenancy

PRA housing within the last 6 months, 25 percent had

Most current residents in PRA-assisted units had fairly

moved in between 7 and 12 months ago, and 26 percent

short durations of tenancy as of September 2018. As

had moved in between 1 and 2 years ago. Only 15 percent

shown in Exhibit A.21, of the 1,851 PRA households

of residents living in FY12 units had lived in their units for

reported living in PRA units, 41 percent had moved into

more than 2 years.

E xhibit A .21: Housing Tenure of Current P R A Resident s through September 2018

Tenure

N

%

6 Months or fewer

755

41%

7-12 months

470

25%

13-24 months

473

26%

25-48 months

153

8%

Total

1,851

100%

Source: Abt Associates analysis of Section 811 PRA Quarterly Reports for quarter ending September 2018.

118

H U D S e c t i o n 811 P R A P ro g r a m – P h a s e I I E v a l u a t i o n

Appendix A: Status of National PRA Program Implementation as of September 2018

A.7 Exits from PRA Units

household exits (32 percent) were initiated by the owner; 8

percent were for non-payment of rent and 24 percent were

Cumulatively, 216 PRA households have exited the PRA

for other reasons. Some 18 percent of residents died, 8

program since the program began—three in 2015, 19

percent moved back into institutional care, and 9 percent

in 2016, 74 in 2017, and 120 in 2018 (Exhibit A.22). Just

left for other or unknown reasons or moved out without

under one-third of households that left of their own accord

giving notice.

(34 percent), 25 percent exited for other housing, and

9 percent left for other reasons. Another one-third of

E xhibit A .2 2 : Reasons Tenant s Lef t P R A Housing through September 2018

Reason

N

%

Tenant initiated–left for other housing

53

25%

Tenant Initiated–other

20

9%

Owner initiative–non-payment of rent

18

8%

Owner Initiated–other

51

24%

Death

38

18%

Institutionalized

17

8%

Unknown/Disappeared or Other

19

9%

Total

216

Source: Abt Associates analysis of Section 811 PRA Quarterly Reports for quarters from December 2014 through September 2018.

A.8 Housing Choice

disability. It is not clear whether PHAs set aside the unit

Vouchers Leveraged

once for a PRA target population household, or the units

would also be available to the target population upon

HUD awarded extra points in the Section 811 PRA NOFA

turnover.

competitions for applicants that obtained commitments

Exhibit A.23 shows the HCVs or other rental units that

from one or more public housing authorities to leverage

the states committed to leveraging and the number of

HCV or other affordable housing units for people with

housing units the grantees reported leased by PHAs

disabilities. The 2012 NOFA awarded points to applicants

as of September 2018, by grant year. Overal , grantees

for setting aside a number of HCVs or other rental units

from both funding years reported that PHAs issued 89

specifical y for the PRA program’s target population.

percent of their total committed units through September

The 2013/2014 NOFA awarded points to applicants

2018. Most FY12 grantees (9 of 12) committed to

with commitments from one or more PHAs to establish

leveraging HCVs or other housing units on their PRA

an admissions preference for the Section 811 target

grant applications. Through September 2018, eight

population. Either grant year commitment could not

FY12 grantees reported issuing 1,521 HCVs or units

include vouchers already set aside as part of the PHA’s

out of 1,437 committed units. Four FY12 state grantees

allocation of NED vouchers.

reported leasing as many or more units than they original y

In their quarterly progress reports to HUD, grantees

committed to leverage.

self-report the number of HCV and other units that PHAs

Of the 23 FY13 grantees, 11 obtained commitments from

leased each quarter. Such data on affordable units

PHAs to issue HCV or other housing units, or to develop

leveraged for the PRA program should be interpreted with

admissions preferences for the state PRA program’s target

caution, as grantees report the data inconsistently. From

population, for their PRA grant applications. Through

administrative interviews conducted in the study states,

September 2018, seven of the states reported leasing a

we learned that grantees use various metrics to count

total of 751 out of 1,060 committed units (71 percent of

units leased to the state’s PRA program target population.

their total commitment). All but one grantee reporting any

Some grantees count every HCV or unit issued by their

leveraged vouchers also reported meeting 100 percent of

agency to a household that includes a person with a

their leveraged unit commitment.

119

H U D S e c t i o n 811 P R A P ro g r a m – P h a s e I I E v a l u a t i o n

Appendix A: Status of National PRA Program Implementation as of September 2018

E xhibit A .23 : HC Vs Leveraged for P R A Proper ties, by Grant Year and S t ate

HCVs Committed to

HCVs Issued

Percentage of Committed

State

Leveraging

through September 2018

HCVs Issued

FY12

Delaware

74

83

112%

Georgia

175

80

46%

Illinois

695

654

94%

Louisiana

125

<10a

<8%

Maryland

97

283

292%

Massachusetts

50

37

74%

Minnesota

60

60

100%

Pennsylvania

152

316

208%

Washington

9

0

0%

Total

1,437

1,521

106%

FY13

Alaska

100

<10a

<10%

Arizona

27

27

100%

California

150

0

0%

Colorado

206

451

219%

Connecticut

75

75

100%

Maine

33

33

100%

Michigan

174

0

0%

New Jersey

103

103

100%

Ohio

95

0

0%

Oregon

40

0

0%

Wisconsin

57

57

100%

Total

1,060

751

71%

Combined Grant Years

Total

2,547

2,256

89%

a Exact value suppressed due to small sample reporting restrictions.

Source: Abt Associates analysis of Section 811 PRA Quarterly Reports for quarters from December 2014 through September 2018.

120

H U D S e c t i o n 811 P R A P ro g r a m – P h a s e I I E v a l u a t i o n

Appendix B: Section 811 PRA Phase II Evaluation Methods

Appendix B: Section

• In-person surveys administered to PRA and PRAC

residents

811 PRA Phase II

The Phase II evaluation focuses on PRA outcomes, costs,

and implementation in six states: California, Delaware,

Louisiana, Maryland, Minnesota, and Washington. These

Evaluation Methods

states were selected because they had made the most

progress leasing PRA units when the study’s evaluation

research design was completed in March 2017.

A Multi-Phase Evaluation

B.1 Overview of the Section 811

The Melvil e Act that authorized the PRA program also

PRA Phase II Evaluation

required an evaluation of the program’s effectiveness.

HUD undertook a multi-phase, independent evaluation of

The Phase II Evaluation of the Section 811 Project Rental

the PRA program:

Assistance (PRA) Program examines whether there is

• Phase I: The first phase of the evaluation, completed

evidence that the PRA approach achieves better short-

in July 2017, consisted of a process evaluation and

term housing, neighborhood, and services outcomes for

case studies to assess the early implementation of

its residents compared to similar people in the Project

the PRA program in 12 states funded in the first round

Rental Assistance Contract (PRAC) program, to similar

of PRA grants (awarded through HUD’s FY12 Notice

people in other HUD housing assistance programs, or

of Funding Availability (NOFA)).75 That first phase was

to similar people enrol ed in Medicaid but not served by

conducted by BCT Partners, with Abt Associates as a

a federal housing assistance program. The evaluation

subcontractor, between 2015 and 2017.

also assesses the PRA program’s progress on its goals

to create effective state-level partnerships, produce

• Phase II: In September 2016, HUD awarded Abt

affordable rental housing units for the PRA target

Associates a contract for the second phase of the

populations, and successful y transition eligible people

evaluation of the PRA program. This phase builds

to those units while providing coordinated access to

on the earlier work, focusing on a subset of 6 states

the services and supports they want and need to live in

selected from the 27 state housing agency grantees

community-based settings.

in the first and second rounds of PRA funding (in

response to HUD’s FY12 and FY13 NOFAs).

The primary construct of the evaluation is an impact

study that compares outcomes for PRA to outcomes for

Phase II Research Questions

individuals in four comparison groups of individuals similar

The overarching research questions for the Phase II

to those assisted by PRA.

evaluation were:

The evaluation’s research approach integrates multiple

• What is the early evidence on how PRA residents fare

quantitative and qualitative data sources:

relative to similar individuals in terms of quality of life,

• Housing, healthcare utilization, and neighborhood

housing and neighborhood characteristics, housing

administrative data from the U.S. Department of

tenure, use of tenancy supports, and healthcare

Housing and Urban Development (HUD), Centers for

utilization patterns?

Medicare & Medicaid Services (CMS), state Medicaid

• How do PRA residents compare to non-elderly people

agencies, and publicly available federal datasets

with disabilities served in other HUD programs,

• PRA program administrative data and program

and to similar people not served in HUD programs,

documents

in terms of demographics, health diagnoses and

• Administrative interviews with HUD staff that

chronic conditions, and historical healthcare utilization

administer the PRA program

patterns?

• Administrative interviews with staff from state housing

• What is the early evidence of the cost-effectiveness of

agency grantees, state Medicaid agencies, and other

the PRA program relative to other HUD programs that

PRA program partners

assist people with disabilities?

75 Final reports for Phase I of the PRA program evaluation can be found at https://www.huduser.gov/portal/publications/section-811-processeval-

report.html

121

H U D S e c t i o n 811 P R A P ro g r a m – P h a s e I I E v a l u a t i o n

Appendix B: Section 811 PRA Phase II Evaluation Methods

• Do state housing agencies and their health agency

Economic Study

partners develop effective, sustainable partnerships

The economic study measured the costs of housing and

that result in a growing inventory of affordable rental

healthcare and supportive services for PRA residents,

units with access to voluntary supportive services for

and estimated the costs of the PRA approach relative to

people with disabilities?

the costs of PRAC, NED, and other HUD programs. Our

Phase II Coordinated Sub-studies

study calculated total costs per participant and separated

costs into categories of program implementation and

The study team addressed these broad questions through

administration, housing, support services, and healthcare

three coordinated sub-studies: the impact study, the

utilization. To the extent possible, we made additional

economic study, and the implementation study.

comparisons of housing and healthcare costs for PRA and

Impact Study

PRAC programs versus estimates of costs of alternative

housing assistance programs, and for healthcare costs to

An impact study of the PRA program compared the quality

the non-HUD group.

of life, housing, and healthcare utilization patterns of PRA

residents and four comparison groups:

Implementation Study

• Group 1: PRAC. Section 811 PRAC residents. Like

The implementation study of the PRA program

PRA, Section 811 provides structured access to

documented the strength and sustainability of PRA

services. Unlike PRA, PRAC residents live in housing

partnerships and how the partnerships influenced

set aside primarily for people with disabilities.

program implementation and contributed to program

successes and challenges. Through interviews with

• Group 2: NED. People assisted by Non-elderly

PRA program staff and partners and surveys of PRA

Disabled (NED) vouchers. The NED program does not

residents, we documented grantees’ and their partners’

typical y provide structured access to services.76

housing strategies for allocating the PRA subsidies to

• Group 3: Other HUD. Non-elderly people with

property owners, for determining PRA unit rent levels and

disabilities participating in several of HUD’s other

locations, for identifying PRA applicants and coordinating

assisted housing programs (public housing, Housing

resident placement into PRA units, and for ensuring that

Choice Voucher, and multifamily housing) that provide

appropriate services are in place for PRA residents as they

rental subsidies to non-elderly people with disabilities

transition to PRA units and through their ongoing tenancy.

but that, unlike PRA and PRAC, do not provide

Qualitative information from the implementation study

structured access to services.

also provided context to the findings of the impact and

• Group 4: Non-HUD. Non-elderly people with similar

economic studies.

characteristics and service utilization patterns to PRA

and PRAC residents who are enrol ed in Medicaid but

B.2 Evaluation Data Sources

who are not served by any of the other HUD programs

The study relied on numerous quantitative and qualitative

in Group 3 and may be in any other housing situation

data sources: HUD administrative data, healthcare data

(for example, living with family, homeless, in market-

from CMS and state Medicaid agencies, publicly available

rate housing, or in institutional settings).

neighborhood data, a survey of Section 811 residents, and

A critical component of the impact study was selecting

administrative interviews with staff who administer the

an analytic sample from the universe of individuals in

PRA program.

the study groups so that the comparison groups are

similar to the PRA resident group in terms of existing

B.2.1 HUD Administrative Data

diagnoses, disability types, prior healthcare utilization,

The study uses several HUD administrative data sources

and demographic characteristics. The study used

to capture information on PRA and PRAC residents,

a multi-stage data matching approach to construct

PRA and PRAC properties, PRA grantees, and PRAC

similar comparison groups and used propensity

property owners. Data sources are the Integrated

score reweighting and regression models to estimate

Real Estate Management System (iREMS), the Tenant

the differences in outcomes between the PRA and

Rental Assistance Certification System (TRACS), HUD’s

comparison groups.

Inventory Management System / PIH Information Center

(IMS/PIC), and HUD Multifamily’s Online Property

76 NED Category II vouchers awarded in 2011 enable non-elderly persons with disabilities to transition from nursing homes and other healthcare institutions into the community. PHAs that received NED Category II vouchers were required to ensure that voucher holders have access to community-based supportive services needed for residents to live independently in the community.

122

H U D S e c t i o n 811 P R A P ro g r a m – P h a s e I I E v a l u a t i o n

Appendix B: Section 811 PRA Phase II Evaluation Methods

Integrated Information Suite (OPIIS). These data sources

all three study components. Exhibit B.2.1 lists the HUD

are combined and used to construct analytical samples

administrative data sources and the types of data we

and to provide administrative data that are important to

obtained from each source.

E xhibit B.2.1: H U D Administrative Dat a S ources

HUD Data Source

Administrative Data

Integrated Real Estate Management System

PRA and PRAC property data

Tenant Rental Assistance Certification System

PRA and PRAC resident data

Inventory Management System / PIH Information Center

NED and Other HUD-assisted comparison group resident data

Office of Multifamily Online Property Integrated Information Suite

PRA and PRAC property data

Section 811 PRA Quarterly Reports

PRA program data

Abt staff requested and received HUD administrative data

Integrated Real Estate Management

from TRACS, iREMS, and IMS/PIC at the end of each

System (iREMS)

quarter during the study period from March 31, 2017,

Information from HUD’s iREMS contains data on the HUD

through September 2018.77 The first sample included

Office of Multifamily Housing’s portfolio of insured and

participant-level data as of the end of each quarter ending

assisted properties. Abt received 36 iREMS data tables

between March 31, 2014, and March 31, 2017. Abt staff

for each quarter between March 31, 2017, and September

received subsequent data sets each quarter between for

30, 2018, for the six study states. iREMS data includes

the quarters ending June 30, 2017; September 30, 2017;

property data for PRAC properties and for units with PRA

December 31, 2017; March 31, 2018; June 30, 2018; and

units under contract.

September 30, 2018.

The iREMS data extracts included the fol owing types of

The study uses HUD administrative data from different

information:

quarters for different purposes. In general, we relied on

the most recent available data at the time of the analysis:

• properties (building type, number of units, number of

assisted units, number of market- rate units, date of

• Abt used the June and September 2017 data files to

construction or last rehabilitation),

pull the match files for the CMS data extract, the state

Medicaid extracts, and the resident survey sample.

• PRA or PRAC-assisted units (bedroom size,

accessibility, contract rent, fair market rent, utility

• From the December 2017 extract, we added any

al owance), and

additional PRA and PRAC residents for the resident

survey sample that were not in the September 2017

• owners (type of owner organization and legal structure,

extract.

owner and management agent contact information).

• Abt used the HUD administrative data extracts from

Data from the iREMS database was used to provide

March 31, 2018, for the descriptive demographic

descriptive characteristics of PRA and PRAC properties.

information about residents, for geocoded location

Owner and management contact information was used to

information to determine neighborhood outcomes, and

identify and conduct outreach to PRA and PRAC residents

for estimating rental subsidies in the economic study.

for surveys.

• Abt used grantee quarterly report data from al

Some of the iREMS data were incomplete or appeared

quarters December 31, 2014, through September 30,

inaccurate for a portion of PRA and PRAC properties.

2018.

For example, the database is missing data accessibility

of most units under lease for both PRA and PRAC

properties. The building type was not populated for the

majority of PRAC properties, and data on the number

77 Abt received HUD administrative data extracts as of March 30, 2017; June 30, 2017; September 30, 2017; December 30, 2017; March 30, 2018; June 30, 2018; and September 30, 2018. Abt received additional data for TRACS and IMS/PIC for the periods ending March 30, 2014; June 30, 2014; September 30, 2014; December 30, 2014; March 30, 2015; June 30, 2015; September 30, 2015; December 30, 2015; March 30, 2016; June 30, 2016; September 30, 2016; and December 30, 2016. Abt anticipates receiving additional data for periods ending March 30, 2018, and June 30, 2018, for all data sources

123

H U D S e c t i o n 811 P R A P ro g r a m – P h a s e I I E v a l u a t i o n

Appendix B: Section 811 PRA Phase II Evaluation Methods

of buildings for many properties appear inaccurate. The

five IMS/PIC data tables for each quarter between March

number of total units in properties with PRA units was

31, 2017, and March 2018. The tables provide resident

also missing or incorrect for approximately 10 percent

information in these HUD programs that is similar to the

of properties with PRA units under contract. To obtain

PRA and PRAC resident information in TRACS. Similar to

corrected information on total unit counts for these

the TRACS data, the IMS/PIC tenant-level data contain

properties, Abt staff contacted the state housing agencies

geocoded addresses that were used to match PRA and

or the property owners.

PRAC residents to neighborhood characteristics and

indicators and contains PII that is used to match housing

Tenant Rental Assistance

records to CMS and state Medicaid enrol ment and claims

Certification System

records.

TRACS is the main system that HUD uses to col ect and

store data on the individuals and families living in HUD

Online Property Integrated Information Suite

multifamily housing. Abt received five household-level

HUD’s OPIIS is a consolidated source of data from

TRACS data tables for each quarter between March

HUD’s various multifamily systems. It includes a variety

2017 and March 2018. These tables provide information

of property-level information, including annual financial

on PRA and PRAC households including financial

statements and contract details. Abt used financial

(household income, subsidies received) and demographic

information from OPIIS to inform the economic study.

characteristics (household composition, age, race,

and gender). TRACS also provides data on units and

Section 811 PRA Quarterly Reports

properties (unit type, unit numbers, property locations,

PRA grantees submit quarterly reports 30 days after the

and contract rents). Abt received TRACS data extracts for

end of each quarter. Quarterly reports include data on the

PRA and PRAC residents in the six study states for each

number and characteristics of PRA units under contract

quarter between March 30, 2014, and March 30, 2018, to

and under lease, and descriptive data on PRA applicant

inform the analytic sample. TRACS data was also used to

and residents. Abt received quarterly reports for every

calculate rental subsidies for the economic study.

quarter from March 30, 2015, through September 2018, for

We used geocoded addresses from TRACS to match

all states in the PRA program.78

PRA and PRAC residents to neighborhood characteristics

B2.2 Healthcare Data

and indicators. We used property identifiers to match

tenant and property data in TRACS and to property data

The study obtained data on chronic conditions and

in iREMS. TRACS also contained personal y identifiable

healthcare utilization from CMS and from state Medicaid

information (PII) necessary for matching housing records

agencies.

to CMS and state Medicaid enrol ment and claims

records.

CMS Research Data Assistance Center

TRACS data have some limitations. PRA and PRAC

The study team obtained Medicaid and Medicare

household data in TRACS are not always current as

enrol ment and claims data from CMS’s Research Data

owners have up to 90 days after a household moves into

Assistance Center (ResDAC) in January 2018. ResDAC

an assisted unit before having to report tenant information

is the data warehouse where Medicare claims and

in TRACS. It was also not clear whether program exit data

administrative files are housed along with Medicaid claim

were entered by owners in a timely manner.

history. The study team used HUD administrative data

on PRA, PRAC, and other HUD-assisted households to

Inventory Management System / PIH

generate a finder file to submit to ResDAC. In response,

Information Center

we received individual-level CMS administrative data for

all the requested HUD-assisted residents matched to

HUD’s IMS/PIC systems store information on units,

Medicaid and Medicare beneficiaries in the most recent

buildings, and residents housed through HUD’s public

data available (event-level Medicare fee-for-service (FFS)

housing, housing choice vouchers (HCV), and NED

claims for 2014 and 2015, and Medicaid FFS claims and

programs. The study team used IMS/PIC data to

managed care encounter data for 2012–2013). We also

select individuals in the NED and other assisted HUD

received these data for a large random sample of adults

comparison groups and for the descriptive analysis of

under the age of 65 and who were residents of the six

demographic and household characteristics. Abt received

study states to serve as the non-HUD comparison group.

78 This includes reports for March 31, 2015; June 30, 2015; September 30, 2015; December 31, 2015; March 31, 2016; June 30, 2016; September 30, 2016; December 31, 2016; March 31, 2017; June 30, 2017; September 30, 2017; December 31, 2017; March 31, 2018; June 30, 2018; and September 30, 2018; during the study period.

124

H U D S e c t i o n 811 P R A P ro g r a m – P h a s e I I E v a l u a t i o n

Appendix B: Section 811 PRA Phase II Evaluation Methods

The ResDAC data included four segments of the Medicare

enrol ment data in order to measure outcomes at a date

Beneficiary Summary File (MBSF) for 2014 and 2015.

after placement in PRA units for as many participants

MBSF data contain individual-level summary information

as possible. We obtained these data for PRA and PRAC

on demographics, enrol ment, and annualized payment

residents, NED voucher recipients, residents of other

and utilization variables for any Medicare beneficiary that

HUD-assisted housing, as well as individuals not in

had coverage at any point in time during the 2-year period.

HUD-assisted housing, between ages 18 and 64, enrol ed

We also received Research Identifiable Files (RIF), which

in Medicaid for at least one month, and with diagnoses

contain event-level Medicare FFS claims for institutional

and medical procedures identified as predictive of PRA

(Part A) and non-institutional (Part B) providers. Claims

participation (the non-HUD comparison group).79 Section

include beneficiary identifiers, provider of service

3.5 discusses how we used the ResDAC data to identify

identifiers, dates of service, diagnosis codes, procedure

historical diagnoses and medical procedures predictive of

codes, and reimbursement amounts. RIFs are organized

PRA participation.

by type of claim. The data we received include event-level

Each state provided us with all fee-for-service claims and

records of inpatient (not including skilled nursing facility

managed care encounter data for Medicaid enrollees

or hospice), outpatient, physician/supplier, and durable

who were matched to individuals in the PRA and HUD

medical equipment services.

comparison groups based on the Social Security Number,

There are limitations to ResDAC data. Medicare

name, gender, and date of birth.

administrative data readily available from CMS for

The state claims data we received represent two periods

research purposes do not include managed care

of time:

encounter data for beneficiaries enrolled in a Medicare

managed care plan (Part C). Even the summary

• California, Delaware, Louisiana, and Maryland

information on utilization and spending in the MBSF Cost

provided Medicaid enrol ment and adjudicated claims

and Use segment does not include utilization covered by

and encounter data from January 1, 2015, through

managed care plans.

December 31, 2016.

We also received data from the Medicaid Analytic eXtract

• Minnesota and Washington provided adjudicated data

(MAX) database. These data included the Medicaid

from January 1, 2015, through June 30, 2017.

Person Summary file and individual FFS claims and

Fee-for-service claims include beneficiary identifiers,

managed care encounter data for 2012 and 2013. Like the

claim type identifiers, providers of service identifiers,

MBSF, the Person Summary file contains demographic

dates of service, diagnosis codes, medical procedure

characteristics, Medicaid enrol ment, and annualized

codes, and reimbursement amounts. For managed care

payment and utilization variables for each beneficiary

enrol ees, Medicaid agencies pay a monthly per member

that had Medicaid coverage at any point in time during

fee to managed care organizations to provide healthcare

the 2 calendar years. Medicaid FFS claims and managed

services required by enrol ees. The organizations submit

care encounter data contain event-level information on

“encounter” claims with the same information about the

beneficiaries’ utilization of inpatient, outpatient, long-term

service, treating provider, and other patient-level detail

care, and pharmacy services, including dates of service

found in FFS claims but, for some states, the encounter

and corresponding diagnosis and procedure codes. FFS

data does not include information on reimbursement to

claims contain information on amounts reimbursed to the

providers.

provider by Medicaid; managed care encounter records

There are limitations to the state Medicaid data. The study

do not include reimbursement information.

team put significant effort into working with the state

State Medicaid Data

Medicaid agencies to procure data fields as uniform as

possible across the six states but, as expected, there was

The study team entered into individual Data Use

significant variability in the content and structure of the

Agreements (DUAs) with state Medicaid agencies in

data. Despite substantial differences between the states’

each of the six study states to access state Medicaid

data, we were able to define a core set of measures of

data claims data for individuals in the analysis sample.

healthcare utilization that were specified the same across

We obtained the most recent available individual-level

the six study states, which are described in Section 3.1.

Medicaid FFS claims, managed care encounter, and

79 We provided the states with specific criteria related to diagnoses and medical procedures to select a sample of Medicaid enrollees for the non-HUD group. To reduce the size of the non-HUD sample to one agreed upon by the Abt and state’s institutional review boards, we asked states to randomly sample from the pool of Medicaid enrollees who matched these criteria (California also restricted the sample to enrollees in PRA ZIP

Codes). The final sample sizes of the non-HUD groups provided to us by the states varied across the six states. We later matched these non-HUD

samples to PRA residents using a propensity-score weighting and multivariate regression based on demographics, healthcare utilization rates, and diagnoses of chronic or disabling conditions.

125

H U D S e c t i o n 811 P R A P ro g r a m – P h a s e I I E v a l u a t i o n

Appendix B: Section 811 PRA Phase II Evaluation Methods

Data limitations related to lags in the availability of

Census-tract level data for these measures were matched

complete Medicaid claims and managed care encounter

to individuals in the comparison groups using geocoded

data col ected directly from states affect the analysis. The

locations from HUD administrative data. Additional y, we

healthcare utilization analysis relies on state Medicaid data

weighted outcomes and adjusted for multiple comparisons

to compare healthcare utilization before PRA residents

as part of the impact study.

moved into their PRA unit with their post-move-in

experiences. We obtained state Medicaid data from al

B.2.4 Survey of Section 811 PRA

states starting from January 2015. Except for two out of

and PRAC Residents

the six states, the latest state Medicaid data available was

Abt conducted in-person surveys with 403 Section 811

through December 2016. Medicaid data from Minnesota

PRA and PRAC residents in the study states to gain their

and Washington was through June 30, 2017. Because of

feedback on their housing and neighborhood, daily life,

these lags, and because the number and pace of PRA

and access to the services and supports they need. The

lease-ups has been modest, we restricted the study

study team used the results of the resident survey to

sample to PRA residents who moved into their units during

compare outcomes of PRA and PRAC residents in the

2016 (or the first half of 2017 if they resided in Minnesota

impact study and to provide the resident perspective for

or Washington), and we only observed seven months of

the implementation study.

health care utilization, on average, after they moved into

• PRA Residents: Abt conducted outreach to al

their units. The reduced sample size may limit our ability

residents in all PRA households who had been living in

to detect impacts, and the results from this brief fol ow-up

PRA units and identified in HUD administrative data as

period may not reflect longer-term patterns that may be

of December 2017.

observed in the future.

• PRAC Residents: Abt included only residents who

B.2.3 Neighborhood Data

moved into PRAC properties in 2013 or later in order to

The impact analysis uses a number of neighborhood

make their tenure more similar to that of PRA residents.

indicators to assess neighborhood quality and

Abt obtained a sample of 100 residents per state by

neighborhood characteristics. The study team uses

randomly selecting PRAC properties with at least

neighborhood indicators from four publicly available

three residents who met our criteria and were within

sources:

10 miles of a PRA property. The maximum of PRAC

residents selected per property was 12. Residents

• The 2017 American Community Survey (ACS) for

were randomly ordered within each property so that,

census tract-level household data on the percent of

if there were more than 12 residents or the sample

households with a person with a disability; on income,

reached 100 residents before sampling all residents

education, race and ethnicity characteristics; and

at a property, there would be no selection bias in who

on urban area designation. We also extracted 2010

was surveyed. Abt selected additional properties and

census-level data on the neighborhood’s housing

residents as replacement residents in case we did not

stock, including the percent that is single-family owner-

obtain our needed completion rates from the sample

occupied and the numbers of units in buildings within

of 100 PRAC residents from each state. In states

census tracts.

where the number of PRAC residents within ten miles

• HUD’s Affirmatively Furthering Fair Housing (AFFH)

of a PRA property was not much higher than 100, Abt

neighborhood data for census-tract level information

included additional PRAC properties within 15 miles of

on percent of households with incomes below the

PRA properties as replacements.

poverty line and on neighborhoods that are Racial y

Abt designed the resident survey instrument largely by

and Ethnical y Concentrated Areas of Poverty

using or adapting items from three existing validated

(RECAP).

surveys: the Money Fol ows the Person (MFP) Quality

• Transit Index and Walkability indexes from the AARP’s

of Life survey, the Consumer Assessment of Healthcare

Livability Index.80

Providers and Systems (CAHPS) Home and Community

• Air quality ratings from the U.S. Department of

Based Services Survey, and HUD’s Resident Satisfaction

Environmental Protection (EPA)’s 2014 National Air

Survey. The resulting 75-item survey took between 20 and

Toxics Assessment database.81

45 minutes to complete. Fol owing skip patterns, not al

residents answered al the questions.

80 https://www.huduser.gov/portal/affht_pt.html#affhassess-tab, http://opportunityindex.org/about/; https://livabilityindex.aarp.org/

81 https://www.epa.gov/national-air-toxics-assessment/2014-national-air-toxics-assessment

126

H U D S e c t i o n 811 P R A P ro g r a m – P h a s e I I E v a l u a t i o n

Appendix B: Section 811 PRA Phase II Evaluation Methods

Abt conducted cognitive testing of the survey instrument

Abt staff completed 403 surveys: 194 surveys with PRA

at six PRAC properties in Massachusetts in April and May

residents and 209 surveys with PRAC residents in the six

2017. After cognitive testing, Abt conducted a pilot test

case study states (Exhibit B.2.2.) The PRAC survey results

with the revised survey instrument with six PRA residents

were weighted and adjusted for multiple comparisons as

in Maryland in November 2017. Abt conducted outreach

part of the impact study.

to PRA and PRAC residents by mail in the six case

study states, using resident addresses from the TRACS

B.2.5 PRA Program Documents

administrative data, and maintained a tol -free number for

The study team reviewed a number of PRA program

residents to call to schedule a time to complete the survey

documents for programs in the six study states. These

in person. As potential survey respondents were selected

include grantee applications and related materials for the

from HUD administrative data that was not always up to

2012 and 2013 funding rounds, Cooperative Agreements

date, in some cases additional eligible residents were

between HUD and the grantees, and partnership

identified and selected when the interviewers were onsite

agreements between grantees and their partners. Where

conducting surveys with other residents.

available, the team also reviewed written PRA program

E xhibit B.2.2 : Sur veys C ompleted by S t ate and Type of Proper t y

State

# of PRA Surveys Completed

# of PRAC Surveys Completed

Total Surveys Completed

California

35

25

60

Delaware

17

24

41

Louisiana

62

27

89

Maryland

21

73

94

Minnesota

33

29

62

Washington

26

31

57

Total

194

209

403

Source: Abt Associates’ internal survey tracking.

policies and procedural documents for details on how

partners to learn how the PRA program is carried out

the grantee and partners implement the grant program

in the study states. Abt staff conducted interviews with

in their state. Examples of procedural documents that

staff who administer the PRA program at state housing

were reviewed are outreach materials to property owners,

agencies and partnering state health agencies, and

property owner applications for PRA subsidies, tenant

staff from a sample of PRA program partners: owners

selection plans, tenant program applications, and PRA

of properties with PRA units, and service providers who

property monitoring and inspections protocols. The study

serve PRA applicants or residents (Exhibit B.2.3).

team also col ected and reviewed relevant statewide

Staff also conducted telephone interviews with staff from

plans and reports to help document external factors

five public housing authorities (PHAs) that set aside or

in the six study states that could potential y affect the

established admissions preferences for a set number of

implementation of the PRA program. These include

HCVs or other rental units specifical y for extremely low-

Olmstead Plans and updates, statewide plans to end

income, non-elderly people with disabilities.

homelessness, state and local affordable housing studies,

housing needs assessments, Analysis of Impediments to

Most of the administrative interviews were completed

Fair Housing Choice, and information on services available

in person during site visits, including all of the grantee

through state Medicaid home and community-based

interviews and all but one of the health agency staff

services (HCBS) waiver programs.

interviews. The study team used NVivo qualitative analysis

software to systematical y review interview responses

B.2.6 Interviews with PRA Program Staff

and code the data by a series of themes and topic areas

The study conducted qualitative interviews between

organized by the study’s research questions.

October 2017 and May 2018 with grantees and program

127

H U D S e c t i o n 811 P R A P ro g r a m – P h a s e I I E v a l u a t i o n

Appendix B: Section 811 PRA Phase II Evaluation Methods

E xhibit B.2.3 : C ompleted Administrative Inter views for P R A Evaluation

# of Property

# of Service

Grantee Agency/ Partnering

Owners

Providers

State

Housing Agencies

Partnering Health Agencies

Interviewed

Interviewed

California Housing Finance Agency

Department of Housing and

Department of Health Care Services

California

4

8

Community Development

Department of Developmental Services

Tax Credit Allocation Committee

Delaware

Delaware State Housing Authority

Department of Health and Social Services

4

3

Louisiana Department of Health

Louisiana

Louisiana Housing Authority

5

4

and Hospitals

Maryland Department of Health

Maryland Department of Housing

Maryland

and Mental Hygiene

4

6

and Community Development

Maryland Department of Disabilities

Minnesota

Minnesota Housing Finance Agency Minnesota Department of Human Services

3

2

Washington Department of

Washington

Department of Social and Health Services

4

4

Commerce

Total

8

8

24

27

Source: Abt Associates’ internal interview tracking.

B.3 The Analytic Sample for the

The impact study compares PRA to each of the

Impact and Economic Studies

counterfactuals (PRAC, NED, or other HUD, and non-

HUD programs) in several key areas (Exhibit B.3.1). The

The objective of the PRA evaluation is to examine early

study does not compare all categories of impacts for al

evidence of the effectiveness of the PRA program for

the comparison groups. The primary comparison group

its residents compared to similar people living in other

is PRAC, and all impacts measured for PRA in the study

settings. The central research question addresses what

are compared to PRAC. Neighborhood outcomes and

might have happened to this group of very low income,

costs are compared for PRAC, NED, and other HUD

non-elderly people with disabilities if the PRA program

groups using administrative data from HUD on housing,

did not exist. To answer this question, several alternative

unit, and tenant characteristics, as well as enrol ment and

counterfactual states could be conceptual y relevant. One

healthcare claims data from CMS and state Medicaid

possibility is to assume that everyone in PRA units might

agencies. Because we only have CMS and state Medicaid

instead have been placed in Section 811 PRAC units.

data on healthcare utilization and spending for the non-

Another possibility is that they would be in other HUD-

HUD group, the comparison of PRA to the non-HUD group

assisted housing programs for persons with disabilities,

is only made for healthcare outcomes and costs.

including the NED voucher program, or in other HUD-

The study uses survey results of PRA and PRAC residents

assisted housing like public housing or the HCV program.

to compare resident perception of their properties and

Still another possibility is that they would be spread across

neighborhoods, use and need for tenancy supports, and

a wide variety of housing options not subsidized by HUD,

how well residents are integrated into their communities.

both institutional and community- based, with or without

structured access to long-term services and supports.

128

H U D S e c t i o n 811 P R A P ro g r a m – P h a s e I I E v a l u a t i o n

Appendix B: Section 811 PRA Phase II Evaluation Methods

E xhibit B.3.1: C ategories of Measured Impact s C ompared to P R A

Impact Category

PRAC

NED

Other HUD

Non-HUD

Housing tenancy

X

Neighborhood characteristics

X

X

X

Perception of property and

X

neighborhood

Use and need for tenancy supports

X

Community integration

X

Healthcare utilization

X

X

X

X

Costs

X

X

X

Abt created an analytic sample from the universe of

estimate the differences in outcomes between the PRA

individuals in the study groups so that the comparison

and comparison groups.

groups are comparable to the PRA resident group in terms

of existing diagnoses, disability types, prior healthcare

B.3.1 Impact Study Outcome Measures

utilization, and demographic characteristics, such as age

Exhibit B.3.2 provides a summary of the outcomes of

and gender. The study uses a multi-stage data-matching

interest for the Impact Study, the measures we used,

approach to construct similar comparison groups and

and the data sources for these measures. Measures are

propensity score reweighting and regression models to

categorized according to research question.

E xhibit B.3.2 : Impact S tudy Research Matrix

Impact Research Domain

Measures

Data Source

How does healthcare utilization post-occupancy among PRA residents compare to PRAC residents, to other HUD assisted housing residents, and to a non-HUD comparison group?

• # of inpatient hospital admissions

• # of inpatient hospital days

• States’ Medicaid

Inpatient admissions

claims/administrative

• # of 30-day inpatient hospital readmissions

data

• # of inpatient hospital admissions for mental health conditions

• # of visits per quarter

• States’ Medicaid

Emergency department visits

claims/administrative

• # of visits not resulting in inpatient admission

data

• States’ Medicaid

Medical transportation

• # of days that emergency or non-emergency medical transportation was used

claims/administrative

data

• # of days in long-term inpatient care

• # of admissions to long-term inpatient care

• States’ Medicaid

Long-term services and supports

• Indicator of any admission to long-term inpatient care

claims/administrative

• Indicator of any use of personal care attendant services

data

• Indicator of any use of case management services

How do quality of life and care PRA residents compare to PRAC residents?

• Participant-reported physical health status

• PRA and PRAC

Health status

• Participant-reported mental health status

resident survey

( cont )

129

H U D S e c t i o n 811 P R A P ro g r a m – P h a s e I I E v a l u a t i o n

Appendix B: Section 811 PRA Phase II Evaluation Methods

E xhibit B.3.2 : Impact S tudy Research Matrix ( cont )

Impact Research Domain

Measures

Data Source

• Support received from informal caregivers

Access to supportive services

• PRA and PRAC

• Housing modification and special equipment needs

and unmet needs

resident survey

• Unmet need for supportive services

• PRA and PRAC

Quality of care

• Participant feedback on quality of care received by service providers

resident survey

• PRA resident perspective on housing choice and whether housing and location

• PRA and PRAC

Transition to Section 811

preferences were met

resident survey

• Types of transition supportive services received

• Residents can be independent in the property

• Residents knowing other people in the property

• PRA and PRAC

Community integration

• Residents knowing other people in the neighborhood

resident survey

• Residents reporting being able to see friends and family when they want to

How do housing and neighborhood characteristics for PRA residents compare to PRAC residents?

• Participant rental subsidy amounts

• Contract rent

Rental subsidies

• Contract rent as a percentage of the area Fair Market Rent

• TRACS data

• Utility allowances

• Total tenant payments

• Average length of stay before exiting

Tenure

• TRACS data

• Reasons for program exits

• Resident perception of housing quality

• Resident perception of privacy in unit and property

• PRA and PRAC

Housing quality

• Resident perception of property safety

resident survey

• Resident-reported maintenance problems with unit or property

• Resident report of wanting to move from home and reason why

• Resident perception of neighborhood safety

• Resident report of being able to get around neighborhood easily

• PRA and PRAC

Neighborhood quality

• Resident report of being able to access transportation when they need it

resident survey

• Participant report of access to grocery stores, pharmacies,

healthcare services, etc.

• Percentage of population below poverty line

• Racially/Ethnically-Concentrated Areas of Poverty (R/ECAP)

• Affirmatively Furthering

Neighborhood demographics

• Percent of adults ages 25 and over with high school diploma or higher

Fair Housing Data

• Disabled population by age

• AARP Livability Index

• Transit: frequency of public transportation

• Opportunity Index

Livability indicators

• Walkability: modeled daily walk trips per household

• Affirmatively Furthering

• Environmental health: potential exposure to harmful toxins

Fair Housing Data

130

H U D S e c t i o n 811 P R A P ro g r a m – P h a s e I I E v a l u a t i o n

Appendix B: Section 811 PRA Phase II Evaluation Methods

B.3.2 Constructing the Analytic Sample

In the first and second stages, we used tenant

characteristics data from HUD, including personally

The analytic sample for the impact study comprises

identifiable information, to match PRA, PRAC, NED, and

PRA and four comparison groups. Individuals living in

other HUD-assisted individuals to beneficiaries found

PRA units represent the primary “treatment” group of

in the ResDAC Medicare database and the Medicaid

interest, and similar individuals the PRAC, NED, other

Analytic eXtract (MAX) database.

HUD, and non-HUD groups provide distinct counterfactual

comparison groups. We did not have access to the

We received the most recent 2 years of final action claims

entire universe of Medicaid and Medicare administrative

and encounter data available from ResDAC as of October

data to match the four comparison groups to the PRA

2017 (Medicaid data for 2012–2013 and Medicare data for

residents, so the primary analytic approach to creating

2014–2015). These data included matches to a finder file

the comparison groups was weighting based on race and

of Social Security numbers (SSNs) provided to ResDAC

ethnicity, gender, age, any dependents, income, rent, prior

to identify individuals in the PRA, PRAC, NED, and other

Medicaid and Medicare enrol ment, diagnoses of chronic

HUD-assisted groups (Exhibit B.3.3). They also included

and disabling conditions, and historic rates of healthcare

three groups of other age-eligible residents of the six

utilization.

study states to make up the non-HUD group:

• Group 1 consists of 869,425 SSNs not in the study

Multi-Stage Process for Constructing the

finder file that appear in both Medicare and Medicaid

Analytic Sample

Control files (have unique ResDAC “Bene IDs” that are

The study team col ected and matched information from

in common).

HUD, CMS, and state Medicaid Agencies’ administrative

• Group 2 consists of 694,662 Bene IDs that are not in

data using a multi-stage process. We estimated

the Medicaid cohort and are not in the finder file.

regression models using propensity-score-based weights

to approximate the “propensity-score double robust”

• Group 3 consists of 3,140,550 Medicaid MSIS

(PS-DR) method, using both a weighted sample and

(Medicaid Statistical Information Statistics) ID/state

regression adjustment to determine the treatment effect

codes that do not have a corresponding Bene ID in the

on treated units.

Medicare cohort and are not in the finder file.

E xhibit B.3.3 : Percent age of H U D Tenant s on September 3 0, 2017,

Enrolled in Medicaid in 2012 –2013 or Medicare in 2014 –2015

Any Match

Medicare Match

Medicaid Match

Both Match

Group

Total

N

%

N

%

N

%

N

%

PRA

426

368

86%

101

24%

362

85%

95

22%

PRAC

6,125

5,561

91%

3,425

56%

5,278

86%

3,142

51%

NED

8,282

7,580

92%

3,821

46%

7,276

88%

3,517

42%

Other HUD

165,082

152,621

92%

73,581

45%

146,248

89%

67,208

41%

Total

179,915

166,130

92%

80,928

45%

159,164

88%

73,962

41%

Sources: Abt Associates analysis of TRACS and PIC household data from the quarter ending on September 30, 2017; Medicaid Analytic eXtract (MAX) Person Summary File for 2012–2013 received through ResDAC; and Medicare Beneficiary Summary File Base A/B/D segment for 2014–2015 received through ResDAC.

The third stage involved using the HUD tenant data for

sample of data we could obtain for the non-HUD sample.

individuals receiving PRA, PRAC, NED, or other HUD

To comply with the agency requests to narrow the sample,

assistance to request matches to beneficiaries found in

we provided each state with a specific set of diagnostic

data from Medicaid agencies in the six study states. We

criteria to use in selecting the non-HUD sample. This

also requested a larger sample of Medicaid enrol ees (not

strategy al owed us to receive coarsely matched data for

receiving HUD assistance) to serve as the non-HUD group

the non-HUD group, which facilitated propensity score

in each state. Because state Medicaid agencies limit the

modeling for refining the group at a later stage. To select

amount of sensitive personal health information they wil

the criteria, we used the ResDAC data matched to PRA

transmit to third parties such as Abt, we were limited in the

residents and the non-HUD sample and analyzed primary

131

H U D S e c t i o n 811 P R A P ro g r a m – P h a s e I I E v a l u a t i o n

Appendix B: Section 811 PRA Phase II Evaluation Methods

diagnosis codes on medical claims and encounter data

chronic and disabling and conditions and historical

to identify clusters of diagnosis codes most predictive of

healthcare utilization patterns. Demographic and

receiving PRA assistance.

housing information from HUD administrative data is

The final stage was to finalize the analytic samples

also included in the PS-DR model.

by implementing the PS-DR method to derive weights

3. PRA and PRAC survey outcomes. We also matched

that will make the comparison groups similar to PRA

the survey samples of PRA and PRAC residents using

residents in demographic characteristics and in healthcare

HUD administrative data and the ResDAC data.

utilization and diagnostic information history.

Descriptive Demographic Characteristics of

Three Analytic Samples

the Study Groups

We created three separate analytic samples during the

In creating the analytic sample, we reviewed observable

final stage. The several different types of outcomes of

characteristics of the individuals whose outcomes we

interest in the impact study are measured using different

were analyzing to describe how PRA residents and the

data sources (HUD administrative data, State Medicaid

other groups differ on the basis of demographic and

data, PRA and PRAC resident surveys, public-use data),

socioeconomic characteristics. The comparison provides

each with their own time-horizons, inclusion criteria, and

context for the impact study research by providing insight

other sorts of limitations. The PS-DR method was used to

into how PRA residents are, on average, different from

derive weights specific to each sample of PRA residents

other non-elderly, non-disabled served by other HUD

and comparison groups for:

housing programs in terms of observable demographic

1. Healthcare utilization outcomes. The analyses

and socioeconomic characteristics.

of healthcare utilization used state Medicaid data

We calculated averages using the sample means of

col ected only during 2015 and 2016, so we restricted

individual-level characteristics for race and ethnicity, age,

the sample to PRA residents who moved in during

household size, unit size, and annual income and subsidy

2016, and individuals in the HUD comparison groups

amount. We selected these variables both because

who moved into their units between 2013 and 2016

they are prominent demographic and socioeconomic

in order to observe utilization by PRA residents

characteristics and because they are included in HUD

after they move into their units.82 The covariates

administrative data. To calculate these averages, we used

used to estimate the propensity-score weights

administrative data from TRACS (for PRA and PRAC) and

and regressions were based on demographic and

IMS/PIC (for NED and other HUD programs) where each

housing-related information from HUD administrative

characteristic is reported for each individual. Our sample

data, as well as diagnoses of chronic and disabling

in this analysis includes all individuals under age 65 in the

conditions and prior healthcare utilization measured

six study states who receive assistance from the PRA,

from the 2015 state Medicaid data.

PRAC, or NED programs and all individuals under age 65

2. Neighborhood outcomes. The analysis of

who report having a disability and who receive assistance

neighborhood characteristics features a more

from other HUD programs.

extensive sample of PRA residents and comparison

B.3.3 Impact Analysis

groups, although the comparison groups were

restricted to individuals who moved into their

Within the coarsely matched samples of PRA residents

units in 2013 or later.83 In order to capture the

and the four comparison groups, regression models

most information possible for the entire sample

corresponding to the equation below were estimated

of individuals, we used the less recent 2012–2013

using a propensity-score-based weight to approximate

Medicaid data and 2014–2015 Medicare data from

the PS-DR method, using both a weighted sample and

ResDAC to match the comparison groups (excluding

regression adjustment to determine the treatment effect

the non-HUD group) to PRA residents based on

on treated units.

82 The earliest year that a PRA resident had moved into their unit was 2013. Additionally, we also include individuals in the PRA and comparison groups who live in Minnesota and Washington and moved in during the first half of 2017 since they could be linked to post-occupancy data on utilization.

83 The year that individuals moved into their residency into the propensity score models (see Appendix section 3.2) was highly predictive of PRA participation since there was no limit placed on how long individuals in the comparison groups have been living in their units. The PRA residents did not start to move in until 2013 and most of the PRA residents in our sample moved in between 2016 and 2017. Including the year the individuals moved into their unit in the propensity score model led to the inability to balance the distributions of other resident characteristics across the PRA and comparison groups. To at least somewhat balance the length of resident tenure between the groups, we restricted the comparison groups to individuals who moved into their unit in 2013 or later.

132

H U D S e c t i o n 811 P R A P ro g r a m – P h a s e I I E v a l u a t i o n

Appendix B: Section 811 PRA Phase II Evaluation Methods

E(Y|X,T) = f(Pa + Xb)

of impacts. Only one of the two models (either the weight

Y represents an outcome, P an indicator of housing type

model or the regression model) is required to be correct

(treatment status), and X a matrix of past characteristics,

to ensure high internal validity (that is, the method is

so that the coefficient a estimates the mean treatment

robust to either assumption failing to hold). However,

effect with associated standard error (primary

there are limitations to causal inference when using a

confirmatory findings). We also estimated exploratory

method based on propensity score adjustment and

models with interactions as shown below.

regression adjustment. Both methods correct only for

selection into treatment that depends only on observable

E(Y|X,T) = f(Pa + Xb + PXc)

factors. If there are unobserved variables that both

In this type of model, c measures treatment effect

affect selection into PRA and directly affect outcomes,

heterogeneity and the overall impact must be calculated

our method will not correct bias that arises from those

as the average partial effects of P. The impact estimates

unobserved confounders. The data were analyzed using

from these exploratory models produced nearly identical

SAS and Stata software, with the primary impact analysis

results on average, so we do not report them here.

conducted in Stata 15, using methods documented in

Emsley et al. (2008) and Nichols (2007).

In all of these models, f() is an inverse link function, (the

identity function f(x)=x for linear regression). All of the

Adjustments to Hypothesis Tests due to

results we report use linear regression but in exploratory

Multiple Comparisons

analysis, we also used the inverse logit f(x)=exp(x)/

[1+exp(x)] for each binary outcome, the exponential

We estimate the effect of the PRA program on several

function f(x)=exp(x) for count or nonnegative outcomes,

different outcomes and types of outcomes. Therefore,

and an inverse ordered logit for ordered categorical

it is necessary to adjust the levels used to define the

outcomes such as a Likert scale. After converting back

statistical significance of individual comparisons in order

to the original scales, impact estimates are essential y

to account for the increased probability of reporting

the same as those that arise from the linear model, so we

false positive results through multiple tests only by

report the same type of results throughout for simplicity.

chance. Such adjustments are designed to control

study-wide error rates and lower the probability of falsely

Prior to the regression analysis, we reweighted

rejecting true null hypotheses. We adjust the p-values for

each comparison group to represent the alternative

multiple comparisons using the Benjamini-Hochberg

counterfactuals. We reweighted each comparison group

(B-H) procedure (Glickman, Rao, and Schultz, 2014),

sample using estimates of the probability of being

which involves the fol owing steps. For each set of

observed in the treatment group (residing in a PRA unit).

outcomes (that is, all survey outcomes, all neighborhood

We predict that probability using a logistic regression

characteristics, and al healthcare utilization outcomes,

(logit) of an indicator of PRA residence on pre-treatment

respectively) and each comparison group (that is, PRA

characteristics (demographics, past diagnoses of chronic

versus PRAC, PRA versus NED, PRA versus other HUD, or

and disabling conditions, and past healthcare utilization).

PRA versus non-HUD):

The model generates a predicted conditional probability p i

for each observation i in the sample. Each observation in

1. Put the individual p-values in ascending order.

the comparison sample receives a weight p /(1- p ) to make

2. Assign ranks to the p-values. For example, the

i

i

the whole comparison sample representative of PRA units’

smal est has a rank of 1; the second smal est has a

outcomes had they not been assigned to PRA units but

rank of 2.

rather the comparison condition. That is, the unweighted

3. Calculate each individual p-value’s B-H critical value,

PRA sample is compared (using regression) to each of

using the formula (i/m)Q, where i is the individual

the other reweighted groups, in turn, to measure what

p-value’s rank, m is the total number of tests, and Q is

the PRA residents’ outcomes might have been had they

the false discovery rate (chosen to be 5 percent).

been instead assigned to PRAC properties, or NED, or

other HUD comparison groups, or remained in institutions

4. Compare the original p-values to the critical B-H from

or acquired alternative housing as in the non-HUD

Step 3; find the p-values that are smal er than the

comparison group.

critical value.

The PS-DR method adjusts for observable differences in

population characteristics to arrive at credible estimates

133

H U D S e c t i o n 811 P R A P ro g r a m – P h a s e I I E v a l u a t i o n

Appendix B: Section 811 PRA Phase II Evaluation Methods

B.3.4 Analysis of Program Exits

residents between 2013 and September 2018. Exhibit

B.3.4 shows the unweighted and unadjusted counts of

Using HUD administrative data from TRACS, the study

program exits observed in iREMS for PRA and PRAC

team also compared program exit rates and reasons

residents.

for exiting for PRA and PRAC residents. We analyzed

program exits and reasons for exiting for PRA and PRAC

E xhibit B.3.4 : Unweighted and Unadjusted C ount s of P R A and P R AC Program E xit s, 2013 –2018

Reason for Program Exit

PRA

PRAC

Total

Owner initiated for nonpayment of rent

46

85

131

Owner initiated--other

36

230

266

Tenant initiated--other

62

933

995

Death of sole family member

26

215

241

Abandoned unit

19

41

60

No exit observed

648

2,770

3,418

Source: Abt analysis of iREMS data between January 2013 and September 2018.

We estimated a Cox proportional hazard model of

Exhibit B.3.5 shows the Cox proportional hazard model

program exit. This model uses the same double-robust

for rates of program exits for PRA and PRAC. The model

(propensity score weighting and regression adjustment)

estimates the amount of time before an event occurs. In

approach as the other impact models. The model

our model, the event is the program exit of the resident

estimates the likelihood of exiting the program conditional

and the model estimates the amount of time residents are

on still being at risk of exit (the hazard rate), taking into

assisted by PRA or PRAC before exiting the unit for any

account the varying lengths of time a resident has already

reason. The Cox model shows that the overall exit rate at

received assistance in addition to individual baseline

any point in time is the same for PRA and PRAC residents.

demographic and health characteristics. Coefficients of

PRA and PRAC residents still at risk of an exit (that is,

the model are interpreted as hazard ratios, which in our

those who have not yet exited) both exit at a steady rate

model are the ratio of the share of PRA residents expected

of about 20 to 25 percent for each year after move-in for

to exit the program in a given month to the share of PRAC

the first 3 years and then the hazard decreased in year

residents expected to exit that program that month.

four, so persistence (or “survival” in the parlance of hazard

Hazard ratios greater than one reflect higher likelihood of

models) levels off in year 4. Sample sizes beyond month

exit for PRA as compared to PRAC, while ratios less than

48 are too small to conclude anything with confidence

one reflect a lower likelihood of exit.

about hazards or persistence beyond month 48 (in year 5).

134

H U D S e c t i o n 811 P R A P ro g r a m – P h a s e I I E v a l u a t i o n

Appendix B: Section 811 PRA Phase II Evaluation Methods

E xhibit B.3.5 : C ox Propor tional Ha zards Regression for Rates

of Program E xit s for P R A and P R AC Resident s — A ny E xit Type

Exhibit B.3.5: Cox Proportional Hazards Regression

for Rates of Program Exits for PRA and PRAC Residents—Any Exit Type

1

.8

.6

Survival

.4

.2

12

24

36

48

60

Months in Unit

PRA

PRAC

Source: Abt analysis of program exit data from TRACS for PRA and PRAC residents as of September 2018.

Exhibit B.3.6 shows hazard ratios for reasons for

initiated reasons. However, this large difference in hazards

exiting the program. Each row is a separate Cox model

for PRA corresponds to a very low prevalence, on the

regression reporting the coefficient on PRA residents

order of half a percent a year (compared to about 1

relative to PRAC. For example, the hazard ratio of 5.74

percent in a thousand for PRAC). In contrast, the point

for owner-initiated nonpayment of rent shows that PRA

estimate on tenant-initiated (other) reasons corresponds to

residents are 5.74 times more likely to exit for nonpayment

a larger gap in prevalence of exit, with exits on the order of

of rent than PRAC residents. In this exhibit, the only

14 percent a year for PRA and 7 percent a year for PRAC;

statistical y significant differences in reasons for exits are

however this difference could be due to chance (that is,

for owner-initiated nonpayment of rent and for tenant-

the hazard ratio does not differ statistical y from 1).

E xhibit B.3.6 : Ha zard Ratios for Reasons for Program E xit s for P R A and P R AC Resident s, 2013 –2018

Exit Definition

Hazard Ratio

Standard Error

P-value

Owner initiated for nonpayment of rent

5.74

1.84

0.00

Owner initiated—other

0.99

0.24

0.95

Tenant initiated

0.50

0.09

0.00

Death

0.90

0.25

0.69

Source: Abt analysis of program exit data from TRACS for PRA and PRAC residents as of September 2018.

Exhibits B.3.7–B.3.10 show the hazard ratios for owner-

initiated nonpayment of rent, tenant-initiated reasons, and

due to death.

135

H U D S e c t i o n 811 P R A P ro g r a m – P h a s e I I E v a l u a t i o n

Appendix B: Section 811 PRA Phase II Evaluation Methods

E xhibit B.3.7: C ox Ha zard Ratio for Program E xit s Due to Owner Initiated Nonpayment of Rent Exhibit B.3.7: Cox Hazard Ratio for Program Exits Due to Owner Initiated Nonpayment of Rent 1

.995

Survival

.99

.985

12

24

36

48

60

Months in Unit

PRA

PRAC

Source: Abt analysis of program exit data from TRACS for PRA and PRAC residents as of September 2018.

E xhibit B.3.8 : C ox Ha zard Ratio for Owner Initiated Reasons O ther T han Nonpayment of Rent Exhibit B.3.8: Cox Hazard Ratio for Owner Initiated Reasons Other Than Nonpayment of Rent 1

.95

Survival

.9

.85

12

24

36

48

60

Months in Unit

PRA

PRAC

Source: Abt analysis of program exit data from TRACS for PRA and PRAC residents as of September 2018.

136

H U D S e c t i o n 811 P R A P ro g r a m – P h a s e I I E v a l u a t i o n

Appendix B: Section 811 PRA Phase II Evaluation Methods

E xhibit B.3.9 : C ox Ha zard Ratio for Tenant Initiated Reasons

Exhibit B.3.9: Cox Hazard Ratio for Tenant Initiated Reasons

1

.8

Survival

.6

-4

12

24

36

48

60

Months in Unit

PRA

PRAC

Source: Abt analysis of program exit data from TRACS for PRA and PRAC residents as of September 2018.

E xhibit B.3.10 : C ox Ha zard Ratio for Program E xit s Due to Death

Exhibit B.3.10: Cox Hazard Ratio for Program Exits Due to Death

1

.999

.998

Survival

.997

.996

.995

12

24

36

48

60

Months in Unit

PRA

PRAC

Source: Abt analysis of program exit data from TRACS for PRA and PRAC residents as of September 2018.

137

H U D S e c t i o n 811 P R A P ro g r a m – P h a s e I I E v a l u a t i o n

Appendix B: Section 811 PRA Phase II Evaluation Methods

B.3.5 Health Conditions and Healthcare

criteria, as defined and identified by CMS using each

Utilization Measures

beneficiaries’ historical fee-for-service claims data.84 They

also use validated criteria to define and identify a set of

This section describes how the study used 2015–2016

flags for other chronic or potential y disabling conditions,

Medicaid data col ected from state Medicaid agencies to

including 9 mental health and tobacco use conditions,

measure chronic and potential y disabling conditions and

15 developmental disorder and disability-related

healthcare utilization outcomes included in the impact

conditions, and 9 other chronic physical and behavioral

analysis.

health conditions. These flags were developed by CMS

Chronic and Potentially Disabling Conditions

specifical y to enhance the research of the Medicare-

Medicaid dual-eligible population.

In order to achieve unbiased comparisons, the analytic

sample for the impact study needed to consist of PRA

We identified Medicaid enrol ees who had any of 27

residents and individuals in the PRAC, NED, other HUD,

chronic conditions or 33 other chronic or potential y

and non-HUD groups who are similar in terms of existing

disabling conditions using the CMS-defined algorithms

diagnoses and disabilities. We used 2015 state Medicaid

for each condition and all diagnosis fields in the state

data to compare and match the study groups based on

Medicaid fee-for-service and managed care encounter

the prevalence of various types of chronic conditions and

data from 2015. We grouped the chronic and potential y

disabilities.

disabling conditions into 19 categories, as shown in

Exhibit B.3.11.

CMS defines a set of beneficiary-level flags for 27

common and chronic conditions based on validated

E xhibit B.3.11: C ondensed C ategories of Chronic and Potentially Disabling C onditions

Category

Conditions

Cancer

Breast Cancer; Colorectal Cancer; Lung Cancer; Endometrial Cancer; Prostate Cancer

Cardiovascular

Acute Myocardial Infarction; Atrial Fibrillation; Chronic Heart Failure; Ischemic Heart Disease; Stroke Intellectual Disabilities and Related Conditions; Learning Disabilities; Autism Spectrum Disorders; ADHD and Other Developmental Disorders or Disabilities

Conduct Disorders; Spina Bifida and Other Congenital Anomalies of the Nervous System; Cerebral Palsy; Cystic Fibrosis and Other Metabolic Developmental Disorders; Other Developmental Delays

Endocrine and Renal

Chronic Kidney Disease; Diabetes; Acquired Hypothyroidism

HIV/AIDS

Human Immunodeficiency Virus and/or Acquired Immunodeficiency Syndrome (HIV/AIDS)

Leukemia and Lymphomas

Leukemia and Lymphomas

Liver Conditions

Liver Disease, Cirrhosis, and Other Liver Conditions

Depression; Major Depression Affective Disorder; Anxiety Disorders; Bipolar Disorders; Personality Disorders; Mental Health

Schizophrenia; Schizophrenia and Other Psychotic Disorders; Post-Traumatic Stress Disorder

Mobility Impairments

Mobility Impairments

Musculoskeletal

Hip Fracture; Osteoporosis; Rheumatoid Arthritis

Epilepsy; Spinal Cord Injury; Migraine and Other Chronic Headache; Multiple Sclerosis and Transverse Myelitis; Muscular Neurological Disorders

Dystrophy; Traumatic Brain Injury and Nonpsychotic Mental Disorders due to Brain Damage; Alzheimer's Disease; Alzheimer’s Disease and Related Disorders or Senile Dementia

Obesity

Obesity

Ophthalmic

Cataract; Glaucoma

Other Chronic Conditions

Anemia; Hyperlipidemia; Hypertension; Benign Prostatic Hyperplasia

Other Conditions

Fibromyalgia; Pressure Ulcers and Chronic Ulcers; Viral Hepatitis

Peripheral Vascular Disease (PVD)

Peripheral Vascular Disease

Pulmonary

Chronic Obstructive Pulmonary Disease; Asthma

Sensory Impairment

Deafness and Hearing Impairment; Blindness and Visual Impairment

Tobacco Use

Tobacco Use

Note: The ICD-9 and ICD-10 diagnosis codes used to define each condition are documented at https://www.ccwdata.org/web/guest/condition-categories.

84 https://www.ccwdata.org/web/guest/condition-categories

138

H U D S e c t i o n 811 P R A P ro g r a m – P h a s e I I E v a l u a t i o n

Appendix B: Section 811 PRA Phase II Evaluation Methods

Healthcare Utilization Outcome Measures

is, three months) 86,87 If the PRA resident or individual in the

Exhibit B.3.12 describes the healthcare utilization

comparison group moved into their unit in the middle of a

measures that we used for the study. To calculate these

month, we counted the healthcare services they received

measures, we used 2016 Medicaid data to examine PRA

at any time during that month as services received after

residents’ healthcare utilization for those who moved in

they moved into their unit. We calculated the same

during 2016, and for individuals in the HUD comparison

measures using the 2015 Medicaid data in order to control

groups who moved into their units between 2013 and

for past utilization in the PS-DR model. We also measured

2016.

baseline utilization rates as the number of events per

85 We accounted for the number of months in 2016

that PRA residents were enrol ed in Medicaid, after

quarter to account for different lengths of Medicaid

they moved into their PRA unit, by calculating individual

enrol ment during 2015.

utilization rates as the number of events per quarter (that

E xhibit B.3.12 : Measures Describing Healthcare U tilization

by Medicaid Enrollees Using Dat a C ollected from S t ates

Measure

Description

The count of unique admissions, for any diagnosis, to an inpatient facility, excluding admissions to skilled nursing facilities, for inpatient rehabilitation, or for hospice. A unique admission and subsequent inpatient stay is defined as a set Number of inpatient hospital admissions

of one or more consecutive inpatient claims. If the patient is transferred to a different provider, this is counted as a single admission as long as the second admission occurs within one day of discharge from the previous provider.

The total number of days in admitted to an inpatient facility, for any diagnosis, excluding admissions to skilled nursing Days admitted to an inpatient hospital

facilities, for inpatient rehabilitation, or for hospice.

This measure is a subset of the inpatient hospital admissions measure. The count of unique admissions, for any Number of 30-day readmissions

diagnosis, to an inpatient facility within 30 days of the discharge date of a previous inpatient admission, excluding to a hospital

admissions to skilled nursing facilities, for inpatient rehabilitation, or for hospice.

This measure is a subset of the inpatient hospital admissions measure. The count of unique admissions to an inpatient facility with a principle diagnosis related to a mental condition, excluding admissions to skilled nursing facilities, for Number of inpatient hospital admissions

inpatient rehabilitation, or for hospice. Mental health diagnoses are defined as: ICD-10-CM in: (F, G442, R37, R480) but for mental health conditions

not in: (F1, F53, F64, F7, F84, F90, F91) or ICD-9-CM in: (2950 through 31699) but not in: (3025, 3026, 303, 304, 305, 310, 3120, 3121, 3122, 3124, 3128, 3129, 31381, 31382, 314).

The count of emergency department visits, for any diagnosis, regardless of whether this led to an inpatient admission.

Number of emergency department visits

Two or more claims for an ED visit with the same date were counted as one ED visit.

Number of emergency department visits

The count of emergency department visits that did not lead to an inpatient admission (for any diagnosis). Two or more not resulting in inpatient admission

claims for an ED visit with the same date were counted as one ED visit.

Days that emergency and non-emergency

The count of days in a given month with at least one claim for medical transportation, regardless of whether it was or was medical transportation were used

not for an emergency.

The count of unique admissions to an inpatient facility that lasted longer than 28 consecutive days, for any reason except Number of admissions for long-term

for hospice or substance abuse. A unique admission and subsequent inpatient stay is defined as a set of one or more inpatient care

consecutive inpatient claims. If the patient is transferred to a different provider, this is counted as a single admission as long as the second admission is on or within one day of discharge from the previous provider.

The total number of days in admitted to an inpatient facility, when the inpatient stay lasted longer than 28 consecutive Days of long-term inpatient care

days, and the person was admitted for any reason except for hospice or substance abuse.

( cont )

85 The earliest year that a PRA resident had moved into their unit was 2013 (some residents resided in their units before being assisted by PRA).

Additionally, we also include individuals in the PRA and comparison groups who live in Minnesota and Washington and moved in during the first half of 2017 since they could be linked to post-occupancy data on utilization.

86 For the non-HUD group, we examined the individuals’ utilization in 2016 and accounted for the number of quarters they were continuously enrolled in Medicaid.

87 The average number of months of follow-up in 2016 (and 2017 for Washington and Minnesota residents) for PRA residents was 6.8 months, 12.0

months for PRAC residents, 15.6 months for the NED group, 11.8 months for the other HUD group, and 9.4 months for the non-HUD group.

139

H U D S e c t i o n 811 P R A P ro g r a m – P h a s e I I E v a l u a t i o n

Appendix B: Section 811 PRA Phase II Evaluation Methods

E xhibit B.3.12 : Measures Describing Healthcare U tilization

by Medicaid Enrollees Using Dat a C ollected from S t ates ( cont )

An indicator (yes/no) for whether the person was admitted to an inpatient facility for longer than 28 consecutive days, Any admission for long-term inpatient care

and for any reason except for hospice or substance abuse, while he or she was enrolled in Medicaid and during the measure period.

Any use of personal care attendant (PCA)

An indicator for whether the person received PCA services while during the measure period. PCA services were defined services

using HCPCS (procedure) codes in: (“T1019”,”T1020”,”S5125”,”S5126”).

An indicator for whether the person received case management services during the measure period. Case management Any use of case management services

services were defined using HCPCS (procedure) codes in: (“T1016”,”T1017”,”T2022”,”T2023”)

Note: The Clinical Classifications Software (CCS) for ICD-9-CM and ICD-10-CM is a diagnosis and procedure categorization scheme that can be employed in many types of projects analyzing data on diagnoses and procedures. ICD-9 and ICD-10 diagnosis and procedure codes are collapsed into a smaller number of clinically meaningful categories ICD-9: https://www.hcup-us.ahrq.gov/toolssoftware/ccs/ccs.jsp. ICD-10: https://www.hcup-us.ahrq.gov/toolssoftware/ccs10/ccs10.jsp.

B.3.6 Collecting Medicaid Administrative Data

3.

for the Non-HUD Comparison Group

For Medicaid and Medicare beneficiaries in all six

study states, we calculated the prevalence of each

From the state Medicaid agencies, we requested

CCS category in the PRA sample and the non-

individual-level Medicaid FFS claims, managed care

HUD sample, respectively. We also calculated the

encounter, and enrol ment data from the six study states

prevalence of CCS categories in the PRA and non-

for PRA and PRAC residents and residents of other

HUD samples of Medicaid and Medicare beneficiaries

HUD-assisted housing, as well as individuals not in

separately for each study state.

HUD-assisted housing who have characteristics identified

4. Using the six-state and single-state samples and CCS

as predictive of PRA participation. The characteristics

indicators, we identified 20 to 30 CCS categories that

identified as predictive of PRA participation were identified

were the most predictive of PRA assistance relative

using the data from ResDAC. We analyzed the pattern

to the non-HUD sample. To do this, we implemented

of diagnosis codes in the 2012–2013 Medicaid data

the Friedman, Hastie, and Tibshirani (2010) coordinate

and 2014–2015 Medicare data to identify clusters of

descent algorithm for elastic net regression in Stata

characteristics that strongly predict participation in the

14 using the module, elasticregress.

PRA.

5. We performed Step 4 using both CCS diagnosis

We took the fol owing steps to identify these clusters:

categories and any available procedure codes (for

1. ICD-9 diagnosis codes, from every diagnosis field

example, ICD-9, Current Procedures Terminology

on the inpatient and outpatient FFS claims and

(CPT) codes) identified on inpatient and outpatient

managed care encounter records in the 2012–2013

claims and encounter records in the Medicaid

Medicaid administrative data, were col apsed into

administrative data that was procured directly from

more clinical y meaningful categories using the

the states and linked to residents of PRA, PRAC, and

U.S. Agency for Healthcare Research and Quality’s

other HUD-assisted housing.

Clinical Classifications Software (CCS).88 We did this

6. For each state, we deemed CCS diagnosis categories

separately using ICD-9 and ICD-10 diagnosis codes

to be strongly predictive if the fol owing criteria were

on the inpatient and outpatient FFS claims in the

met:

2014–2015 Medicare administrative data.

• The CCS category was not related to acute

2. For every Medicaid beneficiary, we created indicators

diagnoses that are uninformative regarding a

for each CCS category equal to 1 if the beneficiary

potential y underlying condition (for example,

ever had an inpatient or outpatient claim in 2012–2013

tonsil itis, influenza), minor injuries (for example,

with a diagnosis fal ing in a given CCS category, and

sprains, superficial injuries), or routine care

0 otherwise. We did the same for every Medicare

(for example, immunizations, screenings,

beneficiary in the 2014–2015 Medicare claims data.

examinations, after-surgery care); AND

88 The Clinical Classifications Software (CCS) for ICD-9-CM is a diagnosis and procedure categorization scheme that can be employed in many types of projects analyzing data on diagnoses and procedures. CCS is based on the International Classification of Diseases, 9th Revision, Clinical Modification (ICD-9-CM), and a uniform and standardized coding system. The ICD-9-CM’s multitude of codes–over 14,000 diagnosis codes and 3,900 procedure codes–are collapsed into a smaller number of clinically meaningful categories that are sometimes more useful for presenting descriptive statistics than are individual ICD-9-CM codes (https://www.hcup-us.ahrq.gov/toolssoftware/ccs/ccs.jsp). CCS has also been created for ICD-10 (https://www.hcup-us.ahrq.gov/toolssoftware/ccs10/ccs10.jsp).

140

H U D S e c t i o n 811 P R A P ro g r a m – P h a s e I I E v a l u a t i o n

Appendix B: Section 811 PRA Phase II Evaluation Methods

• The CCS category was the most predictive

We deem medical procedures to be strongly predictive

of PRA in both of the six-state samples of the

if they were not uninformative (for example, emergency

Medicaid and Medicare administrative data; OR

room visit) or routine procedures (for example, radiology)

• There was greater than one standard deviation

AND they were predictive of PRA relative to residents of

difference in the prevalence of the CCS category

PRAC and other HUD-assisted housing that were linked

between the PRA group and the non-HUD group

to the Medicaid administrative data that was procured

in both of the six-state samples of the Medicaid

directly from the state.

and Medicare administrative data; OR

Exhibit B.3.13 presents the CCS diagnosis categories

• The CCS category was prevalent in at least 20

that meet the predictive criteria. The first row lists 13 CCS

percent of PRA residents in the single-state

diagnosis categories that meet the predictive criteria

sample of Medicaid administrative data; AND

based on comparisons of PRA residents to the non-HUD

group in the six-state samples of the 2012–2013 Medicaid

• The category was the most predictive of

and 2014–2015 Medicare administrative data. The other

PRA in the single-state sample of Medicaid

CCS diagnosis categories that meet the predictive criteria

administrative data; OR

based on comparisons of PRA residents and the non-HUD

• There was greater than one standard deviation

group in each of the single-states samples of the 2012–

difference in the prevalence of the CCS category

2013 Medicaid and 2014–2015 Medicare administrative

between the PRA group and the non-HUD

data are listed in the next six rows.

group in the single-state sample of Medicaid

administrative data.

E xhibit B.3.13 : C C S Diagnosis C ategories that Meet the Predictive Criteria

Sample

Clinical Classification System (CCS) Diagnosis Categories

• 95: Other nervous system disorders

• 155: Other gastrointestinal disorders

• 204: Other non-traumatic joint disorders

• 205: Spondylosis, intervertebral disc disorders, or other back problems

• 211: Other connective tissue disease

Medicare and Medicaid:

• 251: Abdominal pain

All six study states

• 651: Anxiety disorders

(13 categories)

• 652: Attention-deficit, conduct, and disruptive behavior disorders

• 657: Mood disorders

• 659: Schizophrenia and other psychotic disorders

• 661: Substance-related disorders

• 662: Suicide and intentional self-inflicted injury

• 663: Screening and history of mental health and substance abuse codes

• 59: Deficiency and other anemia

• 89: Blindness and vision effects

• 106: Cardiac dysrhythmias

California Medicaid

• 117: Other circulatory disease

(8 categories)

• 133: Other lower respiratory disease

• 159: Urinary tract infections

• 255: Administrative/social admission

• 254: Rehabilitation care; fitting of prostheses; and adjustment of devices

( cont )

141

H U D S e c t i o n 811 P R A P ro g r a m – P h a s e I I E v a l u a t i o n

Appendix B: Section 811 PRA Phase II Evaluation Methods

E xhibit B.3.13 : C C S Diagnosis C ategories that Meet the Predictive Criteria ( cont )

Sample

Clinical Classification System (CCS) Diagnosis Categories

Delaware Medicaid

• 50: Diabetes mellitus with complications

(2 categories)

• 133: Other lower respiratory disease

• 7: Viral infection

• 83: Epilepsy, convulsions

• 89: Blindness and vision effects

• 125: Acute Bronchitis

Louisiana Medicaid

• 126: Other upper respiratory infections

(9 categories)

• 136: Disorders of teeth and jaw

• 159: Urinary tract infections

• 654: Developmental disorders

• 670: Miscellaneous mental health disorders

• 4: Mycoses

• 7: Viral infection

• 50: Diabetes mellitus with complications

• 52: Nutritional deficiencies

• 53: Disorders of lipid metabolism

• 58: Other nutritional; endocrine; and metabolic disorders

Maryland Medicaid

• 59: Deficiency and other anemia

(13 categories)

• 117: Other circulatory disease

• 133: Other lower respiratory disease

• 159: Urinary Tract Infections

• 163: Genitourinary symptoms and ill-defined conditions

• 203: Osteoarthritis

• 255: Administrative/social admission

• 117: Other circulatory disease

• 133: Other lower respiratory disease

Minnesota Medicaid

• 255: Administrative/social admission

(5 categories)

• 650: Adjustment disorders

• 660: Alcohol-related disorders

• 53: Disorders of lipid metabolism

• 58: Other nutritional; endocrine; and metabolic disorders

• 89: Blindness and vision effects

• 106: Cardiac dysrhythmias

Washington Medicaid

• 126: Other upper respiratory infections

(9 categories)

• 127: Chronic obstructive pulmonary disease and bronchiectasis

• 133: Other lower respiratory disease

• 159: Urinary Tract Infections

• 163: Genitourinary symptoms and ill-defined conditions

Note: The Clinical Classifications Software (CCS) for ICD-9-CM and ICD-10-CM is a diagnosis and procedure categorization scheme that can be employed in many types of projects analyzing data on diagnoses and procedures. ICD-9 and ICD-10 diagnosis and procedure codes are collapsed into a smaller number of clinically meaningful categories ICD-9: https://www.hcup-us.ahrq.gov/toolssoftware/ccs/ccs.jsp. ICD-10: https://www.hcup-us.ahrq.gov/toolssoftware/ccs10/ccs10.jsp.

142

H U D S e c t i o n 811 P R A P ro g r a m – P h a s e I I E v a l u a t i o n

Appendix B: Section 811 PRA Phase II Evaluation Methods

B.4 Approach to Economic Study

The linkage also implies that costs associated with

healthcare and disability-related services may differ

The description of the approach to the economic study

across PRA, PRAC, and the other programs and should

includes an overview of the cost structures of the PRA

be considered by policymakers. Although the PRA and

and comparison programs, catalogs data sources used

PRAC programs each have strategies to connect people

for the cost and cost al ocation analyses, and details other

to services in ways that are expected to reduce healthcare

aspects of the economic study methods.

costs, costs of other services and supports may increase.

B.4.1 Cost Structures of HUD Programs Serving

For example, the 811 PRA program might shift costs

Adults with Disabilities

from one domain (for example, healthcare and disability

support services) to another (for example, housing or

Cost structures of the PRA and PRAC, NED and

transportation). In addition, the different administrative

other HUD programs are the result of the programs

structure for the PRA program relative to the PRAC,

requirements and implementation infrastructure. While

NED, and other HUD programs could result in different

the PRA program does not directly fund healthcare and

administrative costs per assisted participant.

disability-related services, housing quality and location

can affect both health and access to healthcare and

B.4.2 Cost Analysis Data Sources

supportive services. Conversely, access to healthcare

Exhibit B.4.1 details the data sources we use for each

and supportive services may enable individuals to live

element of the cost and cost al ocation analyses. Data

successfully in the community. This interdependency

sources are organized into each of the four cost areas:

motivates the requirement that PRA program grantees

housing rental subsidies, housing capital subsidies,

(state agencies that administer housing programs) execute

healthcare and supportive services, and program

an inter-agency partnership agreement with the state

administration.

agency that administers Medicaid.

E xhibit B.4.1: C ost A nalysis Dat a S ources

PRA

PRAC

NED and other HUD

Cost element

Data sources

Data sources

Data sources

Housing rental subsidies

• Rent amounts in TRACS data

• Rent amounts in PIC

Monthly rental subsidies

• Interviews and quarterly report

• Rent amounts in TRACS data

and TRACs data

information on vacancy

• Projected average rental subsidy:

Grantee S811 quarterly reports

Reference comparison for

• Contract rent: TRACS data

• Not Applicable

• Not Applicable

subsidy amounts

• FMR/SAFMR: public data

on HUD’s website

Housing Capital Subsidies

• Grant details from iREMS database

• LIHTC cost certifications and

• Review of sample of PRAC grant

Capital subsidy detail

applications from state housing

applications (29 properties)

• Not included in analysis

agencies or public reporting

• Annual Financial Statements from

OPI S database

( cont )

143

H U D S e c t i o n 811 P R A P ro g r a m – P h a s e I I E v a l u a t i o n

Appendix B: Section 811 PRA Phase II Evaluation Methods

E xhibit B.4.1: C ost A nalysis Dat a S ources ( cont )

Program Implementation and Administrative Costs

• Not Available/Applicable after review

of the following:

• Interviews with senior program and

finance staff

à Annual Financial Statements

• Estimates of PHA costs

Grantee and partner state agencies

from OPI S database

from Turnham et al

• Program documents (applications,

(2015)

quarterly budgets, other reporting)

à Websites, 990 tax forms, and

other available documents for a

sample of grantee organizations

• Interviews & documents

HUD (headquarters and field offices)

• Interviews and documents from HUD

• Interviews & documents from HUD

from HUD

Healthcare and Supportive Services Costs

Healthcare utilization

• Inpatient admissions

• States’ Medicaid

• Emergency room visits

• States’ Medicaid claims/

• States’ Medicaid claims/

claims/administrative

administrative data

administrative data

• Medical transportation

data

• Long-term inpatient care

• Total HCBS costs: Medicare Part

A and B

• Total HCBS costs:

• Total HCBS costs: Medicare Part

Supportive services utilization

Medicare Part A and

• Case management use: States’

A and B

Part B claims

Medicaid claims/administrative data

B.4.3 Cost Allocation Analyses

housing and administration to report a total housing

The costs and cost al ocation analyses develop per-

program cost.

resident, monthly, or annual cost estimates in four areas:

To make the most of the available information on both

housing rental subsidies, housing capital subsidies,

rental and capital subsidies, we use multiple analysis

healthcare and supportive services, and program

samples in our housing cost analyses. Exhibit B.4.2

administration. We then summed annualize costs for

provides sample sizes of individuals and properties for

each of the rental and capital subsidy analysis samples.

E xhibit B.4.2 : Housing Subsidy A nalysis S amples

Analysis Sample

Number of Individuals

Number of Properties

Rental Subsidies

PRA rental subsidy

540

58

PRAC rental subsidy

3,191

451

NED rental subsidy

8,859

NA

Other HUD rental subsidy

145,992

NA

Data

PRA capital subsidies

408

41

PRAC Sample 1: capital grant only

3,170

440

PRAC Sample 2: all sources of capital subsidy

247

29

NA = not applicable.

Note: Research and data collection timelines required us to fix the sample for which we collected PRA capital subsidy data at an earlier time than the entire rental sample. At that time, we sought information for 43 properties that include 420 of the 540 PRA residents in the final analysis sample.

144

H U D S e c t i o n 811 P R A P ro g r a m – P h a s e I I E v a l u a t i o n

Appendix B: Section 811 PRA Phase II Evaluation Methods

Housing Rental Subsidies

identified by grantees that we subsequently matched to

Ongoing rental subsidies are paid monthly to property

HUD administrative data. We selected properties that

owners in the PRA program and each of the programs

had occupied units as of March 2018. The 41 properties

serving our comparison groups. We calculate average

include 97 percent of individuals receiving assistance

rental subsidies directly from HUD administrative data

through PRA in these 43 developments. For these

for all individuals in PRA, PRAC, NED, and other HUD

properties, we acquired cost certifications directly from

programs (public housing, housing choice voucher, and

state agencies where possible, and from websites with

multifamily housing programs). Rental subsidy amounts

information on cost certifications or LIHTC applications

for the PRA and PRAC programs are included in TRACS

where details were not available from the relevant

data, while amounts for NED and other HUD programs are

administering agency.90

from IMS/PIC data.

PRAC Capital Subsidies

Because we observe rental subsidy costs directly at

Identifying capital development subsidies provided

the individual level using HUD administrative data,

directly by the PRAC program is straightforward. The

computing average per-individual rental subsidy costs is

PRAC program capital grant itself is recorded in iREMS.

straightforward. We average observed subsidy amounts

As shown in Exhibit B.4.2, our analysis includes a sample

for all individuals receiving assistance as of March 2018.

of 3,170 individuals in 440 properties for which we have

Rental subsidy detail is available for almost all individuals

both PRAC rental subsidy and PRAC capital grant

in our analysis sample. We compute the averages using

information. PRAC properties may also receive additional

analysis weights derived using the same propensity-score

sources of capital funding (PRAC Sample 1). To assist with

matching approach used in the impact analysis. For the

this evaluation, HUD staff also located digitized archived

cost al ocation analysis, all rental subsidies we observe

images of PRAC grantee applications that included detail

are borne by HUD.89

on all capital sources for 29 PRAC properties (PRAC

Sample 2). PRAC Sample 2 is a subset of properties

Housing Capital Subsidies

from a targeted list of PRAC properties provided to HUD

Estimating the annualized, per-unit value of capital

that, based on our propensity score model, had a high

subsidies is a more involved undertaking. Complicating

incidence of residents that were similar to PRA residents.

this estimation is the methodological challenge of

These 29 properties assist 247 PRAC residents.

apportioning a single development subsidy over many

Complete capital subsidy data is only available for a

units and for the decades over which the property

limited share of our PRAC comparison sample, while

will provide below-market rents to eligible residents.

incomplete capital subsidy detail is available for almost al

Estimating the value of capital subsidies for the properties

individuals in our PRAC comparison sample. In each case,

in our analysis is also complicated by data availability.

we provide subsidy cost estimates for the largest sample

There is no central repository of capital subsidies provided

of individuals for which data is available.

to low-income housing developments, necessitating

primary data col ection effort. For this reason, capital

Estimating Per-Unit Annual Capital

subsidy analysis for properties housing NED and other

Subsidy Costs for PRA and PRAC

HUD-assisted individuals in our comparison groups is

Capital subsidies for PRA and PRAC are awarded for

beyond the scope of this study.

an entire development when it is built (or substantial y

PRA Capital Subsidies

remodeled). To convert this initial amount into a per-unit,

annual cost, the value of the initial subsidy must be both

The study team col ected detail on capital subsidies for

apportioned among units that vary in size and amortized

developments that had received LIHTC capital funding for

over the period for which the property provides low-

properties with any occupied PRA units in them as of the

income rental assistance.

summer of 2017. As reported in Exhibit B.4.2, we found

capital subsidy information for 41 of the 43 developments

We make necessary and standard assumptions to

that had any occupied PRA units as of our cutoff date.

estimate per-unit annualized costs. The capital subsidy is

We created a list of such properties from properties

annualized over the time period for which the respective

89 State and local governments also administer rental assistance programs. These programs typically serve individuals that are eligible for federal subsidies but do not receive them because the federal program is already fully subscribed in their area. We are not aware of any state and local programs that provide additional assistance to recipients also receiving HUD-funded assistance.

90 It is possible that the two properties for which we did not find any information received LIHTC or other federal, state, or local subsidies that we did not discover in our data collection efforts. We searched HUD administrative data and the internet but did not find any additional information for these properties. We dropped them from our analysis rather than assume they receive no subsidies.

145

H U D S e c t i o n 811 P R A P ro g r a m – P h a s e I I E v a l u a t i o n

Appendix B: Section 811 PRA Phase II Evaluation Methods

capital subsidy program requires affordability restrictions

affected by housing quality and location. The measures

(for example, 30 years for LIHTC funding and 40 years for

that we calculated and reported are listed in Exhibit B.3.2.

PRAC grants). We value the entire amount of grants and

Each measure incorporates a measure from the impact

forgivable loans as a subsidy and estimate the value of

analysis of healthcare utilization and a unit cost measure

assistance provided as low-interest, but repayable, loans

developed from the available data.

as the implied lifetime savings relative to a market-rate

We estimated the difference in costs of all disability

loan provided by the lower rate. We use a 3.5-percent

services received through HCBS between PRA and each

discount rate to calculate net present values and to

comparison program by multiplying the impact estimate

amortize these values to annual amounts over the period

for use of PCA services by the average total cost of HCBS

that the property provides below-market rents to eligible

claims observed in 2011 ResDAC data, adjusted for

residents. We calculate the net present value at the time of

inflation.

development of the subsidy provided by low-interest loans

over the life of the loan and add to any non-repayable

For our healthcare utilization quantity measure, we used

lump-sum subsidies to determine the net present value of

the average fee-for-service claim dollar amount in each

all subsidies at the time of development. As a robustness

state for the respective unit of utilization. This amount is

check, we considered 2-percent and 6-percent discount

only observable for the subset of individuals receiving

rates, which did not qualitatively alter our conclusions. To

healthcare in a fee-for-service model, but it is our best

al ocate the total development subsidy over units that vary

available estimate of the cost of the unit of utilization.

in bedroom size, we created an adjustment factor based

The average state fee-for-service costs represent the

on the ratio of FMRs for each bedroom size.91

average costs for PRA residents and individuals in the

For the cost allocation analysis, we identified whether

comparison groups enrol ed in Medicaid in 2015 or 2016,

a given capital subsidy was funded by HUD, the LIHTC

and either matched to individuals in the state Medicaid

program, a state or local affordable housing program, or

agencies database of enrollees, or included in our sample

by some other federal program or private philanthropic

of individuals in the non-HUD comparison group. Fee-for-

source. After calculating average annual rental subsidies

service costs for similar services (for example, inpatient

(monthly amounts multiplied by 12) and estimating the

days) can vary substantial y according to diagnoses,

annualized value of capital subsidies, we added the

patient acuity, and treatments provided. Furthermore, only

amounts together to arrive at our total cost of housing

four of the six states provided information on payments to

subsidies for the PRA and PRAC comparison groups.

the provider from managed care organizations. Therefore,

The sample for these sums is necessarily limited to the

we were limited to estimating average fee-for-service

individuals for which we have both rental and capital

costs for each service in each state.

subsidy detail.

The proportion of Medicaid enrol ees in fee-for-service or

managed care plans varies across states. There is also

Costs of Healthcare and Disability

variation in the types of coverage and levels of payments

Services Utilization

for specific services; the types of waivers offered to

Our analysis of the cost of healthcare and disability

disabled individuals; the extent of services covered under

services utilization primarily builds off the analysis of state

those waivers; the amount of payments covered for dual-

Medicaid data from the impact analysis, with additional

enrol ees; and various other details of Medicaid payment

contextual information about the funding and availability

policies determined by state Medicaid agencies. As a

of services from our qualitative data col ection and

result, the average costs estimated for each service may

supporting research.

not reflect the true population averages across the six

We compared cost estimates of the healthcare and

states and could be skewed due to chance variation in the

disability service utilization data for PRA and our

use of healthcare services by small samples of individuals

comparison groups. Because of data limitations, we do

in each state, during a 2-year period.

not have complete data on costs associated with any

healthcare or disability-related services used. Rather, we

focused on measures that were available and likely to be

91 Rather than multiply the total subsidy amount by

, we multiply by

, which is the FMR-bedroom size weighted

rental revenue share anticipated by a unit with beds=0,1 ,2,3 or 4 bedrooms given there are N

units of each bedroom size in the property and

beds

the bedroom-size FMR is FMRbeds

146

H U D S e c t i o n 811 P R A P ro g r a m – P h a s e I I E v a l u a t i o n

Appendix B: Section 811 PRA Phase II Evaluation Methods

E xhibit B.4.3 : Healthcare U tilization and HC B S C ost A nalysis Measures

Total Per-Individual

Measure

Impact Quality Measure

Unit Cost Measure

Annual Cost Estimate

% difference in PCA use

Disability-related services through

Impact estimate for use of personal care

Average annual total 2011 HCBS Claims

Average annual total 2011

HCBS(PRA vs PRAC difference only)

attendant (PCA) services

in ResDAC

HCBS Claims in ResDAC

Average observed FFS emergency room

Healthcare utilization

Emergency room visits per quarter*4

Visits*Average cost

visit cost by state

Average observed FFS inpatient day cost

Inpatient days per quarter*4

Days*Average cost

by state

Average observed FFS long-term care day

Long-term care days per quarter*4

Days*Average cost

cost by state

Average observed FFS transport trip cost

Transport trips per quarter*4

Trip*Average cost

by state

Notes: We have no direct measure of total costs of disability-related services use or costs of PCA use after entry into PRA. We impute a PRA vs PRAC differential cost in total disability-related services use by multiplying the percentage difference in PCA use we measure in the impact analysis by the total average annual cost of HCBS use we observe in 2011 ResDAC data.

To supplement the quantitative data, we reviewed

Costs of Program Administration

qualitative data to assess the potential for healthcare

We estimated average, per-individual annual costs of

and services utilization that is not captured by Medicaid

program administration for the PRA program and drew

data. We reviewed the variety of data we col ected for the

comparisons to estimates for PRAC and NED and other

study through the lens of possible costs of healthcare and

HUD programs. Our estimates of administrative costs for

service utilization by PRA residents that was not ultimately

the PRA program come from data col ected from PRA

funded by Medicare and Medicaid (and thus visible in our

program grantees and their state agency partners through

prior analysis). This included reviewing notes from our

a written cost and effort survey and from consultation

interviews with service providers, grantees and health

with HUD staff. Costs for the PRAC programs come from

agency partners, and property owners, and reviewing

consultation with HUD staff, and from review of PRAC

responses to relevant questions from our resident survey.

grantee annual financial report data and publicly available

We also scanned PRA rental agreement contracts and

information for a limited number of properties. (We did

grantee’s applications and partnership agreements for

not find information in these resources that informed our

clues of any such services.

administrative cost estimates.) Our estimates for program

For the PRAC population, we similarly noted responses

administration costs for NED and other HUD programs

to relevant resident survey questions and reviewed any

are based on prior research around administrative

relevant administrative interviews (that is, HUD field office

costs for the PHAs that administer these programs and

interviews). In addition, we conducted a review of available

in consultation with experts that conducted this prior

public information around 15 PRAC properties that housed

research (Turnham et al., 2015).

a total of 170 residents, including 21 of the individuals

that we surveyed. For these properties, we reviewed the

sponsoring organization’s website (if any) and 501c(3) tax

filings.

147

H U D S e c t i o n 811 P R A P ro g r a m – P h a s e I I E v a l u a t i o n

148

H U D S e c t i o n 811 P R A P ro g r a m – P h a s e I I E v a l u a t i o n

References

References

Emsley, Richard, et al. 2008. “Implementing Double-Robust Estimators of Causal Effects,” The Stata Journal 8 (3): 334–353.

Finkel, Meryl, Samuel Dastrup, Kimberly Burnett, Thyria Alvarez, Carissa Climaco, Tanya de Sousa. 2016. Small Area Fair Market Rent Demonstration Evaluation Interim Report. U.S. Department of Housing and Urban Development, Office of Policy Development and Research.

Friedman, J., T. Hastie, and R. Tibshirani. 2010. “Regularization Paths for Generalized Linear Models via Coordinate Descent,”

Journal of Statistical Software 33 (1): 1–22.

Galvez, Martha. 2010. What Do We Know About Housing Choice Voucher Program Location Outcomes? A Review of Recent Literature. Urban Institute: Washington, DC.

Glickman, M.E., S.R. Rao, and M.R. Schultz. 2014. “False Discovery Rate Control is Recommended Alternative to Bonferroni-Type Adjustments in Health Studies,” Journal of Clinical Epidemiology 67: 850–857.

McClure, Kirk. 2008. “Deconcentrating Poverty with Housing Programs,” Journal of the American Planning Association 74: 90–99.

Nichols, Austin. 2007. “Causal Inference with Observational Data,” The Stata Journal 7 (4): 507–541.

Turnham, Jennifer, Meryl Finkel, Larry Buron, Melissa Vandawalker, Bulbul Kaul, Kevin Hathaway, Chris Kubacki. 2015. Housing Choice Voucher Program Administrative Fee Study Final Report. U.S. Department of Housing and Urban Development, Office of Policy Development and Research. https://www.huduser.gov/publications/pdf/AdminFeeStudy_2015.pdf.

149

H U D S e c t i o n 811 P R A P ro g r a m – P h a s e I I E v a l u a t i o n

U.S. Department of Housing and Urban Development

Office of Policy Development and Research

Washington, DC 20410-6000

February 2020

Document Outline

- Contents

- List of Exhibits

- Executive Summary

- About the PRA Program Evaluation and This Report

- 1.1 Section 811 PRA Grant Requirements

- 1.2 National Status of PRA Program Implementation

- Status of PRA Grants as of September 2018

- 1.3 The Evaluation of the Section 811 PRA Program

- Phase I Evaluation (2014-2016)

- Phase II Evaluation (2016-2019)

- 1.4 The Phase II Evaluation Design

- Phase II Evaluation Objectives, Research Questions, and Data Sources

- The Evaluation’s Comparison Groups

- Limitations of the Evaluation’s Design

- 1.5 Structure of This Report

- PRA Programs in the Six Study States

- 2.1 PRA Grants in the Study States

- 2.2 State PRA Target Populations

- 2.3 PRA Program Profiles in the Six Study States

- California

- Delaware

- Louisiana

- Maryland

- Minnesota

- Washington

- 2.4 PRA Program Status in the Study States

- Status of Units under Contract and under Lease

- PRA Residents Assisted During 2015-2018

- Duration of PRA Resident Tenancy

- Chapters 3-7: Early Evidence from the PRA Program

- Demographic Characteristics of PRA Residents and Comparison Groups

- 3.1 Household and Demographic Characteristics

- Household Size

- Age

- Race, Ethnicity, and Gender

- Household Income

- Neighborhood Characteristics for PRA Residents and the Comparison Groups

- 4.1 Approach to Comparison of Neighborhood Characteristics

- 4.2 Neighborhood Demographic and Household Characteristics

- Residential Density and Urban/Rural Classification

- Percentage of Residents Who Report a Disability

- Education and Income

- Racially and Ethnically Concentrated Areas of Poverty Neighborhoods

- Neighborhood Livability Indicators

- 4.3 Access to Services and Transportation

- Resident Perception of Neighborhood

- Service Provider Perceptions of Neighborhoods Where PRA Units Are Available

- PRA and PRAC Property Characteristics

- 5.1 Section 811 Property Characteristics

- Building Type

- Property Size

- Age and Development Status of PRA Units under Contract

- For-Profit Status of Owner

- 5.2 Residents’ Satisfaction with Their Housing

- 5.3 Physical Quality of PRA Units

- PRA Resident Views on the Physical Quality of Their Housing

- Maintenance Issues Since Resident Moved into Unit

- Program Partner Perceptions of the Quality of PRA Units

- 5.4 Unit Accessibility Features and Needs of Residents

- Reported Accessibility Needs of PRA Residents

- 5.5 Integration of PRA Units within Properties

- Resident Perspectives on Community Integration and Independence

- PRA and PRAC Community-Based Services

- 6.1 Availability and Use of Community-Based Services

- Housing Locator Services

- Transition Services

- Ongoing Tenancy Supports

- 6.2 Resident Experience with Community-based Services and Alignment with Needs

- Resident Experience with Community-based Services

- Quality of Life and Health Status

- 6.3 Housing Tenancy and Program Exits

- Healthcare Conditions and Utilization

- 7.1 Healthcare Conditions and Prior Healthcare Utilization

- Chronic and Potentially Disabling Conditions

- 7.2 PRA Impacts on Healthcare Utilization

- Prior Healthcare Utilization of PRA Residents and the Comparison Groups

- Early PRA Tenancy Findings

- 7.3 Comparison of Healthcare Utilization for PRA Residents and Each Comparison Group

- PRA and PRAC Comparison

- PRA and NED Comparison

- PRA and Other HUD Comparison

- PRA and Non-HUD Comparison

- Summary

- Economic Study of the PRA Program

- 8.1 Analyzing Costs and Cost-Effectiveness of PRA and Comparison Programs

- Approach to Cost and Cost Allocation Analyses of PRA and Comparison Programs

- Limitations of the Economic Study

- 8.2 Costs of Rental Subsidies

- Average Monthly Per-Unit Rental Subsidies in the PRA Program

- Comparison of Actual and Estimated Average Rental Subsidies

- Comparison of PRA Contract Rent to Fair Market Rent

- Comparison of PRA, PRAC, NED, and Other HUD Monthly Per-Unit Rental Subsidies

- 8.3 Capital Subsidy Costs

- Sources of Capital Financing for PRA and PRAC Properties

- Comparison of Combined Capital and Rent Subsidies in the PRA and PRAC Programs

- 8.4 Costs of Healthcare and Disability-related Services Utilization

- Sources of Assistance for Healthcare and Disability Services for PRA Residents

- Costs of Relevant Healthcare and Disability-Related Services Assistance through Medicaid

- 8.5 Costs of Program Administration

- PRA Program Staffing in State Housing and Health Agencies

- Estimated Annual PRA Program Administrative Costs

- Sources of Funding for PRA Program Administration

- Costs of HUD’s Administration of the PRA Program and Administrative Costs in Comparison Programs

- 8.6 Total Estimated Costs and Cost Effectiveness

- Summary of PRA Program and Comparison Group Costs

- Cost-Effectiveness of the PRA Program

- PRA Program Practices that May Lead to Successful Results

- 9.1 Strategies for Selecting and Prioritizing Target Populations

- Results of Selection and Prioritization of PRA Target Populations

- 9.2 Removing Barriers to Applicant Eligibility for PRA

- Applicant Eligibility Rates by State and Over Time

- Applicants Found Ineligible for the PRA Program

- Applicants Found Ineligible for the Properties

- Strategies Used to Remove Barriers to Eligibility

- 9.3 Identifying and Securing Quality, Cost-Effective PRA Units

- Percent of Estimated PRA Units Under Contract and Under Lease

- Study State Adaptations to Secure More PRA Units

- 9.4 Ensuring PRA Residents Receive Tenancy Supports They Need

- Challenges in Coordinating Tenancy Supports for PRA Residents

- Strategies for Coordinating Tenancy Supports for PRA Residents

- 9.5 Developing Effective and Sustainable PRA Partnerships

- How State Agencies Define Successful PRA Partnerships

- Sustaining PRA Partnerships

- Expanding Beyond the PRA Program Partnership

- 9.6 Summary

- Conclusion on the Overall Effectiveness of the PRA Program

- Increasing the Supply of Affordable Housing for People with Disabilities: PRA grantees have secured units for the program, but at a slower pace than HUD had planned

- Expanding Housing Opportunities for People with Significant Healthcare and Service Needs: The PRA program is housing its intended target populations

- Expanding Housing Opportunities in Integrated Settings: The evidence is mixed

- Building Institutional Capacity at the State Level: PRA grantees established partnerships at the state and local levels, often building on existing relationships from previous supportive housing efforts, but at considerable administrative cost

- 10.2 Policy Implications and Areas for Future Research

- Program Monitoring and Data Sharing

- Greater Flexibility in Identifying PRA Units That Meet Resident Needs and Preferences

- Sustaining and Expanding the PRA Partnerships

- Is PRA an Effective Approach?

- Areas for Future Research

- Appendix A: Status of National PRA Program Implementation as of September 2018

- A.1 Overview of Section 811 PRA Grant Awards

- A.2 Status of Units under Agreement for the PRA Program

- A.3 Status of PRA Grants by State and by Grant Year

- A.4 Characteristics of PRA Units under RAC

- A.5 Status of Applicant Referrals and Lease-Ups

- A.6 Characteristics of Households That Moved into PRA Units

- A.7 Exits from PRA Units

- A.8 Housing Choice Vouchers Leveraged

- Appendix B: Section 811 PRA Phase II Evaluation Methods

- B.1 Overview of the Section 811 PRA Phase II Evaluation

- B.2 Evaluation Data Sources

- Integrated Real Estate Management System (iREMS)

- Tenant Rental Assistance Certification System

- Inventory Management System / PIH Information Center

- Online Property Integrated Information Suite

- Section 811 PRA Quarterly Reports

- CMS Research Data Assistance Center

- State Medicaid Data

- B.3 The Analytic Sample for the Impact and Economic Studies

- Multi-Stage Process for Constructing the Analytic Sample

- Three Analytic Samples

- Descriptive Demographic Characteristics of the Study Groups

- Adjustments to Hypothesis Tests due to Multiple Comparisons

- B.4 Approach to Economic Study

- Housing Capital Subsidies

- Costs of Healthcare and Disability Services Utilization

- Costs of Program Administration

- References

- Exhibit 1.1: Evaluation Objectives, Research Questions, and Data Sources

- Exhibit 2.1: PRA Grant Amounts and Planned Units in the Study States, by State and Grant Year

- Exhibit 1.2: Summary of Program Features for HUD Comparison Programs

- Exhibit 2.4: PRA-Assisted Residents in the Study States by Previous Living Situation, September 2018

- Exhibit 2.3: Status of Planned 811 PRA Units in the Study States, September 2018

- Exhibit 2.2: Target Population of Section 811 PRA Programs

- Exhibit 3.1: Household and Bedroom Size of PRA, PRAC, NED, and Other HUD-Assisted Households in the Study States, March 2018

- Exhibit 3.2: Demographic and Socioeconomic Characteristics of PRA, PRAC, NED, and Other HUD-Assisted Households in the Study States, March 2018

- Exhibit 3.3: Household Income of PRA, PRAC, NED, and Other HUD-Assisted Households in the Study States, March 2018

- Exhibit 3.4: Sources of Household Income of PRA, PRAC, NED, and Other HUD-Assisted Households in the Study States, March 2018

- Exhibit 4.1: Number of Households, Properties, and Census Tracts in Neighborhood Analysis, by Study Group in the Study States, March 2018

- Exhibit 4.2: Neighborhood Characteristics of HUD-Assisted Households in Study Groups, March 2018

- Exhibit 4.3: Neighborhood Demographic Characteristics of HUD-Assisted Households in Study Groups, March 2018

- Exhibit 4.4: Comparison of Neighborhood Livability Indicators of HUD-Assisted Households in Study Groups, March 2018

- Exhibit 4.5: PRA and PRAC Neighborhood Accessibility Measures

- Exhibit 4.6: Section 811 PRA and PRAC Neighborhood Measures

- Exhibit 5.2: Size of Properties with PRA Units and PRAC Units in the Study States, September 2018

- Exhibit 5.1: Section 811 PRA Building Types for Occupied Units in the Study States, September 2018

- Exhibit 5.3: Year of Construction or Last Rehabilitation for PRA and PRAC Properties in the Study States, September 2018

- Exhibit 5.4: Comparison of Section 811 PRA and PRAC Resident Satisfaction with Housing

- Exhibit 9.1: Grantee-Reported Living Situation of PRA Applicants and Residents, September 2018

- Exhibit 8.11: Per-Unit Annual Cost Estimates for PRA and Comparison Programs

- Exhibit 8.10: Section 811 PRA Grant Administrative Fees Awarded and Received, Program Years 2015-2018

- Exhibit 8.9: Estimated Annual Grantee and State Agency Partner PRA Program Administrative Costs

- Exhibit 8.8: State Housing and Health Agency Staff Administering the PRA Program, in Full-Time Equivalents

- Exhibit 8.7: Section 811 PRA Program State Agencies in Six States

- Exhibit 8.6: Healthcare and Disability-Related Services Costs for PRA, PRAC, NED, and Other HUD Residents

- Exhibit 8.5: Annualized Per-Unit Total Housing Subsidy for PRA and PRAC

- Exhibit 8.4: Annualized Per-Unit Capital Subsidy for PRA and PRAC

- Exhibit 8.2: Monthly Rental Subsidy for PRAC, NED, and Other HUD

- Exhibit 8.1: Monthly Rental Subsidy and Contract Rent for Units in PRA Program

- Exhibit 7.7: Average Rates of Healthcare Utilization for PRA Residents Relative to Medicaid Enrollees Not Receiving HUD Assistance, 2016

- Exhibit 7.6: Average Rates of Healthcare Utilization by PRA Residents Relative to Recipients of Other HUD Assistance, 2016

- Exhibit 7.5: Average Rates of Healthcare Utilization by PRA Residents Relative to NED Voucher Holders, 2016

- Exhibit 7.4: Average Rates of Healthcare Utilization of PRA Residents Relative to PRAC Residents, 2016

- Exhibit 7.3: Unweighted Comparisons of Healthcare Utilization, 2015 (prior to moving in for PRA residents)

- Exhibit 7.2: Unweighted Comparisons of Chronic and Potentially Disabling Conditions among PRA Residents and Individuals in the Comparison Groups Enrolled in Medicaid in 2015 and 2016

- Exhibit 7.1: Healthcare Utilization Comparison Samples

- Exhibit 6.5: Hazard Ratios for Reasons for Program Exits for PRA and PRAC Residents, 2013–2018

- Exhibit 6.4: Reasons Tenants Had Left PRA Housing as of September 2018

- Exhibit 6.3: Statistically Significant Differences in Reported Health and Quality of Life of Section 811 Residents

- Exhibit 6.2: Use of Tenancy Supports by PRA and PRAC Residents

- Exhibit 6.1: The Three Phases of Community-based Services

- Exhibit 5.8: Section 811 Community Integration Measures

- Exhibit 5.7: Reported Accessibility Needs of Section 811 PRA and PRAC Residents

- Exhibit 5.6: Unresolved and Resolved Maintenance Issues Reported by PRA Residents

- Exhibit 5.5: Section 811 PRA Resident Report of Property Condition

- Exhibit 9.2: Applicants Found Eligible for the PRA Program in the Six Study States, Program Years 2015–2018

- Exhibit 9.3: Reasons Applicants were Found Ineligible for the PRA Program, Program Years 2015–2018

- Exhibit 9.4: Percentage of Applicants That Did Not Meet PRA Program Requirements, Program Years 2015–2018

- Exhibit 9.9: Change in Estimated PRA Units as a Result of Average Expected Increases in Contract Rent

- Exhibit 9.8: Units Identified, Under Contract, and Under Lease for PRA, September 2018

- Exhibit 9.7: Status of Planned Section 811 PRA Units by State, September 2018

- Exhibit 9.6: Percentage of PRA Applicants Whose Lease Applications Were Not Accepted by Owners, Program Years 2015–2018

- Exhibit A.9: Section 811 PRA Units under RAC and under Lease as of September 2018, by Bedroom Size

- Exhibit A.8: Implementation Status of FY13 Grantees as of September 2018, by State

- Exhibit A.7: Implementation Status of FY12 Grantees as of September 2018, by State

- Exhibit A.6: Occupancy Status of Units under RAC as of September 2018, by Grant Year

- Exhibit A.5: Status of ARACs with FY13 Grantees through September 2018, by State

- Exhibit A.4: Status of ARACs with FY12 Grantees through September 2018, by State

- Exhibit A.3: Contracts Executed with FY13 Grantees through September 2018, by State

- Exhibit A.2: RACs Executed with FY12 Grantees through September 2018, by State

- Exhibit A.1: FY12 and FY13 PRA Grant Funding Awards and Planned PRA Units

- Exhibit A.10: Section 811 PRA Units under RAC as of September 2018, by Accessibility

- Exhibit A.13: Estimated and Actual Rental Assistance Amounts through September 2018, by State

- Exhibit A.12: Contract Rent and Rental Assistance Payment for Units under Lease as of September 2018, by Bedroom Size

- Exhibit A.11: Grantee Commitment for Units Rent Levels as of September 2018, by Grant Year and State

- Exhibit A.14: Section 811 PRA Grants Target Populations (FY12 Grantees)

- Exhibit B.4.3: Healthcare Utilization and HCBS Cost Analysis Measures

- Exhibit B.4.2: Housing Subsidy Analysis Samples

- Exhibit B.4.1: Cost Analysis Data Sources

- Exhibit B.3.13: CCS Diagnosis Categories that Meet the Predictive Criteria

- Exhibit B.3.12: Measures Describing Healthcare Utilization by Medicaid Enrollees Using Data Collected from States

- Exhibit B.3.11: Condensed Categories of Chronic and Potentially Disabling Conditions

- Exhibit B.3.10: Cox Hazard Ratio for Program Exits Due to Death

- Exhibit B.3.9: Cox Hazard Ratio for Tenant Initiated Reasons

- Exhibit B.3.8: Cox Hazard Ratio for Owner Initiated Reasons Other Than Nonpayment of Rent

- Exhibit B.3.7: Cox Hazard Ratio for Program Exits Due to Owner Initiated Nonpayment of Rent

- Exhibit B.3.6: Hazard Ratios for Reasons for Program Exits for PRA and PRAC Residents, 2013–2018

- Exhibit B.3.5: Cox Proportional Hazards Regression for Rates of Program Exits for PRA and PRAC Residents—Any Exit Type

- Exhibit B.3.4: Unweighted and Unadjusted Counts of PRA and PRAC Program Exits, 2013–2018

- Exhibit B.3.3: Percentage of HUD Tenants on September 30, 2017, Enrolled in Medicaid in 2012–2013 or Medicare in 2014–2015

- Exhibit B.3.2: Impact Study Research Matrix

- Exhibit B.3.1: Categories of Measured Impacts Compared to PRA

- Exhibit B.2:3: Completed Administrative Interviews for PRA Evaluation

- Exhibit B.2.2: Surveys Completed by State and Type of Property

- Exhibit B.2.1: HUD Administrative Data Sources

- Exhibit A.23: HCVs Leveraged for PRA Properties, by Grant Year and State

- Exhibit A.22: Reasons Tenants Left PRA Housing through September 2018

- Exhibit A.21: Housing Tenure of Current PRA Residents through September 2018

- Exhibit A.20: Previous Living Situation of PRA Residents through September 2018, by Grant Year

- Exhibit A.19: Living Situation of Ineligible Applicants through September 2018, by Grant Year

- Exhibit A.18: Living Situation of Applicants on the Waitlist through September 2018, by Grant Year

- Exhibit A.17: Living Situation of Applicants through September 2018, by Grant Year

- Exhibit A.16: Referral Source of Applicants through September 2018, by Grant Year

- Exhibit A.15: Section 811 PRA Grants Target Populations (FY13 Grantees)

Table of Contents

Contents

List of Exhibits

Executive Summary

About the PRA Program Evaluation and This Report

1.1 Section 811 PRA Grant Requirements

1.2 National Status of PRA Program Implementation

Status of PRA Grants as of September 2018

1.3 The Evaluation of the Section 811 PRA Program

Phase I Evaluation (2014-2016)

Phase II Evaluation (2016-2019)

1.4 The Phase II Evaluation Design

Phase II Evaluation Objectives, Research Questions, and Data Sources

The Evaluation’s Comparison Groups

Limitations of the Evaluation’s Design

1.5 Structure of This Report

PRA Programs in the Six Study States

2.1 PRA Grants in the Study States

2.2 State PRA Target Populations

2.3 PRA Program Profiles in the Six Study States

California

Delaware

Louisiana

Maryland

Minnesota

Washington

2.4 PRA Program Status in the Study States

Status of Units under Contract and under Lease

PRA Residents Assisted During 2015-2018

Duration of PRA Resident Tenancy

Chapters 3-7: Early Evidence from the PRA Program

Demographic Characteristics of PRA Residents and Comparison Groups

3.1 Household and Demographic Characteristics

Household Size

Age

Race, Ethnicity, and Gender

Household Income

Neighborhood Characteristics for PRA Residents and the Comparison Groups

4.1 Approach to Comparison of Neighborhood Characteristics

4.2 Neighborhood Demographic and Household Characteristics

Residential Density and Urban/Rural Classification

Percentage of Residents Who Report a Disability

Education and Income

Racially and Ethnically Concentrated Areas of Poverty Neighborhoods

Neighborhood Livability Indicators

4.3 Access to Services and Transportation

Resident Perception of Neighborhood

Service Provider Perceptions of Neighborhoods Where PRA Units Are Available

PRA and PRAC Property Characteristics

5.1 Section 811 Property Characteristics

Building Type

Property Size

Age and Development Status of PRA Units under Contract

For-Profit Status of Owner

5.2 Residents’ Satisfaction with Their Housing

5.3 Physical Quality of PRA Units

PRA Resident Views on the Physical Quality of Their Housing

Maintenance Issues Since Resident Moved into Unit

Program Partner Perceptions of the Quality of PRA Units

5.4 Unit Accessibility Features and Needs of Residents

Reported Accessibility Needs of PRA Residents

5.5 Integration of PRA Units within Properties

Resident Perspectives on Community Integration and Independence

PRA and PRAC Community-Based Services

6.1 Availability and Use of Community-Based Services

Housing Locator Services

Transition Services

Ongoing Tenancy Supports

6.2 Resident Experience with Community-based Services and Alignment with Needs

Resident Experience with Community-based Services

Quality of Life and Health Status

6.3 Housing Tenancy and Program Exits

Healthcare Conditions and Utilization

7.1 Healthcare Conditions and Prior Healthcare Utilization

Chronic and Potentially Disabling Conditions

7.2 PRA Impacts on Healthcare Utilization

Prior Healthcare Utilization of PRA Residents and the Comparison Groups

Early PRA Tenancy Findings

7.3 Comparison of Healthcare Utilization for PRA Residents and Each Comparison Group

PRA and PRAC Comparison

PRA and NED Comparison

PRA and Other HUD Comparison

PRA and Non-HUD Comparison

Summary

Economic Study of the PRA Program

8.1 Analyzing Costs and Cost-Effectiveness of PRA and Comparison Programs

Approach to Cost and Cost Allocation Analyses of PRA and Comparison Programs

Limitations of the Economic Study

8.2 Costs of Rental Subsidies

Average Monthly Per-Unit Rental Subsidies in the PRA Program

Comparison of Actual and Estimated Average Rental Subsidies

Comparison of PRA Contract Rent to Fair Market Rent

Comparison of PRA, PRAC, NED, and Other HUD Monthly Per-Unit Rental Subsidies

8.3 Capital Subsidy Costs

Sources of Capital Financing for PRA and PRAC Properties

Comparison of Combined Capital and Rent Subsidies in the PRA and PRAC Programs

8.4 Costs of Healthcare and Disability-related Services Utilization

Sources of Assistance for Healthcare and Disability Services for PRA Residents

Costs of Relevant Healthcare and Disability-Related Services Assistance through Medicaid

8.5 Costs of Program Administration

PRA Program Staffing in State Housing and Health Agencies

Estimated Annual PRA Program Administrative Costs

Sources of Funding for PRA Program Administration

Costs of HUD’s Administration of the PRA Program and Administrative Costs in Comparison Programs

8.6 Total Estimated Costs and Cost Effectiveness

Summary of PRA Program and Comparison Group Costs

Cost-Effectiveness of the PRA Program

PRA Program Practices that May Lead to Successful Results

9.1 Strategies for Selecting and Prioritizing Target Populations

Results of Selection and Prioritization of PRA Target Populations

9.2 Removing Barriers to Applicant Eligibility for PRA

Applicant Eligibility Rates by State and Over Time

Applicants Found Ineligible for the PRA Program

Applicants Found Ineligible for the Properties

Strategies Used to Remove Barriers to Eligibility

9.3 Identifying and Securing Quality, Cost-Effective PRA Units

Percent of Estimated PRA Units Under Contract and Under Lease

Study State Adaptations to Secure More PRA Units

9.4 Ensuring PRA Residents Receive Tenancy Supports They Need

Challenges in Coordinating Tenancy Supports for PRA Residents

Strategies for Coordinating Tenancy Supports for PRA Residents

9.5 Developing Effective and Sustainable PRA Partnerships

How State Agencies Define Successful PRA Partnerships

Sustaining PRA Partnerships

Expanding Beyond the PRA Program Partnership

9.6 Summary

Conclusion on the Overall Effectiveness of the PRA Program

Increasing the Supply of Affordable Housing for People with Disabilities: PRA grantees have secured units for the program, but at a slower pace than HUD had planned

Expanding Housing Opportunities for People with Significant Healthcare and Service Needs: The PRA program is housing its intended target populations

Expanding Housing Opportunities in Integrated Settings: The evidence is mixed

Building Institutional Capacity at the State Level: PRA grantees established partnerships at the state and local levels, often building on existing relationships from previous supportive housing efforts, but at considerable administrative cost

10.2 Policy Implications and Areas for Future Research

Program Monitoring and Data Sharing

Greater Flexibility in Identifying PRA Units That Meet Resident Needs and Preferences

Sustaining and Expanding the PRA Partnerships

Is PRA an Effective Approach?

Areas for Future Research

Appendix A: Status of National PRA Program Implementation as of September 2018

A.1 Overview of Section 811 PRA Grant Awards

A.2 Status of Units under Agreement for the PRA Program

A.3 Status of PRA Grants by State and by Grant Year

A.4 Characteristics of PRA Units under RAC

A.5 Status of Applicant Referrals and Lease-Ups

A.6 Characteristics of Households That Moved into PRA Units

A.7 Exits from PRA Units

A.8 Housing Choice Vouchers Leveraged

Appendix B: Section 811 PRA Phase II Evaluation Methods

B.1 Overview of the Section 811 PRA Phase II Evaluation

B.2 Evaluation Data Sources

Integrated Real Estate Management System (iREMS)

Tenant Rental Assistance Certification System

Inventory Management System / PIH Information Center

Online Property Integrated Information Suite

Section 811 PRA Quarterly Reports

CMS Research Data Assistance Center

State Medicaid Data

B.3 The Analytic Sample for the Impact and Economic Studies

Multi-Stage Process for Constructing the Analytic Sample

Three Analytic Samples

Descriptive Demographic Characteristics of the Study Groups

Adjustments to Hypothesis Tests due to Multiple Comparisons

B.4 Approach to Economic Study

Housing Capital Subsidies

Costs of Healthcare and Disability Services Utilization

Costs of Program Administration

References

Exhibit 1.1: Evaluation Objectives, Research Questions, and Data Sources

Exhibit 2.1: PRA Grant Amounts and Planned Units in the Study States, by State and Grant Year

Exhibit 1.2: Summary of Program Features for HUD Comparison Programs

Exhibit 2.4: PRA-Assisted Residents in the Study States by Previous Living Situation, September 2018

Exhibit 2.3: Status of Planned 811 PRA Units in the Study States, September 2018

Exhibit 2.2: Target Population of Section 811 PRA Programs

Exhibit 3.1: Household and Bedroom Size of PRA, PRAC, NED, and Other HUD-Assisted Households in the Study States, March 2018

Exhibit 3.2: Demographic and Socioeconomic Characteristics of PRA, PRAC, NED, and Other HUD-Assisted Households in the Study States, March 2018

Exhibit 3.3: Household Income of PRA, PRAC, NED, and Other HUD-Assisted Households in the Study States, March 2018

Exhibit 3.4: Sources of Household Income of PRA, PRAC, NED, and Other HUD-Assisted Households in the Study States, March 2018

Exhibit 4.1: Number of Households, Properties, and Census Tracts in Neighborhood Analysis, by Study Group in the Study States, March 2018

Exhibit 4.2: Neighborhood Characteristics of HUD-Assisted Households in Study Groups, March 2018

Exhibit 4.3: Neighborhood Demographic Characteristics of HUD-Assisted Households in Study Groups, March 2018

Exhibit 4.4: Comparison of Neighborhood Livability Indicators of HUD-Assisted Households in Study Groups, March 2018

Exhibit 4.5: PRA and PRAC Neighborhood Accessibility Measures

Exhibit 4.6: Section 811 PRA and PRAC Neighborhood Measures

Exhibit 5.2: Size of Properties with PRA Units and PRAC Units in the Study States, September 2018

Exhibit 5.1: Section 811 PRA Building Types for Occupied Units in the Study States, September 2018

Exhibit 5.3: Year of Construction or Last Rehabilitation for PRA and PRAC Properties in the Study States, September 2018

Exhibit 5.4: Comparison of Section 811 PRA and PRAC Resident Satisfaction with Housing

Exhibit 9.1: Grantee-Reported Living Situation of PRA Applicants and Residents, September 2018

Exhibit 8.11: Per-Unit Annual Cost Estimates for PRA and Comparison Programs

Exhibit 8.10: Section 811 PRA Grant Administrative Fees Awarded and Received, Program Years 2015-2018

Exhibit 8.9: Estimated Annual Grantee and State Agency Partner PRA Program Administrative Costs

Exhibit 8.8: State Housing and Health Agency Staff Administering the PRA Program, in Full-Time Equivalents

Exhibit 8.7: Section 811 PRA Program State Agencies in Six States

Exhibit 8.6: Healthcare and Disability-Related Services Costs for PRA, PRAC, NED, and Other HUD Residents

Exhibit 8.5: Annualized Per-Unit Total Housing Subsidy for PRA and PRAC

Exhibit 8.4: Annualized Per-Unit Capital Subsidy for PRA and PRAC

Exhibit 8.2: Monthly Rental Subsidy for PRAC, NED, and Other HUD

Exhibit 8.1: Monthly Rental Subsidy and Contract Rent for Units in PRA Program

Exhibit 7.7: Average Rates of Healthcare Utilization for PRA Residents Relative to Medicaid Enrollees Not Receiving HUD Assistance, 2016

Exhibit 7.6: Average Rates of Healthcare Utilization by PRA Residents Relative to Recipients of Other HUD Assistance, 2016

Exhibit 7.5: Average Rates of Healthcare Utilization by PRA Residents Relative to NED Voucher Holders, 2016

Exhibit 7.4: Average Rates of Healthcare Utilization of PRA Residents Relative to PRAC Residents, 2016

Exhibit 7.3: Unweighted Comparisons of Healthcare Utilization, 2015 (prior to moving in for PRA residents)

Exhibit 7.2: Unweighted Comparisons of Chronic and Potentially Disabling Conditions among PRA Residents and Individuals in the Comparison Groups Enrolled in Medicaid in 2015 and 2016

Exhibit 7.1: Healthcare Utilization Comparison Samples

Exhibit 6.5: Hazard Ratios for Reasons for Program Exits for PRA and PRAC Residents, 2013–2018

Exhibit 6.4: Reasons Tenants Had Left PRA Housing as of September 2018

Exhibit 6.3: Statistically Significant Differences in Reported Health and Quality of Life of Section 811 Residents

Exhibit 6.2: Use of Tenancy Supports by PRA and PRAC Residents

Exhibit 6.1: The Three Phases of Community-based Services

Exhibit 5.8: Section 811 Community Integration Measures

Exhibit 5.7: Reported Accessibility Needs of Section 811 PRA and PRAC Residents

Exhibit 5.6: Unresolved and Resolved Maintenance Issues Reported by PRA Residents

Exhibit 5.5: Section 811 PRA Resident Report of Property Condition

Exhibit 9.2: Applicants Found Eligible for the PRA Program in the Six Study States, Program Years 2015–2018

Exhibit 9.3: Reasons Applicants were Found Ineligible for the PRA Program, Program Years 2015–2018

Exhibit 9.4: Percentage of Applicants That Did Not Meet PRA Program Requirements, Program Years 2015–2018

Exhibit 9.9: Change in Estimated PRA Units as a Result of Average Expected Increases in Contract Rent

Exhibit 9.8: Units Identified, Under Contract, and Under Lease for PRA, September 2018

Exhibit 9.7: Status of Planned Section 811 PRA Units by State, September 2018

Exhibit 9.6: Percentage of PRA Applicants Whose Lease Applications Were Not Accepted by Owners, Program Years 2015–2018

Exhibit A.9: Section 811 PRA Units under RAC and under Lease as of September 2018, by Bedroom Size

Exhibit A.8: Implementation Status of FY13 Grantees as of September 2018, by State

Exhibit A.7: Implementation Status of FY12 Grantees as of September 2018, by State

Exhibit A.6: Occupancy Status of Units under RAC as of September 2018, by Grant Year

Exhibit A.5: Status of ARACs with FY13 Grantees through September 2018, by State

Exhibit A.4: Status of ARACs with FY12 Grantees through September 2018, by State

Exhibit A.3: Contracts Executed with FY13 Grantees through September 2018, by State

Exhibit A.2: RACs Executed with FY12 Grantees through September 2018, by State

Exhibit A.1: FY12 and FY13 PRA Grant Funding Awards and Planned PRA Units

Exhibit A.10: Section 811 PRA Units under RAC as of September 2018, by Accessibility

Exhibit A.13: Estimated and Actual Rental Assistance Amounts through September 2018, by State

Exhibit A.12: Contract Rent and Rental Assistance Payment for Units under Lease as of September 2018, by Bedroom Size

Exhibit A.11: Grantee Commitment for Units Rent Levels as of September 2018, by Grant Year and State

Exhibit A.14: Section 811 PRA Grants Target Populations (FY12 Grantees)

Exhibit B.4.3: Healthcare Utilization and HCBS Cost Analysis Measures

Exhibit B.4.2: Housing Subsidy Analysis Samples

Exhibit B.4.1: Cost Analysis Data Sources

Exhibit B.3.13: CCS Diagnosis Categories that Meet the Predictive Criteria

Exhibit B.3.12: Measures Describing Healthcare Utilization by Medicaid Enrollees Using Data Collected from States

Exhibit B.3.11: Condensed Categories of Chronic and Potentially Disabling Conditions

Exhibit B.3.10: Cox Hazard Ratio for Program Exits Due to Death

Exhibit B.3.9: Cox Hazard Ratio for Tenant Initiated Reasons

Exhibit B.3.8: Cox Hazard Ratio for Owner Initiated Reasons Other Than Nonpayment of Rent

Exhibit B.3.7: Cox Hazard Ratio for Program Exits Due to Owner Initiated Nonpayment of Rent

Exhibit B.3.6: Hazard Ratios for Reasons for Program Exits for PRA and PRAC Residents, 2013–2018

Exhibit B.3.5: Cox Proportional Hazards Regression for Rates of Program Exits for PRA and PRAC Residents—Any Exit Type

Exhibit B.3.4: Unweighted and Unadjusted Counts of PRA and PRAC Program Exits, 2013–2018

Exhibit B.3.3: Percentage of HUD Tenants on September 30, 2017, Enrolled in Medicaid in 2012–2013 or Medicare in 2014–2015

Exhibit B.3.2: Impact Study Research Matrix

Exhibit B.3.1: Categories of Measured Impacts Compared to PRA

Exhibit B.2:3: Completed Administrative Interviews for PRA Evaluation

Exhibit B.2.2: Surveys Completed by State and Type of Property

Exhibit B.2.1: HUD Administrative Data Sources

Exhibit A.23: HCVs Leveraged for PRA Properties, by Grant Year and State

Exhibit A.22: Reasons Tenants Left PRA Housing through September 2018

Exhibit A.21: Housing Tenure of Current PRA Residents through September 2018

Exhibit A.20: Previous Living Situation of PRA Residents through September 2018, by Grant Year

Exhibit A.19: Living Situation of Ineligible Applicants through September 2018, by Grant Year

Exhibit A.18: Living Situation of Applicants on the Waitlist through September 2018, by Grant Year

Exhibit A.17: Living Situation of Applicants through September 2018, by Grant Year

Exhibit A.16: Referral Source of Applicants through September 2018, by Grant Year

Exhibit A.15: Section 811 PRA Grants Target Populations (FY13 Grantees)