Evaluating MTW Flexibility

for Smaller PHAs

Baseline Report

U.S. Department of Housing and Urban Development | Office of Policy Development and Research

DISCLAIMER

The contents of this report are the views of the authors and do not necessarily reflect the views or policies of the U.S. Department of Housing and Urban Development or the U.S. Government.

Evaluating MTW Flexibility for Smaller PHAs:

Baseline Report

Prepared for

U.S. Department of Housing and Urban Development

Office of Policy Development and Research

Prepared by

Judy Geyer, Tanya de Sousa, Larry Buron, and Laura Peck

with Tyler Morrill, Edoardo Ortiz, Cara Sierks, Douglas Walton, and Will Yetvin Abt Associates Inc.

September 2021

Foreword

Moving to Work (MTW) is a demonstration program for public housing agencies (PHAs) that provides them the opportunity to design and test innovative, locally designed strategies that use federal dollars more efficiently, help residents find employment and become self-sufficient, and increase housing choices for low-income families. MTW allows PHAs exemptions from many existing public housing and voucher rules and provides funding flexibility with how they use their federal funds. PHAs in the MTW demonstration have pioneered a number of innovative policy interventions that have been proven to be successful at the local level and subsequently rolled out to the rest of the country’s PHAs. In 2016, Congress directed the U.S. Department of Housing and Urban Development (HUD) to expand the MTW demonstration by 100 agencies, with an emphasis on ensuring that the expansion included rigorous evaluation on the innovative ideas made possible by MTW flexibilities. As of early 2023, 126 PHAs nationwide have been granted MTW designation.

Congress also required HUD to create a Research Advisory Committee to advise the Secretary of HUD on what research topics should be studied. That committee is comprised of existing MTW

agency leadership, tenant representatives, knowledgeable researchers, and HUD staff. The committee made a number of recommendations, many of which the Secretary directed HUD to implement. Each research question reflects a MTW expansion “cohort.”

This report reflects the first MTW report about the first cohort. The research question to be answered with this first cohort is: how does providing MTW regulatory flexibility impact smaller PHAs. This first cohort includes 31 “smaller” high-performing PHAs, each administering 1,000

or fewer total units.

The evaluation explores how smaller PHAs use the flexibility to achieve the statutory objectives of the MTW program and what the consequences of that flexibility are for housing authority operations and tenants. To date, research regarding MTW has been specific to large PHAs and has relied on descriptive studies. However smaller PHAs constitute a large share of all PHAs and face unique staff capacity and financial challenges as compared to larger PHAs. This evaluation will fill a critical knowledge gap on how smaller PHAs use the flexibility offered by their new MTW designation.

This report, Evaluating MTW Flexibility for Smaller PHAs: Baseline Report, is the first of five annual reports that will be generated under this evaluation. Using data from PHA applications and telephone interviews, the report explores PHA motivations for those participating in MTW

and the interrelated objectives that these new MTW agencies hope to pursue using the flexibilities afforded through their MTW designation to improve program operations for tenants.

In addition, the report lays the groundwork for future reports by detailing the outcome measures that will be used to assess the impact of MTW designation and by documenting and confirming the soundness of the experimental research design for this evaluation. HUD acknowledges that

Baseline Report – Evaluating MTW Flexibility for Smaller PHAs ▌pg. ii

these outcome measures have been selected for an independent evaluation of the smaller PHAs in the flexibility cohort. In the future, HUD intends to explore additional metrics that the Department can use to evaluate the way MTW agencies have been meeting the goals of the program set out in statute.

These 31 PHAs are just beginning their tenure as new MTW agencies, and this report is therefore largely descriptive in nature, setting the stage for future outcomes reports. This report describes a set of PHAs with ambitious goals to improve their administrative efficacy and meet the needs of tenants. HUD looks forward to learning from their progress in the coming years.

Solomon J. Greene

Principal Deputy Assistant Secretary for Policy Development and Research

U.S. Department of Housing and Urban Development

Baseline Report – Evaluating MTW Flexibility for Smaller PHAs ▌pg. iii

Contents

List of Exhibits ..................................................................................................................................... vi

Executive Summary .......................................................................................................................... viii

1.

Introduction to the MTW Flexibility for Smaller PHAs Evaluation .................................. 1

1.1 Core Components of MTW .............................................................................................. 2

1.1.1 Statutory Objectives ............................................................................................ 2

1.1.2 Statutory Requirements ....................................................................................... 2

1.1.3 Funding Flexibility .............................................................................................. 3

1.1.4 Statutory and Regulatory Waivers ...................................................................... 3

1.2 Overview of the First Cohort of MTW Expansion Agencies ........................................... 5

1.2.1 Eligibility and Requirements for Flexibility Cohort ........................................... 5

1.2.2 Implementation of Flexibility Cohort Demonstration to Date ............................ 5

1.3 Overview of the Evaluation Design ................................................................................. 7

2.

The First MTW Expansion Cohort in Context ................................................................... 11

2.1 Prior Research on MTW Outcomes ............................................................................... 11

2.2 Flexibility Cohort PHAs Compared to All Smaller PHAs ............................................. 13

3.

Motivation for Applying for MTW and Plans for Using Its Flexibility ............................ 16

3.1 Motivations for Applying for the MTW Demonstration ................................................ 17

3.1.1 Factors Motivating Application ........................................................................ 17

3.1.2 Most Important Statutory Objectives in PHAs’ Decision to Apply .................. 18

3.2 Flexibility Cohort PHAs’ Plans for Using MTW Flexibility ......................................... 18

3.2.1 Developing a Plan for Part 2 of the MTW Application .................................... 19

3.2.2 Current Plans for MTW Implementation .......................................................... 20

3.2.3 MTW Funding Flexibility ................................................................................. 25

3.2.4 New Funding Sources ....................................................................................... 28

3.3 Changes in Flexibility Cohort PHAs’ Plans over Time ................................................. 28

3.4 Control PHAs’ Plans for Using MTW Flexibility ......................................................... 30

4.

Baseline Values of Outcome Variables................................................................................. 33

4.1 PHA Cost-Effectiveness................................................................................................. 34

4.2 Household Self-Sufficiency ........................................................................................... 35

4.3 Household Housing Choice ............................................................................................ 37

4.3.1 Opportunity Neighborhoods ............................................................................. 38

4.3.2 Affordable Housing ........................................................................................... 40

Baseline Report – Evaluating MTW Flexibility for Smaller PHAs ▌pg. iv

Contents

4.3.3 Hard-to-House Populations ............................................................................... 42

4.4 Other Tenant Outcomes ................................................................................................. 42

4.4.1 Households Served and Household Exits ......................................................... 42

4.4.2 Rent Burden ...................................................................................................... 44

5.

Summary ................................................................................................................................. 47

Appendix A. List and Map of Study PHAs ....................................................................................... 49

Appendix B. Comparison of Treatment PHAs to Control and Comparison PHAs ...................... 54

B.1 Baseline Characteristics and Balance of the RCT PHAs ............................................... 54

B.2 Baseline Characteristics and Balance of the QED PHAs ............................................... 56

Appendix C. Outcome Measures’ Definitions & Construction ...................................................... 58

References ............................................................................................................................................ 66

Key Terms Regarding PHAs .............................................................................................................. 68

Baseline Report – Evaluating MTW Flexibility for Smaller PHAs ▌pg. v

Contents

List of Exhibits

Exhibit 1-1. MTW Statutory Objectives ....................................................................................................... 2

Exhibit 1-2. MTW Statutory Requirements .................................................................................................. 3

Exhibit 1-3. Implementation of First Cohort of MTW Expansion ................................................................ 6

Exhibit 1-4. Random Assignment Probabilities, by Region ......................................................................... 7

Exhibit 2-1. Comparison of RCT PHAs versus All Smaller, Eligible PHAs.............................................. 14

Exhibit 3-1. Most Important MTW Objectives Motivating PHAs’ Initial Application .............................. 18

Exhibit 3-2. Flexibility Cohort PHAs’ Intentions to Use MTW Waivers, by Program Type ..................... 21

Exhibit 3-3. MTW Fund Flexibility ............................................................................................................ 25

Exhibit 3-4. Treatment PHAs’ Intent to Use MTW Funding Flexibility .................................................... 26

Exhibit 3-5. RCT PHAs’ Intent to Use MTW Funding Flexibility ............................................................. 30

Exhibit 4-1. Baseline Levels of PHA Cost (per Household per Month) ..................................................... 35

Exhibit 4-2. Baseline Levels of Employment and Earnings among Adults in Nonelderly,

Nondisabled Households ............................................................................................................... 37

Exhibit 4-3. Households’ Housing Choice: Low-Poverty Neighborhood Location at Baseline ................. 40

Exhibit 4-4. Households’ Housing Choice: Supply of Quality, Affordable Housing Levels at

Baseline .......................................................................................................................................... 41

Exhibit 4-5. Households’ Housing Choice: Hard-to-House Population Levels at Baseline ....................... 42

Exhibit 4-6. Other Tenant Outcomes: Households Served Levels at Baseline ........................................... 43

Exhibit 4-7. Other Tenant Outcomes: Households Exit Levels at Baseline ............................................... 44

Exhibit 4-8. Other Outcomes: Rent Burden in Public Housing at Baseline ............................................... 45

Exhibit 4-9. Other Outcomes: Rent Burden in Housing Choice Voucher Programs at Baseline ............... 46

Exhibit A-1. List of 33 Treatment PHAs .................................................................................................... 49

Exhibit A-2. List of 10 Control PHAs ........................................................................................................ 50

Exhibit A-3. List of 99 Comparison PHAs ................................................................................................. 51

Exhibit A-4. Map of PHAs in the Study ..................................................................................................... 53

Exhibit B-1. Comparison of PHA and Household Characteristics at Treatment and Control PHAs .......... 54

Exhibit B-2. Comparison of Plans to Use MTW Waivers at Treatment and Control PHAs ...................... 56

Exhibit B-3. Comparison of Treatment and Comparison PHAs ................................................................. 57

Exhibit C-1. PHA Cost and Cost-Effectiveness (per Household per Month) ............................................. 58

Exhibit C-2. Self-Sufficiency on Nonelderly, Nondisabled Households .................................................... 60

Baseline Report – Evaluating MTW Flexibility for Smaller PHAs ▌pg. vi

Contents

Exhibit C-3. Households’ Housing Choice: Opportunity Neighborhoods .................................................. 61

Exhibit C-4. Households’ Housing Choice: Supply of Quality, Affordable Housing ................................ 62

Exhibit C-5. Households’ Housing Choice: Hard-to-House Populations ................................................... 63

Exhibit C-6. Other Tenant Outcomes: Households Served ........................................................................ 64

Exhibit C-7. Other Tenant Outcomes: Households Exits ........................................................................... 65

Exhibit C-8. Other Outcomes: Rent Burden ............................................................................................... 65

Baseline Report – Evaluating MTW Flexibility for Smaller PHAs ▌pg. vii

Executive Summary

Moving to Work (MTW) is a demonstration program of HUD that gives public housing agencies (PHAs) regulatory and statutory flexibility to test ways to achieve three statutory objectives:

• Reduce cost and achieve greater cost-effectiveness in federal expenditures.

• Give incentives to families with children where the head of household is working, seeking work, or preparing for work by participating in job training, educational programs, or programs that assist people to obtain employment and become economically self-sufficient.

• Increase housing choices for low-income families.

In addition to enjoying statutory and regulatory flexibility, MTW PHAs can flexibly use their funding for the Housing Choice Voucher (HCV) and public housing programs.

The MTW demonstration was originally authorized in 1996. There were 39 mostly large PHAs participating in 2016 when Congress authorized HUD to expand the MTW demonstration1. HUD was directed to include an additional 100 PHAs, with at least half of them required to be smaller PHAs, defined as PHAs administering 1,000 or fewer combined HCV and public housing units. HUD decided to add PHAs to the demonstration in cohorts, with each cohort of new MTW PHAs testing a different policy change. The first cohort of the MTW expansion—the subject of this report— focuses broadly on the overall impact of MTW flexibility on smaller PHAs. 2

This report documents the characteristics of the PHAs and their tenants prior to the PHA’s designation as MTW and their participation in the evaluation. It also explores PHAs’ motivations for participating in MTW and presents baseline measures of the outcomes that will be used to measure impacts in future analyses.

The First Cohort of MTW Expansion

The Flexibility Cohort is the first set of PHAs to receive MTW designation under the 2016 act expanding MTW. To be eligible for this cohort, a PHA had to be smaller and be a high performer (that is, rated a

“high performer” in the public housing or HCV programs and not “troubled” under either).

PHAs that receive MTW designation pursuant to the 2016 act are subject to the Operations Notice for the Expansion of the Moving to Work Demonstration Program (MTW Operations Notice). 3 The MTW

1 See Section 239 of the Consolidated Appropriations Act, 2016 (Public Law 114-113) 2 For the purposes of the MTW Expansion, “smaller” PHAs are defined as PHAs with less than 1,000 combined units. This definition includes moderate-sized PHAs that are not defined as small in HUD statutes (For example, HUD defined a small PHA as having no more than 249 public housing units in the Small PHA Deregulation

(https://www.federalregister.gov/documents/2002/08/14/02-20547/deregulation-for-small-public-housing

agencies) and as less than 550 combined units in the Housing Opportunity Through Modernization Act

(https://www.federalregister.gov/documents/2020/10/08/2020-21400/housing-opportunity-through-

modernization-act-of-2016-housing-choice-voucher-hcv-and-project-based).

3 https://www.federalregister.gov/documents/2020/08/28/2020-18152/operations-notice-for-the-expansion-of-the-

moving-to-work-demonstration-program

Baseline Report – Evaluating MTW Flexibility for Smaller PHAs ▌pg. viii

Executive Summary

Operations Notice includes a list of pre-approved waivers from program rules that MTW PHAs can apply to their program. Examples of waivers are:

• Using alternative rent policies such as offering rents by income tier, stepped rents, and alternative minimum rents.

• Offering short-term assistance (less than one year).

• Permitting voucher payment standards outside of the basic range.

• Establishing housing development programs that use MTW funds to acquire, renovate, and/or build units that are not PHA-owned units.

A full list of pre-approved waivers is in Appendix I of the MTW Operations Notice. MTW PHAs can also apply for “agency-specific waivers,” which are waivers that are not explicitly listed in the MTW

Operations Notice. Agency-specific waivers require HUD approval prior to implementation.

Evaluation Design

HUD contracted with Abt Associates to conduct an independent evaluation of the first cohort of expansion agencies. The evaluation includes both a process study and an impact study. The goal of the evaluation is to answer two questions:

• How do smaller PHAs use their MTW flexibility?

• What are the consequences (impacts) of MTW flexibility on smaller PHAs and their tenants?

Forty-three PHAs completed the initial application process in 2018–19 and met the eligibility requirements for the first cohort of expansion PHAs. HUD randomly assigned the 43 PHAs into a treatment group (33 PHAs) and a control group (10 PHAs). The 33 Treatment PHAs were invited to apply for MTW designation. The remaining 10 Control PHAs were not offered the opportunity to apply for MTW designation under the first cohort, but they may apply to future cohorts. By the close of the application period, 31 of the 33 Treatment PHAs completed their applications successfully. In January 2021, HUD announced the list of PHAs that were offered MTW designation under the first cohort of expansion agencies.

HUD intends to support the independent evaluation of this first cohort of the MTW expansion for five years. The evaluation is structured to produce this Baseline Report (2021), which covers the year before first cohort of expansion PHAs are officially designated MTW agencies, and a series of Annual Reports (2022–25) documenting the activities and outcomes associated with these PHAs in the year they are designated as MTW and the three years after being designated MTW. These reports will include a process study addressing how the first cohort of expansion PHAs use their MTW flexibility and an impact study measuring the impact of MTW on PHA and tenant outcomes. The process and impact studies for the Baseline Report are describe below. In addition, the Baseline Report describes the characteristics of the PHAs participating in the evaluation and compares them to other smaller PHAs to provide context for the evaluation.

Baseline Report – Evaluating MTW Flexibility for Smaller PHAs ▌pg. ix

Executive Summary

Process Study

For this Baseline Report, the process

Key to PHA Sample Names

study explores PHA motivations for

Flexibility Cohort PHAs: 31 PHAs offered MTW

applying to MTW, plans on how to

designation as part of the MTW Flexibility for

use MTW, and how those plans have

Smaller PHAs Cohort. These PHAs are officially

changed since their initial 2018–19

designated as an MTW agency when they sign

application. In future Annual Reports,

their MTW Annual Contributions Contract (ACC)

the process study will address the

Amendment.

research question about how

Comparison PHAs: 99 PHAs that did not apply for

Treatment PHAs use their MTW

MTW but were selected to be part of the QED

flexibility to meet the

comparison group based on the similarity of

observable characteristics to the Flexibility Cohort

demonstration’s goals of increasing

PHAs.

cost-effectiveness, self-sufficiency,

Control PHAs: 10 PHAs that applied and were

and housing choice.

determined eligible for MTW but were not invited

to apply for MTW designation as part of first

Impact Study

cohort of expansion agencies.

This Baseline Report also sets the

Initial MTW PHAs: The PHAs that participated in

context for future impact studies by

MTW prior to the MTW expansion. At the time

analyzing whether the evaluation

Congress authorized the expansion in 2016,

design provides a way to clearly

there were 39 MTW PHAs.

estimate the causal impact of MTW

Quasi-experimental Design (QED) PHAs: 132

designation on cost-effectiveness,

PHAs that include both the Treatment and

self-sufficiency, and housing choice

Comparison PHAs.

in an unbiased manner (whether the

Randomized control trial (RCT) PHAs: 43 PHAs

evaluation has internal validity). The

that include both Treatment and Control PHAs.

report also considers whether the 43

Treatment PHAs: 33 PHAs (including all Flexibility

randomized control trial (RCT) PHAs

Cohort PHAs) that applied and were determined

are representative of all eligible

eligible for MTW, then were invited to apply for

PHAs, and thus whether the study’s

MTW designation as part of the first cohort of

expansion agencies.

findings likely represent the

consequences of all smaller and high-

performing PHAs having MTW flexibility (whether the evaluation has external validity). Finally, this report presents the definitions and baseline values of the outcome measures that the impact study will use in the analyses to be presented in Annual Reports starting in 2022.

The impact study will include two types of impact analyses:

• Experimental analysis: comparing the average outcomes of the 33 PHAs randomly assigned to the treatment group versus the average outcomes of the 10 PHAs randomly assigned to the control group.

This experimental analysis, leveraging the RCT design, will assess the effect of MTW flexibility on the statutory objectives of cost-effectiveness, self-sufficiency, and housing choice.

• Quasi-experimental analysis: comparing the average outcomes of the 33 PHAs randomly assigned to the treatment group versus the average outcomes of a matched comparison group of 99 PHAs selected by the evaluation team. This quasi-experimental analysis will assess the same outcomes as the experimental analysis related to the MTW statutory objectives, but its larger sample size will permit analysis by subgroups of PHAs within the treatment group and comparison group.

Baseline Report – Evaluating MTW Flexibility for Smaller PHAs ▌pg. x

Executive Summary

Key Findings from this Baseline Report

The most important objective motivating PHAs to apply for MTW was improving cost-effectiveness.

Operational goals related to increasing cost-effectiveness by reducing administrative burden and simplifying program rules were the primary motivator for about half of the PHAs (48%) to apply for MTW designation. PHAs also have goals around the self-sufficiency and housing choice objectives: encouraging employment, breaking intergenerational cycles of subsidy assistance and long-term subsidy assistance, expanding the services provided to tenants, increasing the number of quality affordable housing units, developing housing for specific populations, and improving landlord participation in the HCV program. Funding flexibility was also an important draw for becoming an MTW PHA.

Nearly all Flexibility Cohort PHAs plan to use MTW flexibility to change their reexamination process or tenant rent policies.

More than 90 percent of Flexibility Cohort PHAs are interested in altering the reexamination process used to verify income for determining rent (such as by instituting less frequent reexamination schedules or allowing tenants to self-certify assets) and in changing their tenant rent calculation procedures. More than half of the PHAs plan to use their MTW flexibility to add landlord incentives (68%), implement work requirements (58%), or make changes to their Family Self-Sufficiency (FSS) programs (58%).

Most Flexibility Cohort PHAs narrowed their planned use of waivers during the application process.

Between their initial letter of interest submitted in 2018–19 and their full application in 2020, all PHAs changed how they planned to use the MTW waivers. These changes often narrowed the PHA’s focus as PHA staff thought more about which activities would best suit the agencies’ internal goals and the needs of its tenants, reviewed the policies of current MTW PHAs, and assessed what they believed could be accomplished in the near term.

Many Flexibility Cohort PHAs do not yet have definitive plans for using their funding flexibility, but nearly all intend to use this flexibility at some point.

Twenty-seven of the 31 Flexibility Cohort PHAs expected to use funding flexibility to help achieve at least one of the three statutory objectives or to help improve PHA operations at some point during their participation in MTW.

Analysis of baseline data shows that the impact study will provide unbiased estimates of the effect of MTW designation.

For the experimental analysis, the Treatment and Control PHAs are well balanced, with no more differences in their baseline characteristics than one would expect due to chance. 4 For the quasi-experimental analysis, there are no statistically significant differences in observable baseline characteristics between the Treatment and Comparison PHAs.

4 Two of the 33 PHAs offered the opportunity to apply for MTW designation under first cohort of expansion agencies did not complete their application, and thus did not receive MTW designation. Therefore, the impact measured by comparisons between the Treatment PHAs and Control PHAs is technically “the impact of the invitation to apply for MTW designation.” In this report we shorten the phrase to “the impact of MTW

designation,” but the experimental analysis will always compare the full set of 33 Treatment PHAs to the 10

Control PHAs.

Baseline Report – Evaluating MTW Flexibility for Smaller PHAs ▌pg. xi

Executive Summary

The 43 RCT PHAs are more urban than are all PHAs eligible to participate in this cohort.

The 43 RCT PHAs differ from the set of all smaller, high-performing PHAs that were eligible to apply for designation in this MTW cohort in several ways. Many of these differences are because PHAs that applied were more likely to be in metropolitan areas. Compared to all eligible PHAs, the RCT PHAs are larger, have jurisdiction in areas with higher fair market rents, and serve a higher proportion of households with special purpose vouchers. The RCT PHAs are also less likely to be public housing-only PHAs (and more likely to be either HCV-only or both HCV and public housing PHAs) than all eligible PHAs. These differences need to be kept in mind when considering how the findings would apply if considering allowing all high-performing, smaller PHAs to have the flexibility allowed in MTW .

In anticipation of the impact study, the evaluation team has pre-specified more than 30 outcome measures of PHA cost-effectiveness, tenant self-sufficiency, housing choice, and other tenant outcomes.

The definitions and baseline averages for more than 30 outcomes that will be used to measure impacts in the subsequent Annual Reports are shown in this report. The outcomes are broken out by the three MTW

statutory objectives, with emphasis on what we judged to be the most important outcome for each objective. These three outcomes are: total operating and administrative expenditure per household per month for the cost effectiveness objective; annual earnings of adults in nonelderly, nondisabled households for the self-sufficiency objective; and percentage of HCV households living in low-poverty census tracts for the housing choice objective.

Baseline Report – Evaluating MTW Flexibility for Smaller PHAs ▌pg. xii

1. Introduction to the MTW Flexibility for Smaller PHAs Evaluation

Moving to Work (MTW) is a demonstration program of the U.S. Department of Housing and Urban Development (HUD) that gives public housing agencies (PHAs) regulatory and statutory flexibility to change how they operate to achieve certain policy goals. As directed by Congress, HUD is expanding the MTW demonstration using multiple cohorts of interested PHAs to test the impact of various policies made possible through MTW flexibility. This is the Baseline Report for an evaluation of the first cohort of the expansion, which was limited to smaller PHAs. 5 This report documents the characteristics of the PHAs in the evaluation and their tenants prior to MTW designation. It also explores PHAs’ motivations for participating in MTW and presents baseline measures of the outcomes that will be used to measure impacts in future analyses.

The evaluation of first cohort of expansion agencies will continue for another 4 years, each year producing an Annual Report addressing the demonstration’s two overarching research questions about the impact of MTW designation:

• How do smaller PHAs use their MTW flexibility?

• What are the consequences of MTW flexibility for smaller PHAs and their tenants?

This introductory chapter provides background information as context for the evaluation. It describes the core components of MTW, clarifies how the first cohort fits into the MTW expansion, and provides an overview of the evaluation.

Chapter 2 provides context for the evaluation by providing a brief review of the literature based on the experience of the PHAs that were designated MTW before the expansion (the Initial MTW PHAs) and compares the RCT PHAs in the evaluation to all eligible PHAs to gauge how representative the results are likely to be if MTW was offered to all smaller PHAs.

Chapter 3 examines the motivations PHAs had for applying to be a part of MTW and describes how PHAs that will receive MTW designation plan to use their MTW flexibility.

Chapter 4 presents the outcome measures that the evaluation will use to describe the consequences of MTW flexibility for smaller PHAs and their tenants. We define outcome measures for cost-effectiveness, household self-sufficiency, housing choice, and other tenant outcomes and present the study sample’s baseline values for those outcomes.

The final chapter is a summary of the major points in this document, with an emphasis on the internal and external validity of the impacts that will be measured in future reports.

The appendices provide technical details. Appendix A lists the PHAs in the study, and appendix B details baseline characteristics of these PHAs. Appendix C provides details on the definition and derivation of each outcome measure.

5 “Baseline” denotes the status of the PHAs as of 2020 or earlier, before Flexibility Cohort PHAs began to operate with MTW authority.

Baseline Report – Evaluating MTW Flexibility for Smaller PHAs ▌pg. 1

Chapter 1: Introduction to the MTW Flexibility for Smaller PHAs Evaluation 1.1 Core Components of MTW

MTW is a demonstration program that gives PHAs regulatory and statutory flexibility to test ways to increase the cost-effectiveness of federal housing programs, encourage greater self-sufficiency of households receiving housing assistance, and increase housing choices for low-income families. In addition to statutory and regulatory flexibility, MTW PHAs can use their funding for the Housing Choice Voucher (HCV) and public housing programs flexibly.

Section 204 of the Omnibus Consolidated Rescissions and Appropriations Act of 1996 (1996 Act) authorized the MTW demonstration, in which 39 PHAs were participating as of 2016. Section 239 of the Consolidated Appropriations Act of 2016 (2016 Act) authorized HUD to expand the MTW demonstration by 100 high-performing PHAs. This cohort is the first set of PHAs to receive MTW designation under the 2016 Act.

All MTW PHAs are expected to pursue the program’s statutory objectives and meet its statutory requirements. We describe the objectives and requirements below, as well as key provisions related to funding flexibility and statutory and regulatory waivers.

1.1.1 Statutory Objectives

The MTW demonstration has three statutory objectives that reflect what Congress sought to accomplish by giving PHAs statutory and regulatory flexibility (exhibit 1-1).

Exhibit 1-1. MTW Statutory Objectives

Statutory Objective

Shorthand Term

1. To reduce cost and achieve greater cost effectiveness in Federal expenditures Cost-effectiveness

2. To give incentives to families with children where the head of household is Self-sufficiency

working, seeking work, or is preparing for work by participating in job training, educational programs, or programs that assist people to obtain employment and become economically self-sufficient

3. To increase housing choices for low-income families

Housing choice

Source: Section 204 of the Omnibus Consolidated Rescissions and Appropriations Act of 1996

The statutory objectives provide a key organizing principle for the evaluation. The evaluation will focus on the policy changes that PHAs implement in pursuit of the statutory objectives and the outcomes of those changes for the PHAs and their tenants. (This report uses the term “tenant” to refer to individuals and families participating in the public housing and HCV programs.) The evaluation will document policy changes made in pursuit of other goals, but the evaluation’s main outcome measures relate to the statutory objectives.

1.1.2 Statutory Requirements

The MTW demonstration has 5 statutory requirements, shown in exhibit 1-2. The statutory requirements are standards that all MTW PHAs must meet, regardless of which policy and program changes they implement. The requirements serve as guardrails to ensure that policies implemented in pursuit of the statutory objectives do not have major adverse impacts on low-income families—for example, by making it harder for the lowest-income households to access the program, by reducing the number of families that can access housing assistance, or by reducing the quality of the housing available to assisted households.

HUD’s Office of Public and Indian Housing (PIH) will monitor MTW PHAs’ compliance with the

Baseline Report – Evaluating MTW Flexibility for Smaller PHAs ▌pg. 2

Chapter 1: Introduction to the MTW Flexibility for Smaller PHAs Evaluation statutory requirements for all expansion cohorts. The statutory requirements are not a focus of this evaluation.

Exhibit 1-2. MTW Statutory Requirements

Statutory Requirement

1. MTW agencies must ensure that at least 75 percent of the families assisted are very low-income families, in each fiscal year, as defined in Section 3(b)(2) of [the United States Housing Act of 1937].

2. MTW agencies must establish a reasonable rent policy, which shall be designed to encourage employment and self-sufficiency by participating families, consistent with the purpose of this demonstration, such as by excluding some or all of a family’s earned income for purposes of determining rent.

3. MTW agencies must continue to assist substantially the same total number of eligible low-income families as they would have assisted absent MTW flexibility.

4. MTW agencies must serve a mix of families (by family size) under MTW that is comparable to what they would have served absent MTW flexibility.

5. MTW agencies must ensure that housing assisted under the demonstration program meets housing quality standards established or approved by the Secretary.

Source: Section 204 of the Omnibus Consolidated Rescissions and Appropriations Act of 1996

1.1.3 Funding Flexibility

MTW PHAs can apply funding flexibility across the four funding streams for the public housing and HCV programs that are subject to MTW. 6 Funding flexibility means that MTW PHAs can merge their funding streams and deploy funding to meet any of the eligible uses under any of the funding streams.

MTW PHAs can also use their MTW funds in support of local, non-traditional (LNT) activities. 7 Funding flexibility is one of the mechanisms—along with statutory and regulatory waivers (discussed next)—that PHAs use to implement innovative policies and practices in pursuit of the statutory objectives. Funding flexibility can be a critical tool for implementing LNT activities, such as funding housing development.

1.1.4 Statutory and Regulatory Waivers

MTW PHAs receive waivers that exempt them from many of the statutory and regulatory requirements of the HCV and public housing programs. These waivers allow PHAs to implement program and policy changes to meet the MTW demonstration’s statutory objectives.

How PHAs access the waivers is somewhat different for the expansion PHAs than for the 39 Initial MTW

PHAs already participating in 2016. Each of the Initial MTW PHAs has an MTW Agreement with HUD

that outlines the administrative structure of that PHA’s MTW demonstration. Under these agreements, 6 The four funding streams are (1) public housing Operating Fund Program grants; (2) public housing Capital Fund Program grants; (3) HCV Housing Assistance Payment funds; and (4) HCV Administrative Fees. The programs subject to MTW are public housing, tenant-based HCV assistance, project-based HCV assistance, and HCV

homeownership assistance. Programs not part of the MTW demonstration are Mainstream Vouchers, Moderate Rehabilitation Renewals, HUD-Veterans Affairs Supportive Housing (HUD-VASH) Vouchers, Nonelderly Disabled (NED) Vouchers, and Family Unification Program (FUP) Vouchers. Under the Rental Assistance Demonstration (RAD) program, PHAs may convert public housing to Project-Based Vouchers (PBVs) or Project-Based Rental Assistance (PBRA). RAD PBVs are subject to MTW, but RAD PBRA units are not.

7 “Local, non-traditional (LNT) activities” refers to allowable uses of funds that are outside of the HCV and public housing programs. Examples of LNT activities include shallow subsidy programs, self-sufficiency programs, and housing development programs. We refer to the households served through LNT activities as “LNT households.”

Baseline Report – Evaluating MTW Flexibility for Smaller PHAs ▌pg. 3

Chapter 1: Introduction to the MTW Flexibility for Smaller PHAs Evaluation PHAs request HUD’s permission for agency-specific statutory and regulatory waivers through an MTW

Plan that they prepare and submit to HUD annually. Instead of submitting an MTW Plan each year, expansion PHAs will complete and submit an MTW Supplement to the PHA Plan, annually. 8 The MTW

Supplement identifies the waivers that the PHA plans to implement and indicates how the PHA will use each waiver to make program and policy changes.

PHAs that receive MTW designation pursuant to the 2016 Act are subject to the Operations Notice for the Expansion of the Moving to Work Demonstration Program (MTW Operations Notice). 9 The MTW

Operations Notice provides the expansion PHAs a menu of common waivers to choose from and instructions for the development of written impact analyses and hardship policies, where applicable.

MTW PHAs can also apply for waivers not listed in the MTW Operations Notice, called “agency-specific waivers.”

Flexibility Cohort PHAs can request three types of waivers that allow them to operate outside of the traditional rules and regulations:10

• MTW Waivers: HUD pre-approved more than 70 MTW waivers across 17 categories. These waivers are listed in appendix I of the MTW Operations Notice. 11 PHAs can implement any of these waivers without additional HUD review so long as (1) the waiver/activity is included in the PHA’s approved MTW Supplement; (2) the PHA has updated its Admissions and Continued Occupancy Policy or its Administrative Plan to include the waiver/activity; and (3) the waiver/activity is implemented with all required safe harbors, where applicable.

• Safe Harbor Waivers: HUD required safe harbors for a number of MTW waivers to protect tenants from hardship. Some, but not all, of these safe harbors can be modified or waived through an approved safe harbor waiver if the PHA determines that the prescribed safe harbor does not fit its needs, priorities, or local market conditions.

• Agency-Specific Waivers: PHAs can also request an individualized waiver of the requirements under the 1937 Housing Act and associated regulations to implement changes or new, innovative activities to meet their local priorities or needs. Agency-specific waivers must be reviewed and approved by HUD on a case-by-case basis.

8 Information on the MTW Supplement to the PHA Plan can be found here:

https://www.hud.gov/program_offices/public_indian_housing/programs/ph/mtw/expansion/mtwsupplement

9 The final MTW Operations Notice was published on August 28, 2020 (Docket No. FR-5994-N-05).

10 HUD may also issue “cohort-specific waivers,” if there are waivers not included in appendix I of the MTW

Operations Notice deemed necessary for implementing the required cohort study. These waivers will be provided in the Selection Notice for the cohort. There were no cohort-specific waivers for the first cohort of expansion agencies.

11 Examples of waivers are alternative rent policies such as rents by income tier, stepped rents, and alternative minimum rents; short-term assistance (less than one year); voucher payment standards outside the basic range; and housing development programs that use MTW funds to acquire, renovate, or build units that are not PHA-owned units. For a full list of pre-approved waivers, see appendix I of the Final Operations Notice for the Expansion of the Moving to Work Demonstration Program:

https://www.federalregister.gov/documents/2020/08/28/2020-18152/operations-notice-for-the-expansion-of-the-

moving-to-work-demonstration-program#h-75.

Baseline Report – Evaluating MTW Flexibility for Smaller PHAs ▌pg. 4

Chapter 1: Introduction to the MTW Flexibility for Smaller PHAs Evaluation Though the MTW waivers listed in the MTW Operations Notice are pre-approved, subject to public notice and inclusion in the MTW Supplement, PHAs can also apply for additional waivers, referred to as agency-specific waivers, that HUD will review and approve on a case-by-case basis. HUD will have 75

days to review the MTW Supplement. After that 75-day period, PHAs can implement the waivers without any additional HUD approval.

1.2 Overview of the First Cohort of MTW Expansion Agencies

The Flexibility Cohort is the first set of PHAs to receive MTW designation under the 2016 expansion.

The 2016 Act directed HUD to identify one specific policy change to be implemented by the PHAs in each cohort. For this cohort, HUD determined that the one policy change could be any permissible use of MTW flexibility at a smaller PHA. Thus, this cohort tests the overall impact of MTW flexibility at smaller PHAs.

1.2.1 Eligibility and Requirements for Flexibility Cohort

Eligibility for the first cohort of expansion PHAs was limited to

PHAs administering a combined 1,000 or fewer HCV and public

Eligibility for Flexibility Cohort

housing units at the time of application (“small” PHAs). PHAs

• Limited to PHAs administering a

may be administering only HCV units, only public housing units,

combined 1,000 or fewer HCV and

or both; there is no minimum size. Flexibility Cohort PHAs must

public housing units.

also be “high performers” under either the Public Housing

• Limited to high performers under the

Public Housing Assessment System

Assessment System (PHAS) or the Section Eight Management

or the Section Eight Management

Assessment Program (SEMAP) at the time of applying for the

Assessment Program.

demonstration (and not “troubled” under either).

• Certify to being willing to participate in

the flexibility cohort evaluation even if

Like all other PHAs receiving MTW designation as part of the

assigned to its control group.

2016 expansion, this cohort of MTW PHAs must implement

their MTW demonstration in accordance with the Operations

Notice. However, in this first cohort, HUD is not requiring the PHAs to implement any particular waiver or program and policy change. It is completely up to each PHA how it uses the MTW flexibility, within the parameters of the Operations Notice. What HUD will learn from this cohort is how smaller PHAs elect to use MTW flexibility and how that affects PHA and tenant outcomes.

By focusing on smaller PHAs, this cohort fills a gap in the knowledge base on the effects of statutory, regulatory and funding flexibility on smaller PHAs and their tenants. Smaller PHAs account for about 80

percent of the PHAs that administer the HCV and public housing programs. Only 1 of the 39 Initial MTW

PHAs is smaller, so research is limited on how MTW flexibility could benefit smaller PHAs.

1.2.2 Implementation of Flexibility Cohort Demonstration to Date

Exhibit 1-3 provides a timeline for the implementation of the first cohort of the MTW expansion as of September 2021.

Baseline Report – Evaluating MTW Flexibility for Smaller PHAs ▌pg. 5

Chapter 1: Introduction to the MTW Flexibility for Smaller PHAs Evaluation Exhibit 1-3. Implementation of First Cohort of MTW Expansion

Part 1 of MTW

Random Assignment

Part 2 of MTW

Flexibility Cohort

MTW ACC

Application

of Eligible Applicants

Application

PHAs Announced

Amendment

Signed

October 2018 -

August 2020 -

Spring 2021 or

May 2019

November 2019

December 2020

January 2021

later

43 Eligible PHA

33 Treatment PHAs

31 of the 33 Treatment PHAs 31 PHAs offered MTW

27 designated

Applicants

10 Control PHAs (randomly

invited to apply completed

designation (Flexibility

MTW by

assigned within regions to

the application

Cohort PHAs)

September 2021.

ensure geographic diversity)

Other 4 expected

to be designated

by end of 2021.

Note: A PHA must sign the MTW Annual Contributions Contract (ACC) Amendment before it is officially designated an MTW agency.

Source: Abt created based on HUD communications

The application period to determine the interested and eligible PHA pool for first MTW cohort of expansion PHAs opened in October 2018 and closed in May 2019. Forty-three PHAs completed the initial application process and met the eligibility requirements for this cohort.

In November 2019, HUD randomly assigned the 43 PHAs into a treatment group (33 PHAs) and a control group (10 PHAs). Geographic diversity of PHAs was a stated goal in the 2018 Selection Notice for the first cohort of expansion PHAs12 and was taken into account in the random assignment process. HUD

accomplished this by stratifying the applicant PHAs into five geographic regions and then randomly assigning them to the treatment and control groups within each region. HUD allocated the treatment group for each region in proportion to the share of eligible applicants in each region. Exhibit 1-4 shows the number of applications from each region and the probability of assignment to the treatment group.

The goal of the allocation was to assign the same percentage of eligible applicants to the treatment group in each region, although rounding led to small differences across regions. Probability of selection into the treatment group ranged from 71 percent (West) to 82 percent (Southeast). 13

In August 2020, immediately following the publication of the final Operations Notice, HUD invited the 33 PHAs that had been randomly assigned to the treatment group to complete the application for MTW.

The remaining 10 PHAs that had been randomly assigned to the control group were not offered the opportunity to continue the application process for MTW designation under this cohort, but they may apply to future cohorts of the MTW expansion.

The PHAs in the treatment group were provided approximately 4 months to complete their applications.

In January 2021, HUD announced that 31 PHAs were offered MTW designation under the first cohort of 12 Source: PIH Notice 2018-17, available at PIH-2018-17MTWDemonstrationProgram.pdf (hud.gov).

13 For more details on random assignment and other methodological issues, see Turnham et al., 2021.

Baseline Report – Evaluating MTW Flexibility for Smaller PHAs ▌pg. 6

Chapter 1: Introduction to the MTW Flexibility for Smaller PHAs Evaluation the expansion. The other two treatment PHAs elected not to continue the application, so they were not offered the opportunity to become an MTW agency.

Exhibit 1-4. Random Assignment Probabilities, by Region

Probability of

Geographic Region

Eligible Applicants Treatment Group

Control Group

Assignment to

Allocation

Allocation

Treatment Group

(%)

Northeast

12

9

3

75

Southeast

11

9

2

82

Midwest

5

4

1

80

Southwest

8

6

2

75

West

7

5

2

71

Total

43

33

10

77

Note: The regions were defined in the 2018 Selection Notice as follows: Northeast: CT, DC, DE, MA, MD, ME, NH, NJ, NY, PA, RI, VA, VT, and WV; Southeast: AL, FL, GA, KY, MS, NC, PR, SC, TN, and U.S. Virgin Islands; Midwest: IL, IN, MI, MN, OH, and WI; Southwest: AR, IA, KS, LA, MO, NE, NM, OK, and TX; and West: AK, AZ, CA, CO, HI, ID, MT, ND, NV, OR, SD, UT, WY, WA, American Samoa, Guam, and Northern Mariana Islands.

Source: 2018 Selection Notice and HUD Communications

An onboarding period extended from January to April 2021 during which time HUD delivered webinars and held office hours with the 31 PHAs offered MTW designation to help them prepare for becoming an MTW agency. The first step was for PHAs to get Board signatory approval for their MTW Annual Contributions Contracts (ACC) Amendment. The Amendment sets the terms of MTW participation and requirements to be followed. The month after the Amendment is executed, funding flexibility is allowed across the standard eligible public housing and HCV activities (that is, activities that do not require an MTW waiver). Between March and September 2021, 27 of the 31 Flexibility Cohort PHAs signed their MTW ACC Amendment. The rest of the Flexibility Cohort PHAs are expected to sign theirs sometime during 2021.

To use MTW flexibility for relief from standard requirements or policies, the Flexibility Cohort PHAs must submit an MTW Supplement to their PHA Plan describing their proposed use of MTW authority.

HUD must approve each PHA’s MTW Supplement before the PHA can begin implementing policies and activities that use MTW waivers described in the Operations Notice or agency-specific waivers not included in the Operations Notice. As of September 2021, several Flexibility Cohort PHAs had started working on their online MTW Supplement, but none had finished. Given the public process that PHAs must complete before submitting the MTW Supplement to HUD, no PHAs are likely to complete their submission until fall 2021.

1.3 Overview of the Evaluation Design

HUD contracted with Abt Associates to conduct an independent evaluation of the first cohort of expansion agencies. The goal of the evaluation is to document how smaller PHAs use their MTW

flexibility and analyze the impact of MTW flexibility on smaller PHAs and their tenants. To address these questions, this evaluation has both a process study and an impact study.

Process Study Design

In this Baseline Report, the process study explores motivations of PHAs for applying to MTW, the plans on how to use their MTW authority, and how those plans have changed since their initial 2018–19

applications. The data sources are an initial MTW Application Survey that all Treatment and Control

Baseline Report – Evaluating MTW Flexibility for Smaller PHAs ▌pg. 7

Chapter 1: Introduction to the MTW Flexibility for Smaller PHAs Evaluation PHAs completed in 2018–19; Part 2 of the MTW application—a narrative description of how PHAs plan to implement the MTW program upon designation—that Treatment PHAs completed in fall 2020; 14 and interviews with staff at Treatment and Control PHAs in March 2021.

In future Annual Reports, the process study will address the research question about how PHAs use their MTW flexibility to meet the demonstration’s three statutory objectives. The process study will document the programs and policies implemented by the Flexibility Cohort PHAs as described in their MTW

Supplement. The evaluation team will also interview staff from these PHAs to learn more about their MTW experiences in deciding on and implementing MTW waivers and staff from the Control PHAs to learn about policy changes they may be making within the regular program rules.

Impact Study Design

Key to PHA Sample Names

This Baseline Report sets the context

for the impact study that will estimate

Flexibility Cohort PHAs: 31 PHAs offered MTW

the impact of MTW on smaller PHAs’

designation as part of the MTW Flexibility for

cost-effectiveness, tenants’ self-

Smaller PHAs Cohort. These PHAs are officially

sufficiency, and tenants’ housing

designated as an MTW agency when they sign

their MTW Annual Contributions Contract (ACC)

choice. This report considers whether

Amendment.

the Treatment PHAs are representative

Comparison PHAs: 99 PHAs that did not apply for

of all smaller and high-performing

MTW but were selected to be part of the quasi-

PHAs, and whether the impact study’s

experimental design (QED) comparison group

findings likely represent the

based on the similarity of observable

consequences of all such PHAs having

characteristics to Flexibility Cohort PHAs.

MTW flexibility (whether the

Control PHAs: 10 PHAs that applied and were

evaluation has “external validity”).

determined eligible for MTW but were not invited

to apply for MTW designation as part of first

This report also presents the

cohort of expansion agencies.

definitions and baseline values of the

Initial MTW PHAs: The PHAs that participated in

outcome measures the impact study

MTW prior to the MTW expansion. At the time

will use in its analyses to be presented

Congress authorized the expansion in 2016,

in Annual Reports starting in 2022.

there were 39 MTW PHAs.

The outcomes relate to the statutory

Quasi-experimental design (QED) PHAs: 132

objectives of cost-effectiveness, self-

PHAs that include both the Treatment PHAs and

the Comparison PHAs.

sufficiency, and housing choice. For

each statutory objective, the evaluation

Randomized control trial (RCT) PHAs: 43 PHAs

team identified one confirmatory

that include both the Treatment PHAs and

Control PHAs.

measure of MTW impact as well as

numerous other exploratory outcomes

Treatment PHAs: 33 PHAs (including all Flexibility

Cohort PHAs) that applied and were determined

eligible for MTW, then were invited to apply for

MTW designation as part of the first cohort of

14 Part 2 of the MTW application required applicant PHAs to provide a high-level vision for their MTW program, their plans for future community and resident engagement, background on their PHA operating and inventory information, plans for their local MTW program which may have included a description or list of which MTW

waivers they would pursue, their proposed use of MTW funds, evidence of significant partnerships, and key dates and milestones for their MTW program over the initial 2 to 3 years.

Baseline Report – Evaluating MTW Flexibility for Smaller PHAs ▌pg. 8

Chapter 1: Introduction to the MTW Flexibility for Smaller PHAs Evaluation of interest (see Turnham et al., 2021). 15

Comparing baseline characteristics of treatment and control groups is standard practice for determining how well random assignment worked. Turnham et al. (2021) examined PHA characteristics at the time of random assignment and found that, as expected from an evaluation design that uses random assignment, there were no more statistically significant differences between the Treatment and Control PHAs than would be expected by chance. Appendix B of this report describes the balance testing reported by Turnham et al. (2021) and adds new balance tests using information from PHAs’ MTW applications in 2018–19, which show no statistically significant differences in plans for using the 16 waivers asked about on the application.

• Experimental analysis: comparing the average outcomes of the 33 PHAs randomly assigned to the treatment group (Treatment PHAs) versus the average outcomes of the 10 PHAs assigned to the control group (Control PHAs). This experimental analysis, leveraging the RCT design, will assess the effect of MTW flexibility on the statutory objectives of cost-effectiveness, self-sufficiency, and housing choice. We refer to these 43 PHAs as the “RCT PHAs.” 16

• Quasi-experimental analysis: comparing the average outcomes of the Treatment PHAs to the average outcomes of a matched comparison group of 99 PHAs (Comparison PHAs). During the research design phase of this evaluation in 2020, HUD and the evaluation team recognized that the small size of the control group ( n= 10) limits the impact study’s ability to analyze the effect of MTW

for certain types of PHAs (such as those with only voucher programs) or for subgroups of PHAs implementing the same type of program and policy changes. To supplement the findings of the experimental analysis, the evaluation team selected a matched comparison group of 99 PHAs similar in observable characteristics to the Treatment PHAs. (The comparison group does not include the 10

PHAs in the control group.) This quasi-experimental design (QED) and its larger comparison group allows analyses of MTW outcomes and subgroup differences. 17 We refer to the 132 PHAs (33

Treatment PHAs plus 99 Comparison PHAs) as the “QED PHAs.”

Appendix A lists the Treatment PHAs, Control PHAs, and Comparison PHAs.

The experimental design permits unbiased analysis of the impact of MTW by comparing outcomes for the Treatment and Control PHAs. However, the experiment’s statistical power to detect the impact is limited by the small number of Control PHAs. The quasi-experimental design’s larger study sample substantially improves the impact study’s ability to detect statistically significant differences in outcomes between MTW and non-MTW PHAs.

15 See Chapter 4 for discussion and specification of the confirmatory and exploratory tests of these outcomes.

16 Two of the 33 PHAs offered the opportunity to apply for MTW designation under first cohort of expansion agencies did not complete their application, and thus did not receive MTW designation. Therefore, the impact measured by comparisons between the Treatment PHAs and Control PHAs is technically “the impact of the opportunity to apply for MTW designation.” In this report we shorten the phrase to “the impact of MTW

designation,” but the experimental analysis will always compare the full set of 33 Treatment PHAs to the 10

Control PHAs.

17 Turnham et al. (2021) describe the approach to selecting the comparison group PHAs in more detail and the results of baseline balance tests, finding no statistically significant differences between the Treatment PHAs and the Comparison PHAs. See Appendix B of this Baseline Report for more details.

Baseline Report – Evaluating MTW Flexibility for Smaller PHAs ▌pg. 9

Chapter 1: Introduction to the MTW Flexibility for Smaller PHAs Evaluation We refer to the analyses based on the RCT PHAs as experimental, connoting a higher degree of confidence in the impact estimates; and the analyses based on the Comparison PHAs as quasi-experimental, to be interpreted with more caution. That the Comparison PHAs did not elect to apply for MTW makes it possible that they are systematically different on unobservable characteristics from the Treatment PHAs. Any systematic differences could introduce bias into the estimates. Therefore, the findings from the larger QED sample are less definitive than findings from the smaller RCT sample about whether MTW designation is the sole reason for any impact observed.

The QED analysis will supplement, but not replace, the RCT analysis. One challenge with running concurrent RCT and QED analyses is that results may conflict, and therefore be difficult to interpret. We will report both sets of findings but consider the RCT findings the strongest evidence. If the QED

findings generally agree with the RCT findings on the direction of impacts, that result will strengthen our confidence in using the QED approach for subgroup analysis. The Research Design/Data Collection and Analysis Plan provides more detail on our planned methods for examining the complementarity of the RCT and QED results (see Turnham et al., 2021).

The outcomes analyzed in this Baseline Report and in future Annual Reports are based primarily on HUD

administrative data: public housing development sizes and tenant data on neighborhood choice and self-sufficiency from the Inventory Management System/PIH Information Center (PIC); PHA cost data from the Financial Data Schedule (FDS); PHA voucher lease and budget data from the Voucher Management System (VMS); public housing development inspection scores from HUD’s Real Estate Assessment Center (REAC); and MTW activities from the MTW Supplement. In addition, we use earnings data from the National Directory of New Hires (NDNH) and data on neighborhood poverty rates, income, and rents from the American Community Survey. 18

18 See Turnham et al. (2021) for more detail on data sources used in this evaluation.

Baseline Report – Evaluating MTW Flexibility for Smaller PHAs ▌pg. 10

2. The First MTW Expansion Cohort in Context

This chapter considers the evaluation of MTW flexibility for smaller PHAs in the context of the existing research literature on MTW and considers the Flexibility Cohort PHAs in the context of what we know about all smaller PHAs that were eligible to participate. Why might this evaluation identify impacts of MTW designation that are different than what has already been found about large PHAs? Do we expect the findings from this evaluation to likely represent average changes across all PHAs if all smaller PHAs were to receive MTW designation? Section 2.1 presents the research to date; Section 2.2 reflects on whether the research sample is representative of all smaller PHAs that were eligible to participate in MTW.

2.1 Prior Research on MTW Outcomes

Prior research has explored how PHAs use MTW flexibility to pursue the program’s three statutory objectives and the impact of these activities on tenant outcomes. To date, such research has relied on descriptive studies of program implementation and non-experimental studies of the impact of MTW on PHA and tenant outcomes. The evaluations that HUD is sponsoring for the MTW expansion cohorts—

including this evaluation—will provide the first experimental analyses of the impact of PHA policy and program changes made possible by MTW flexibility.

A recent study that categorized the activities of the Initial MTW PHAs found that they use funding flexibility and waivers widely to pursue all three statutory objectives (cost-effectiveness, self-sufficiency, and housing choice). In-depth interviews with eight MTW PHAs found that most funding that was moved was shifted out of the HCV program and into capital projects (Levy et al., 2020).

Several studies draw on interviews with PHA staff and find that most Initial MTW PHAs implement changes designed to reduce costs or improve cost-effectiveness (Abravanel et al., 2004; Khadduri et al., 2014; Webb et al., 2015). These types of changes are relatively easy to implement and generally involve reducing the complexity of rent calculations, reducing the frequency of income recertifications, or reducing the number of housing inspections the PHA conducts. However, these changes generally did not reduce costs. For example, a 2017 study evaluated MTW PHA and tenant outcomes using a matched comparison group of non-MTW PHAs. It found that MTW PHAs do not appear to be more cost-efficient than their peers (Buron et al., 2017). A recent cost study used 15 years of historical data to study MTW

cost-effectiveness. That study found that MTW status is associated with an increase in HUD funding, an increase in the number of households receiving assistance, a significant increase in financial reserves per household, but no significant change in cost per assisted household. 19 The study concludes that MTW

19 It is not clear why an increase in HUD funding is associated with MTW status for Initial MTW PHAs. Stacy et al.

(2020) noted that the increase in the number of households assisted started the year before the PHAs were designated MTW agencies. This finding on households assisted may at least partially explain the increase in funding. Stacy et al. wrote: MTW agencies are funded based on the number of households they were assisting when they joined the demonstration. Thus, agencies could increase the base funding in their MTW contracts by increasing the number of assisted households they served in the year before they joined MTW. Additional qualitative data collection with MTW agency staff would be useful to determine if they were intentionally serving more households in anticipation of their MTW contracts. (p. 213)

Baseline Report – Evaluating MTW Flexibility for Smaller PHAs ▌pg. 11

Chapter 2: Research Sample

PHAs were able to realize some cost-efficiencies in areas other than subsidy payments on behalf of households, although it did not identify the specific activities that reduced costs (Stacy et al., 2020).

Efforts to promote self-sufficiency were less common in the early days of the MTW demonstration but now are more prevalent, including changes to rent rules to promote employment, new self-sufficiency services for tenants, and work requirements as a condition of receiving assistance or the implementation of time-limited assistance. A non-experimental evaluation of the impact of MTW found suggestive evidence of positive impacts on several measures of self-sufficiency and housing choice. These impacts include an increase in the share of all households served by the PHA that had been newly admitted in the past year, an increase in the share of households that experienced income gains, and an increase in the share of households that left assistance after experiencing income gains (Treskon et al., 2021). Other research has found that MTW PHAs tend to achieve better outcomes than their peers on activities related to self-sufficiency (Buron et al., 2017).

Research that has examined how PHAs use MTW flexibility to increase housing choice found that nearly all PHAs had implemented or planned to implement at least one activity that aimed to increase housing choice in their annual MTW Plans as of 2015 (Galvez et al., 2018). Many of these activities related to increasing the number of units available through new development or adjusting policies related to occupancy, admission, and inspections. About half of MTW PHAs used their flexibility to expand housing choice by increasing the use of project-based vouchers (PBVs). However, few PHAs pursued activities that explicitly aimed to increase families’ geographic mobility or neighborhood choice.

A recent descriptive study found that compared with traditional PHAs, MTW PHAs serve households with similar characteristics as those in traditional PHAs; added new households to their assistance portfolios at a faster rate than the traditional PHAs; and served a larger share of households through PBVs (Galvez et al., 2021). Another study found that MTW PHAs use their funding flexibility to implement non-traditional activities to serve special populations not easily reached by traditional housing assistance—for example, families experiencing homelessness or domestic violence survivors (Buron et al., 2017).

This evaluation will fill an important knowledge gap on how smaller PHAs use MTW flexibility. Only one of the 39 Initial MTW PHAs (Keene Housing) has fewer than 1,000 units. As a result, there is little existing evidence on how smaller PHAs might use MTW flexibility. One recent study conducted in-depth interviews of staff at several smaller PHAs in Illinois about the potential benefits and challenges of MTW

designation. Several administrators identified funding flexibility and regulatory flexibility as key benefits.

However, many administrators were worried that the time and cost of applying for MTW might outweigh the potential benefits (Greenlee et al., 2018).

Smaller PHAs constitute a large share of all PHAs but a small share of all units in programs administered by PHAs. About 80 percent of the nearly 3,800 PHAs in the United States have fewer than 1,000 units; however, they have only about 20 percent of all housing units in PHA programs. 20 Smaller PHAs tend to have fewer staff, less sophisticated program management systems, and a smaller funding base with which 20 Analysis of 2009–20 data from HUD’s Picture of Subsidized Households dataset, accessed on 5/13/2021 at

https://www.huduser.gov/portal/datasets/assthsg.html.

Baseline Report – Evaluating MTW Flexibility for Smaller PHAs ▌pg. 12

Chapter 2: Research Sample

to experiment. Some are in rural areas, but about half of smaller PHAs are in metropolitan areas (Fischer and Sard, 2016).

Because of their size, smaller PHAs face unique challenges compared to larger PHAs. PHAs without sufficient scale to devote staff time to plan and implement new initiatives are less likely to take advantage of options that provide additional types of housing opportunities such as supportive housing for people with disabilities. Similarly, smaller PHAs are less able to spare staff time to develop partnerships with community agencies that could improve families’ finances or help homeless individuals navigate the housing application process and find appropriate units if they receive vouchers. Smaller PHAs tend to score lower than larger PHAs on the Section 8 Management Assessment Program, HUD’s voucher performance measurement system (Fischer and Sard, 2016). Starting in 2008, PHAs with fewer than 550

units that meet certain qualifications are exempt from annual reporting requirements.

The United States Government Accountability Office (GAO, 2012) has found that smaller PHAs are less cost-effective in administering vouchers than large PHAs are, in large part because of lesser economies of scale. Because many administrative and oversight tasks cost about the same regardless of the number of families served, smaller PHAs have higher per-family administrative costs than do larger PHAs (Turnham et al., 2015). This evaluation will provide important insight into how smaller PHAs use their MTW

flexibility to address some of these challenges.

2.2 Flexibility Cohort PHAs Compared to All Smaller PHAs

Next, we consider the extent to which the PHAs in the evaluation reflect all smaller high-performing PHAs. At the time of randomization, we identified 3,725 PHAs that were administering a combined 1,000

or fewer HCV and public housing units and that were in the continental United States (excluding Alaska, Hawaii, Puerto Rico, and other territories because these regions were not represented in the set of PHAs that applied to participate in the first cohort of the MTW expansion). Of the 3,725 PHAs that met the size requirement to be eligible to apply for this first cohort of the MTW expansion, only 55 percent (2,065

PHAs) were also high-performers (and in good standing) and thus eligible to participate in the demonstration. This implies that 2,022 smaller PHAs were eligible to participate in this cohort of the MTW expansion but chose not to apply to participate. So, the question is, how does our RCT sample of 43 PHAs that chose to apply for MTW compare to the full set of 2,065 smaller PHAs that were eligible to apply, but chose not to? 21

The 43 RCT PHAs differ from the set of all smaller PHAs eligible for this MTW cohort in several ways, as shown in exhibit 2-1. PHAs that applied were more likely to be in metropolitan areas: about 70 percent of the RCT PHAs versus 47 percent. Correlated with their metropolitan location, the RCT PHAs are in areas with higher fair market rents ($1,008 per month versus $908 per month). RCT PHAs have more units on average (525 versus 260), are less likely to be public housing-only PHAs, more likely to be either HCV-only or both HCV and public housing PHAs, and have higher average expenditures per unit ($657

versus $481) than all eligible PHAs. The RCT PHAs also serve a smaller share of households in which 21 We consider only the RCT sample because the QED sample is designed to mirror the RCT sample (that is, to have the same characteristics as the RCT sample). Therefore, any differences between the RCT PHAs and all smaller PHAs that were eligible would also be apparent in a comparison of the QED PHAs and all smaller PHAs that were eligible.

Baseline Report – Evaluating MTW Flexibility for Smaller PHAs ▌pg. 13

Chapter 2: Research Sample

the head or co-head is aged 62 or older or has a disability (57 percent versus 63 percent) and a higher share of households in special purpose voucher programs (7 percent versus 4 percent).

Exhibit 2-1. Comparison of RCT PHAs versus All Smaller, Eligible PHAs

All Smaller,

RCT

Eligible

PHAs

PHAs

Characteristics

(n=43)

(n=2,065)

Difference

p-Value

PHA Characteristics

Program Types

Percentage of PHAs with HCV and public housing

51.2

32.9

18.2 p.p.

.000a

Percentage of PHAs with only HCV

37.2

23.2

14.0 p.p.

Percentage of PHAs with only public housing

11.6

42.4

-30.8 p.p.

Program Size

Number of public housing units (excluding PHAs with no

246

128

118

.000

public housing units)

Number of HCV units (excluding PHAs with no HCV units)

418

283

135

.000

Number of units (HCV and/or public housing)

525

260

265

.000

Size category: fewer than 250 units 20.9

60.2

15.0 p.p.

.000a

250–499 units

11.6

20.3

-48.6 p.p.

500–749 units

34.9

12.1

14.5 p.p.

750–999 units

32.6

6.0

20.5 p.p.

Other PHA Characteristics

Occupancy rate (all programs)b

89.9

91.0

-1.1 p.p.

.400

Average HUD expenditure per unit per month

$657

$481

$176

.000

PHA headquarters is in a metropolitan area

69.8

47.0

23.8 p.p.

.002

Fair Market Rent (FMR) for 2-bedroom unit

$1,008

$908

$100

.054

80% of Area Median Income (AMI) for 4-person household

$58,222

$56,050

$2,172

.221

Ratio of 2-bedroom FMR to 80% of AMI for 4-person

0.204

0.191

0.013

.013

household

Average weekly wage for local government workers in the

$838

$766

$72

.121

county where the PHA is located

Characteristics of Assisted Households (HCV and Public Housing)

Months on the waiting list

20.3

71.2

-50.9

.317

Months since move-in

86.6

81.0

5.6

.107

Percentage of households where head or co-head is aged

57.3

63.0

-5.7 p.p.

.002

62 or older OR has a disability

Average income of nondisabled adults aged 18-61

$17,763

$16,627

$1,135

.076

Percentage of HCV households living in low-poverty

19.7

18.7

1.0 p.p.

.395

census tracts

Percentage of households in special voucher programs

7.1

4.1

3.0 p.p.

.071

Notes: All eligible PHAs (n=2,065) includes the RCT PHAs (n=43) and the eligible PHAs that did not apply for MTW (n=2,022). Two PHAs in the RCT were missing data for all 2019 Picture of Subsidized Households (POSH) metrics except for public housing and HCV unit counts; 2018

POSH data were substituted for these metrics for these two PHAs.

a p-value is from a Chi-square test instead of a t-test.

b Occupancy rate is the number of occupied units divided by the number of units available.

Key: p.p.=percentage points. AMI=Area median income.

Sources: HUD 2018 and 2019 Picture of Subsidized Households, HUD December 2019 Inventory Management System/PIH Information Center (PIC), HUD FY20 Fair Market Rents, HUD December 2019 Voucher Management System (VMS), HUD FY2019 Section 8 administrative data, Bureau of Labor Statistics 2019 Q3 local government wages for all industries, counties, and establishment sizes

Baseline Report – Evaluating MTW Flexibility for Smaller PHAs ▌pg. 14

Chapter 2: Research Sample

Although this demonstration cannot be certain whether its findings can be used to infer the impact of MTW designation on all smaller eligible PHAs and their tenants, the differences between the RCT PHAs and all smaller eligible PHAs can be used to develop hypotheses about how the findings can be extrapolated. We make the following conjectures:

• Experimental impacts on cost-effectiveness may be larger than the average impact of conferring MTW designation on all eligible PHAs because the RCT PHAs are larger than the average PHA with less than 1,000 units, and the fixed costs of operating a housing authority are a larger percentage of the budget for the “smallest” of the smaller PHAs.

• Experimental impacts on self-sufficiency may be higher than expected if all eligible PHAs received MTW designation. The RCT PHAs are more likely to be in metropolitan areas than the average PHA with less than 1,000 units, and it is thoroughly documented in the literature that poverty rates are higher, on average, in rural areas compared to metropolitan areas (Guzman et al., 2018). Employment rates and incomes are also higher in urban areas than in rural areas (USDA, 2018).

• Experimental impacts on housing choice may be either higher or lower than expected if all eligible PHAs received MTW designation. Metropolitan areas may offer more choice in neighborhoods, but whether the increased choice would be affordable to those using a housing subsidy is not clear.

Baseline Report – Evaluating MTW Flexibility for Smaller PHAs ▌pg. 15

3. Motivation for Applying for MTW and Plans for Using Its Flexibility This chapter describes PHAs’ motivations for applying for MTW designation and their intended uses of MTW flexibility. It is based on information the PHAs provided in the application process, including the MTW Application Survey and the plans they described in Part 2 of the MTW Application, and from interviews with PHA staff.

The MTW Application Survey was completed by all 43 PHAs that completed Part 1 of the application process (exhibit 1.3) and met the eligibility requirements for the first MTW cohort of expansion agencies (that is, the RCT PHAs). 22 The survey asked a variety of questions about which aspects of MTW were most important to the applicant PHAs and how they would use their flexibility if they were designated an MTW PHA. Following completion of the MTW Application Survey, the 43 PHAs were randomly assigned to the treatment or control group. HUD invited the 33 Treatment PHAs to submit Part 2 of the MTW Application, documenting their completion of the required steps to receive MTW designation and outlining their plans for using MTW waivers.

In December 2020, 31 of the 33 Treatment PHAs submitted Part 2 of the MTW Application. HUD

offered those 31 applicants—which we refer to as Flexibility Cohort PHAs— MTW designation in January 2021. When these PHAs sign the MTW ACC Amendment (expected by the fall of 2021), they will be officially designated MTW agencies. In March 2021, the evaluation team conducted (1) telephone interviews with staff from the 31 Flexibility Cohort PHAs and (2) an online survey of the 10 Control PHAs and the two Treatment PHAs that did not complete Part 2 of the application. The Flexibility Cohort interviews asked about current plans for using MTW flexibility, the motivation for their planned activity, and how plans had changed since they submitted Part 1 of their applications. The online survey asked about current or planned activities related to cost-effectiveness, self-sufficiency, and housing choice that could be undertaken by a PHA without MTW designation. 23

This chapter begins with a description of PHAs’ motivations for applying for MTW, including the factors motivating their applications and which statutory objectives were most important to them at the time of initial application (Section 3.1). The next section discusses how PHAs developed their responses to Part 2

of the MTW Application, which stakeholders were involved in the process, and PHAs’ intended use of MTW waivers and funding flexibility to support programmatic and operational changes (Section 3.2).

The third section discusses how PHAs’ plans for MTW flexibility have changed between the two parts of the application process and in response to the COVID-19 pandemic (Section 3.3). The chapter concludes with a discussion of activities within the HUD’s traditional operating regulations that Control PHAs are currently implementing or planning to undertake in the next year to improve cost-effectiveness, tenant self-sufficiency, and housing choice (Section 3.4).

22 This survey was called the MTW Expansion Cohort 1 Baseline Survey when it was fielded in 2018. We use the term Application Survey to minimize confusion between it and the baseline data collection the evaluation team conducted with PHAs following their designation as Flexibility Cohort PHAs by HUD.

23 For the PHA interview guides and additional detail about the data sources and methodology used in this data collection, refer to the evaluation’s Research Design/Data Collection and Analysis Plan (Turnham et al., 2021).

Baseline Report – Evaluating MTW Flexibility for Smaller PHAs ▌pg. 16

Chapter 3: Motivation for Applying for MTW and Plans for Using Its Flexibility Annual Reports starting in 2022 will describe how the Flexibility Cohort PHAs actually use their MTW

flexibility to address the MTW statutory objectives and local goals, what factors influence their program design choices, what challenges they face in implementing new program and policy changes, and how these program and policy changes affect PHA operations.

3.1 Motivations for Applying for the MTW Demonstration

The MTW application process required a PHA’s Executive Director to submit to HUD an initial letter of interest along with a signed resolution from the PHA’s Board of Commissioners (or equivalent governing body) approving the PHA’s decision to apply for MTW designation. The decision to apply for MTW was largely made by PHA leadership without the direct involvement of their Board of Commissioners or other community stakeholders. Most PHAs said it was an easy decision to apply, although it did come with some concerns. This section describes PHAs’ motivations for and concerns about applying for MTW.

3.1.1 Factors Motivating Application

Factors that helped motivate PHA leadership to apply for MTW designation included prior familiarity with the Initial MTW PHAs’ programs, excitement about new opportunities for programmatic innovation, and the desire for more funding flexibility. Reducing administrative burden, simplifying program rules, and increasing tenant self-sufficiency were the most frequent goals PHAs had for their MTW flexibility.

These are discussed in more detail in Section 3.2.2.

About one-third of the 31 Flexibility Cohort PHAs interviewed said that their prior familiarity with the MTW demonstration or conversations with other MTW-designated PHAs about what they were able to do

motivated their decision to apply. Some executive directors learned about MTW from association meetings such as the National Association of Housing and Redevelopment Officials (NAHRO) or HUD

listening sessions. Others were exposed to MTW through their prior work at HUD or through their relationships with staff at existing MTW PHAs, who encouraged them to apply. Others felt compelled to pursue any HUD-provided opportunity to foster innovation. One PHA reported that it applied for the flexibility that MTW would give to take advantage of opportunities or needs that came up in the future, rather than any specific planned activity.

Seven PHA interviewees said that MTW funding flexibility was the major draw for them to apply. The ability to merge HCV Housing Assistance Payment (HAP) and administrative fee funds was especially important for these smaller PHAs, where staff must be very careful about the amount spent on administration. For example, one executive director from an HCV-only PHA noted that its voucher utilization is around 85 percent, and this affects its administrative fees and ability to fund a landlord incentive program to boost lease-up rates. PHAs were also motivated by other benefits of funding flexibility, including the use of MTW funds to fill gaps in financing housing developments, expand service provision, and increase the number of PHA staff.

More than three-fourths of PHA interviewees noted some, usually minor, concern over making the decision to apply for MTW designation. The most common source of PHA concern was staff capacity and apprehension over the additional work required to roll out new policies and programs, develop new trainings, and learn new rules or tools (such as for revised tenant rent policies). A few PHA leaders also reported concerns among PHA staff or Board members about how they would handle change and anxiety about not knowing how their MTW activities would be implemented:

Baseline Report – Evaluating MTW Flexibility for Smaller PHAs ▌pg. 17

Chapter 3: Motivation for Applying for MTW and Plans for Using Its Flexibility

“My staff are still a little worried. Some staff have been here a long time, and [MTW] is

…changing things up… it’s important to maintain their engagement and allow them a voice at the table.”

– PHA Executive Director

The short timeline for getting feedback from tenants and putting together responses to Part 2 of the MTW

Application also contributed to PHAs feeling uncertain about applying for MTW. Despite these concerns, PHA leadership made it clear that the goals that PHAs hoped to accomplish through MTW designation far outweighed concerns about applying.

3.1.2 Most Important Statutory Objectives in PHAs’ Decision to Apply

As part of the initial application in 2018–19, the 43 RCT PHAs were asked to complete a survey (the MTW Application Survey) about their priorities for and planned uses of MTW. 24 This survey was for information only; the answers did not affect their chances of becoming an MTW agency. The PHAs were asked to rank how important each of the three statutory objectives was in motivating their decision to apply for MTW designation. About 91 percent of PHAs identified promoting self-sufficiency as “very important” in motivating them to apply, followed by improving cost-effectiveness (70 percent) and increasing housing choice (61 percent). Exhibit 3-1 displays these results.

Exhibit 3-1. Most Important MTW Objectives Motivating PHAs’ Initial Application

# of PHAs Identifying Objective

% of PHAs Identifying Objective

Statutory Objective

as Very Important

as Very Important

Promoting self-sufficiency

39

90.7

Improving cost-effectiveness

30

69.8

Increasing housing choice

26

60.5

Source: MTW Application Survey of 43 RCT PHAs, fielded in 2018–19

These priorities reflect specific programmatic and operational goals that motivated PHAs’ application for MTW. Goals around both the self-sufficiency and housing choice objectives included keeping tenants employed, breaking intergenerational cycles of subsidy assistance and long-term subsidy assistance, expanding service provision, increasing the number of quality affordable housing units, developing housing for specific populations, and improving landlord participation in the HCV program. Goals related to cost-effectiveness revolved around a desire to reduce administrative burden for PHA staff and simplify program rules for PHA tenants. The next section describes how PHAs intend to use MTW flexibilities to achieve these goals.

3.2 Flexibility Cohort PHAs’ Plans for Using MTW Flexibility

The 33 Treatment PHAs were invited to complete Part 2 of the MTW Application in the fall of 2020. The 31 PHAs that completed this application specified how they planned to use MTW flexibility now that 24 For detail on PHAs’ planned use of MTW waivers at this stage in the application process, please see appendix B, exhibit B-2. Section 3.2.2 discusses PHAs’ most recent intended use of MTW waivers, as documented in Part 2

of the MTW application and discussed during interviews with the evaluation team.

Baseline Report – Evaluating MTW Flexibility for Smaller PHAs ▌pg. 18

Chapter 3: Motivation for Applying for MTW and Plans for Using Its Flexibility they were closer to being designated an MTW agency. This section reports on PHAs’ plans at that step in the application process.

3.2.1 Developing a Plan for Part 2 of the MTW Application

Part 2 of the MTW Application required Treatment PHAs to develop a narrative description of how they plan to use MTW flexibility to address their programmatic and operational goals. For most Flexibility Cohort PHAs, PHA staff were responsible for shaping the development of their response to Part 2. PHA staff created their plans for using MTW flexibility by having internal discussions about current challenges, including staff and resident difficulties. They reviewed which activities staff currently spent most of their time completing and which activities staff saw a need to improve; PHA staff then brainstormed potential improvement ideas.

Some PHA staff reported reviewing every available MTW waiver to (1) determine whether it would be applicable to their agency and tenants or helpful for their goals, and (2) anticipate the expected impact if they did implement the waiver. Other PHAs read through existing MTW Plans from Initial MTW PHAs and adopted practices they wanted to emulate. In a few cases, PHA Boards, PHA residents, and community stakeholders provided important feedback in brainstorming and planning sessions. For example, one PHA decided to increase its focus on self-sufficiency after hearing interest from their local school board, city council, and PHA Board. A few PHAs hired consultants with prior MTW experience to develop their submission for Part 2 of the MTW Application, either because the PHA lacked the capacity to submit its response before the deadline, or the PHA sought to bring in additional expertise. For the majority of PHAs, staff developed and wrote the plan submitted for Part 2 of the MTW Application themselves.

Prior to submitting their response to Part 2 of the MTW Application, PHAs were required to undergo a public process which included notifying current residents of the PHA’s plans to participate in MTW, holding at least two resident hearings to collect resident feedback on the PHA’s proposed plans for MTW, and holding a public hearing on the PHA’s draft response to Part 2. 25 Following the public hearing, the PHA’s Board of Commissioners was required to review and approve the application prior to its submission to HUD.

Although PHAs presented drafts of their response to Part 2 of the MTW Application to residents, their Board of Commissioners, and other community stakeholders, asking for feedback, PHA staff reported that these parties played a minimal role in shaping the document. For about two-thirds of PHAs, residents, Boards, and community stakeholders generally supported the PHA’s plans but did not provide substantive feedback. In many cases, PHA-held sessions for resident feedback were not well attended—some PHAs attributed this to the challenges created by the COVID-19 pandemic and the inability to hold large in-person gatherings. Though PHA Board and resident feedback was largely supportive, some PHA staff noted that it was difficult to gauge these parties’ level of understanding of MTW. For example, one common concern among residents was to the name “Moving to Work.” Once PHA staff explained the larger goals of the program, they alleviated resident concerns.

25 As part of the public hearing process, the draft Part 2 MTW application response must have been made available for public review and comment for at least 30 days prior to submission to HUD. For more detail on the requirements for developing and submitting Part 2 of the MTW Application please see Section 5 of Notice PIH-2018–17: https://www.hud.gov/sites/dfiles/PIH/documents/PIH-2018-17MTWDemonstrationProgram.pdf

Baseline Report – Evaluating MTW Flexibility for Smaller PHAs ▌pg. 19

Chapter 3: Motivation for Applying for MTW and Plans for Using Its Flexibility

“Resident meetings solidified our plans [for MTW] more than changed them.”

– PHA Staff

Only two PHAs reported making changes to their draft plans in response to feedback from residents. At the first PHA, a resident expressed a need for consideration for heads of household who take care of severely disabled children. This prompted the PHA to consider exemptions from possible work requirements for caretakers. At the second, a rural PHA, staff heard during resident listening sessions that transportation was a significant challenge for seniors who may no longer be able to drive and have difficulty getting to places for appointments or to socialize. PHA staff knew that transportation was an issue, but they learned that many seniors felt isolated and stuck in their housing units because of their lack of access to transportation. The PHA is now planning to provide transportation assistance as part of MTW.

3.2.2 Current Plans for MTW Implementation

All the Flexibility Cohort PHAs (100 percent) reported plans to use MTW waivers to adjust their reexamination policies or schedules for income verification; 90 percent planned to adjust tenant rent policies. Exhibit 3-2 lists the MTW waivers PHAs intend to use, as reported in Part 2 of the MTW

Application and interviews.

Intended Use of MTW Waivers to Support the Statutory Objectives

Waivers to Support Cost Effectiveness. PHA staff considered that MTW flexibility could be instrumental in shifting staff time and resources away from program administration and toward supporting tenants, improving tenant relations, and maintaining properties. Simplifying rent calculations and reducing the number of recertifications were the two most common ways PHA leaders believed they could reduce administrative burden on their staff. Other PHAs planned to hire more staff or unify operational standards across their public housing and HCV programs to make it easier for staff to do their jobs. 26

All Flexibility Cohort PHAs planned to adjust their reexamination policies. Changes to reexamination policies include adjusting the reexamination schedule to occur biennially or triennially and allowing tenants to self-certify their assets up to a certain amount (typically no more than $50,000). PHAs anticipate these activities will free up staff time by reducing the interviews or paperwork needed to complete them. PHAs also said these policies would reduce the burden on tenants to collect and submit the required paperwork. 27

26 MTW does not provide PHAs with additional funding to hire staff. PHA staff did not indicate where the funding to pay for additional staff would be drawn from, but presumably it would come from cost savings realized from other MTW activities.

27 A few PHAs also viewed their planned changes to the reexamination schedule as supporting tenant self-sufficiency. During the interviews, two PHAs discussed plans to waive interim reexaminations for tenants whose incomes increase so that increases in income will not be disclosed until the next scheduled reexamination, which could be an additional one to two years from the last examination, depending on the timing of the change in income and the PHA’s redesigned policy. These PHAs believe this policy may incentivize some tenants to increase work effort/earnings.

Baseline Report – Evaluating MTW Flexibility for Smaller PHAs ▌pg. 20

Chapter 3: Motivation for Applying for MTW and Plans for Using Its Flexibility Exhibit 3-2. Flexibility Cohort PHAs’ Intentions to Use MTW Waivers, by Program Type

# of PHAs

% of PHAs

with a

% of PHAs

# of PHAs Planning to

Public

Planning to

with an HCV Use Waiver Housing Use Waiver

Total % of

Program

(out of all

Program

(out of all

Total # of

PHAs

Planning to PHAs with Planning to PHAs with a

PHAs

Planning to

Use Waiver

an HCV

Use Waiver

PH

Planning to Use Waiver

Waiver

(n-28)

program)

(n=18)

program) Use Waiver

(n=31)

Reexamination

28

100.0

18

100.0

31

100.0

Tenant Rent Policy

25

89.3

15

83.3

28

90.3

Landlord Leasing Incentivesa

21

75.0

--

--

21

67.7

FSS Program with MTW

Flexibility

18

64.3

8

44.4

18

58.1

Work Requirements

15

53.6

9

50.0

18

58.1

Housing Quality Standardsa

15

53.6

--

--

15

48.4

Payment Standards and Rent

Reasonablenessa

15

53.6

--

--

15

48.4

Local Non-Traditional Activities

21

75.0

15

83.3

23

74.2

PBV Program Flexibilitya

11

39.3

--

--

11

35.5

Term-Limited Assistance

7

25.0

6

33.3

9

29.0

MTW Self-Sufficiency Program

5

17.9

5

27.8

7

22.6

Short-Term Assistance

5

17.9

3

16.7

6

19.4

De-concentration of Poverty in

Public Housing Policyb

--

--

2

11.1

2

6.5

Increase Elderly Age

2

7.1

2

11.1

2

6.5

Public Housing as an Incentive

for Economic Progressb

--

--

2

11.1

2

6.5

Acquisition without Prior HUD

Approvalb

--

--

1

5.6

1

3.2

Moving-On Policyc

0

0.0

0

0.0

0

0.0

Notes: See Appendix I of the Operations Notice for more information on these waivers. Of the 31 Flexibility Cohort PHAs, 15 operate both an HCV and public housing program, 13 operate only an HCV program, and 3 operate only a public housing program. Thus, in total, 28 operate an HCV program and 18 operate a public housing program. Some of the waivers represented in the table have multiple activities that fall under that waiver. They have been aggregated so that any PHA reporting interest in one or more of the activities under a waiver category is only represented once per category.

a These MTW waivers are limited to the HCV program. Though some PHAs that operate both an HCV and public housing program indicated interest in using these waivers, they are not included in the columns for PHAs with a public housing program planning to use a waiver.

b These MTW waivers are limited to the public housing program. Though some PHAs that operate both an HCV and public housing program indicated interest in using these waivers, they are not included in the columns for PHAs with an HCV program planning to use a waiver.

c This waiver was added between the initial and final Operations Notice. It allows waivers for Housing Quality Standards’ inspections, income calculations, and tenant rents to enable individuals or families to transition from permanent supportive housing by providing mainstream housing options.

Key: FSS=Family Self-Sufficiency program. PBV=project-based voucher.

Source: Responses to Part 2 of the MTW Application and baseline interviews in March 2021 with PHA staff among the 31 Flexibility Cohort PHAs

Baseline Report – Evaluating MTW Flexibility for Smaller PHAs ▌pg. 21

Chapter 3: Motivation for Applying for MTW and Plans for Using Its Flexibility Documentation of some assets [is] hard. We’d rather spend time growing clients’ assets than having them certify them. – PHA Executive Director

Most PHAs also planned to implement activities under the tenant rent policy waiver in ways that would improve cost-effectiveness: implementing an alternative utility allowance or eliminating utility reimbursement payments; eliminating select deductions or replacing existing deduction(s) with a standard deduction; using alternate income inclusions or exclusions when conducting tenant rent calculations; and setting a higher minimum rent. 28 Many of these policies are meant to simplify the tenant rent calculations, which is expected to free up staff time for other tasks. In addition to reducing the amount of effort required to administer tenant rent policies, simpler rules also save the time staff must spend explaining the rules, policies, and calculations to tenants. These policies may also reduce the burden on tenants to produce required documentation. Some of the planned activities may reduce the average voucher rent subsidy or increase rents for public housing (thereby reducing the PHAs’ costs but increasing tenant rent burden), permitting the PHA to invest the additional revenue in other initiatives or serve more households from their waiting lists.

More than half of PHAs with an HCV program planned to make changes to the implementation of Housing Quality Standards (HQS) inspections, such as an alternative inspection schedule under which inspections are conducted less often. PHAs expect this will reduce the time staff spend conducting and tracking the inspections and save on other costs such as paper and travel. Some PHAs noted concerns about not inspecting units annually, because that could lead to needed repairs going unreported and unresolved. In future years, the unit would fail inspection and need to be reinspected. The accumulated repair needs might be so great that landlords drop out of the program rather than making them. In the long run, costs to the PHA might increase. To address these concerns, some PHAs planned to limit changes to inspection schedules to subpopulations of tenants. PHAs report that units occupied by elderly households often have few required repairs each year.

Waivers to Support Self-Sufficiency. PHAs identified a number of goals related to improving the self-sufficiency of their tenants. Some PHAs expressed a desire to serve more people by reducing the average amount of subsidy or encouraging turnover. Other PHAs said they wanted to incentivize increased earnings and savings; yet others wanted to get unemployed tenants to return to work.

Though changes to tenant rent policies were identified by most PHAs as a means to improve self-sufficiency, PHAs planned to implement various activities within this waiver category. The most common plans were for higher minimum rents, stepped rent policies, and alternative income inclusions or exclusions. PHAs noted that their current minimum rent may be so low that it acts as a disincentive for some tenants to increase their work hours or wages or to find new employment when in between jobs.

PHA interviewees believe that setting a higher minimum rent than permitted under regular program rules will encourage work effort. Similarly, stepped rents, by increasing the amount tenants are required to pay the longer they are on the program is expected to encourage work as tenants are required to take on more of the cost of market rent. Five PHAs stated that they hoped MTW could help them decrease the time 28 PHAs mentioned higher minimum rents under both the cost-effectiveness and self-sufficiency objectives.

Baseline Report – Evaluating MTW Flexibility for Smaller PHAs ▌pg. 22

Chapter 3: Motivation for Applying for MTW and Plans for Using Its Flexibility tenants spend in their public housing or HCV programs so the PHAs could serve more people and reduce their waitlists.

“[MTW activities] are designed with that younger generation in mind.… By breaking the cycle and not having our HCV children become the applicants of tomorrow, the demand [for vouchers] can be reduced or the supply better allocated to new families and fresh opportunities to break family cycles [of direct subsidy assistance].” – PHA Staff Because not all households are capable of increasing their earnings to afford a higher rent, tenant rent policies will have built-in safe harbors and may only apply to a subset of tenants such as nonelderly, nondisabled households. Activities under the alternative income inclusions or exclusions waiver are often meant to encourage work by offsetting work expenses such as transportation and uniforms. One PHA planned to create an earned income disregard of 10 percent.

More than half of PHAs planned to implement activities under one of the waiver categories that relate to operating a self-sufficiency program. The MTW Self-Sufficiency Program waiver permits MTW PHAs to operate self-sufficiency and training programs without some of the constraints of current program requirements. The Family Self-Sufficiency (FSS) program with MTW Flexibility waiver permits PHAs with existing FSS programs, or any successor programs, to be exempted from certain FSS requirements.

Activities related to these waivers frequently include plans for an alternative family selection process.

Some PHAs are even considering making participation in their self-sufficiency program mandatory for nonelderly, nondisabled adult household members. PHAs are also planning to adjust their policies to address increases in income of tenants participating in a self-sufficiency program. PHAs are considering setting policies that would allow them to disregard income increases when determining tenant rent or change the amount of money that tenants can move to escrow or savings while participating in the self-sufficiency program. These activities were largely in the initial planning stages at the time of the interviews, and PHAs are continuing to work out the details.

Slightly more than one-half of PHAs planned to foster self-sufficiency by implementing work requirements for nonelderly, nondisabled tenants aged 18 or older. To make sure a work requirement does not place an undue hardship upon tenants, HUD established safe harbor policies. Any work requirement policy designed by the PHA must be within the bounds of the safe harbors outlined in the waiver unless the PHA receives approval to waive the safe harbors. PHAs hope that work requirements will increase household income so that tenants can take on a higher share of rent and eventually not need a housing subsidy.

Waivers to Support Housing Choice. PHAs described two goals related to improving housing choice for their tenants or their communities: (1) improving landlord participation in HCV programs, and (2) increasing the number of quality affordable housing units.

Three-quarters of PHAs with HCV programs planned to implement landlord leasing incentives such as offering damage claims to landlords if a tenant leaves a unit in need of repairs, providing landlords with vacancy loss payments to encourage their continued participation in the HCV program, or providing signing bonuses (for example, one month’s rent) to encourage new landlords to begin participating in the HCV program. One PHA interviewee pointed to the need for landlord incentives to bring small private

Baseline Report – Evaluating MTW Flexibility for Smaller PHAs ▌pg. 23

Chapter 3: Motivation for Applying for MTW and Plans for Using Its Flexibility landlords back to the market after the COVID-19 pandemic, during which some small landlords were turned off from renting after rent collections dropped and eviction prohibitions made it difficult to replace tenants who were not paying. Another PHA reported many apartment complexes in its community will not lease to HCV holders. This PHA is interested in providing signing bonuses to attract new landlords, open up more housing opportunities for HCV tenants, and increase HCV lease-up rates.

About half of PHAs planned to implement changes to their payment standards by setting their payment standard at a higher percentage of the area fair market rent than permitted for PHAs without MTW

flexibility. PHAs anticipate that higher payment standards will attract new landlords and allow the PHA to support rentals in higher-cost neighborhoods, thus opening up more housing choice for tenants.

More than one-third of PHAs with an HCV program planned to use the PBV flexibility permitted to MTW PHAs to improve housing choice. Planned activities include increasing the PBV cap at the program or project level so that more units can be set aside for voucher holders willing to live in specific housing developments. Some plan to eliminate the PBV competitive selection requirement so that PBVs can be awarded directly to non-public housing properties owned by the PHA without the requirement that the PBVs be used to improve, develop, or replace a public housing property.

Plans for Local, Non-Traditional Activities

As part of the MTW waiver flexibility, PHAs are allowed to support LNT activities not allowed under the 1937 Act. As part of the waivers included in the Operations Notice, HUD approved three LNT activities: (1) rental subsidy programs provided to a third-party entity (not the landlord or tenant) that selects tenants, administers the subsidies, and provides supportive services; (2) service provision of self-sufficiency or supportive services29 not otherwise permitted under public housing or HCV programs or that are provided to low-income individuals not already receiving public housing or HCV assistance; and (3) housing development programs to acquire, renovate, or build affordable (nonpublic housing) units for low-income families.

About three-quarters of the Flexibility Cohort PHAs intend to use LNTs at some point during their tenure as an MTW agency. Many of these PHAs expressed a desire to create or expand housing development

programs, often using PHA resources for gap financing, or to provide new self-sufficiency or supportive service programs. For about one-third of PHAs, one of the motivating factors in applying for MTW was the ability to develop housing to address issues such as a high percentage of old and substandard housing, a shortage of multi-bedroom affordable housing for families, or a lack of one-bedroom affordable housing for individuals. Providing housing for people experiencing homelessness was an important goal for some PHAs.

Other PHAs were interested in providing education and training to help their tenants return to work. A few PHAs planned to use MTW as a springboard for strengthening partnerships with local educational institutions and workforce boards to train tenants to fill available jobs in the region, or even for the PHA to develop its own educational programs. This may include training front-line staff at partner organizations to better respond to the needs of the PHA’s tenants. However, most PHAs did not have 29 The final MTW Operations Notice in 2020 defines these eligible services as HUD-approved “services for participants of other PHA-owned or managed affordable housing that is not public housing or HCV assistance; services for low-income non-participants; services and/or incentives to attract applicants to developments, or portions thereof, which can be difficult to market; or supportive services.”

Baseline Report – Evaluating MTW Flexibility for Smaller PHAs ▌pg. 24

Chapter 3: Motivation for Applying for MTW and Plans for Using Its Flexibility concrete plans for the LNTs it would implement and planned to wait to implement LNTs until after it was well underway with using its MTW authority.

3.2.3 MTW Funding Flexibility

One of the core components of MTW flexibility is the ability for PHAs to merge four funding streams: (1) public housing operating funds, (2) public housing capital funds, (3) HCV HAP (subsidy payment) funds, and (4) their HCV administrative fees to create an “MTW Fund” as shown in exhibit 3-3.

Exhibit 3-3. MTW Fund Flexibility

Source: Modified from graphic provided by HUD’s Office of Policy Development and Research In the MTW Application Survey, the 43 RCT PHAs were asked whether they would use this MTW

funding flexibility if they were designated an MTW agency. The results of the survey are displayed in exhibit 3-4 for the 33 Treatment PHAs only. 30 Most (81 percent) intended to use funding flexibility. 31

This varied slightly depending on whether PHAs operated a public housing program, an HCV program, or both. For example, among PHAs that operated only an HCV program, about 70 percent intended to use funding flexibility if designated an MTW agency, whereas 100 percent of the PHAs with only public housing programs intended to use their operating and capital funds flexibly.

30 Exhibit 3-5 shows the responses of Control PHAs.

31 As indicated by a response of “yes” or “maybe” to the question “Does the PHA plan to use MTW funding flexibility?” on the MTW Application Survey.

Baseline Report – Evaluating MTW Flexibility for Smaller PHAs ▌pg. 25

Chapter 3: Motivation for Applying for MTW and Plans for Using Its Flexibility Exhibit 3-4. Treatment PHAs’ Intent to Use MTW Funding Flexibility

PHA Intends

Treatment PHAs with

PHAs with Only

PHAs with Only PH

All Treatment

to Use MTW

HCV and PH

HCV Programs

Programs

PHAs

Funding

Programs

Flexibility

#

%

#

%

#

%

#

%

Yes

11

64.7

7

53.8

3

100.0

21

63.6

Maybe

4

23.5

2

15.4

0

0.0

6

18.2

No

2

11.8

4

30.8

0

0.0

6

18.2

Notes: Among the 33 Treatment PHAs, 17 operated both an HCV and public housing program, 13 operated only an HCV program, and 3

operated only a public housing program.

Source: MTW Application Survey of 43 RCT PHAs fielded in 2018–19; responses of 33 Treatment PHAs only

Among the 31 Flexibility Cohort PHAs, plans for using funding flexibility had not changed as of their responses to Part 2 of the MTW Application. Most of them (27 of 31, or 87 percent) expected to use their funding streams flexibly to help achieve one of the three statutory objectives or to help improve PHA operations during their participation in MTW. However, more than one-half (52 percent) either did not expect to use this funding flexibility in the first year or did not have a definite timeline for their plans. A few of the PHAs decided to wait to use this funding flexibility until after they implemented their initial MTW-related changes, or they were not sure whether they would ever use it.

Using Funding Flexibility to Achieve Statutory Objectives

Flexibility Cohort PHAs that do plan to use funding flexibility expect to create an MTW fund to support their activities and cover MTW expenses from the single fund. Though most PHAs did not yet have definite plans, PHAs expect that such flexibility will help them meet the MTW Statutory Objectives.

Cost-effectiveness. PHA staff, excited about the opportunity to use funding flexibility to improve PHA cost-effectiveness, were interested in standardizing programs to operate more efficiently or directing funding to supplement programs at different times, depending on need. PHAs reported that with more ability to tailor funding to programs that need it, they would be able to improve overall operations.

“We had hoped to combine all of our assets, with all the money in one pot and then we pay for things as needed…this will allow us to administer the properties more equally.”

– PHA Executive Director

Self-sufficiency. Only a few PHAs reported plans to use funding flexibility to support self-sufficiency efforts. Of the PHAs that did, staff plan to use funding flexibility to devote more resources to self-sufficiency supports. For example, one PHA would like to use HCV and public housing funds to create a new self-sufficiency center and job development programs.

Housing choice. More than a third of PHAs planned to use funding flexibility to promote housing choice.

Of those PHAs, about one-half expect to use funding flexibility to purchase property or support the development of new housing. The other one-half plan to offer or increase landlord incentives. PHAs that have plans to acquire new or develop existing property are considering using some of their funds as gap financing for their development activities. No PHA has immediate plans to develop new housing during its first year as an MTW agency. More immediately, PHAs may use funding flexibility to allocate more

Baseline Report – Evaluating MTW Flexibility for Smaller PHAs ▌pg. 26

Chapter 3: Motivation for Applying for MTW and Plans for Using Its Flexibility money to HCV programs to serve more households on the waiting lists. PHAs that planned to use funding flexibility to offer new or supplement landlord incentives often made this their first planned activity.

“We are thinking about moving some of the funds from public housing to Section 8 so that we can serve more voucher holders because currently our funds are maxed out just on getting the rents for voucher holders. Then we can also possibly serve more households ... and help tenants actually stay in the local area.” – PHA Executive Director

Using Funding Flexibility for Additional Operational Improvements

In addition to using funding flexibility to pursue MTW’s three statutory objectives, PHAs may use the funding flexibility to add staff. No PHA planned to use their funding flexibility to hire additional staff in the first year, but five PHAs are considering adding MTW-specific staff over time such as an MTW

Coordinator to develop programming and help tenants navigate MTW changes.

Three PHAs planned to use their funding flexibility to provide additional support to specific populations.

One PHA planned to fund rental assistance or utility deposits for young people aging out of foster care.

Another PHA planned to help tenants to meet a new work requirement by providing funds to help with work-related expenses such as childcare and transportation costs. A third PHA was considering using funds for a transportation service for seniors.

Uncertainties around Funding Flexibility

Half of the PHAs either did not plan to use their funding flexibility in the first year or were unsure about whether they would ever use funding flexibility. PHAs had three main reasons for waiting as discussed below.

Many PHAs decided to prioritize implementing MTW waivers in the first year and understand how those changes affected their operations before deciding whether to use funding flexibility. One PHA described MTW waivers as having the potential to create cascading changes, so it wanted to observe the impact of those waivers and any unintended consequences before deciding how to use funding flexibility. Another PHA considered pursuing funding flexibility in the first year but decided to wait until it understood the financial implications of using MTW waivers, which could lead either to cost savings or to additional costs within each funding stream.

“We do not have any plans to use funding flexibility this year because it is all changing too much. In the same way we will take… small bites that won’t choke us, we have no plans to use funding flexibility this year.” – PHA Executive Director

A few PHAs interested in funding flexibility did not want to pursue the option while they had stable, healthy budgets. These PHAs are open to using funding flexibility in the future, but they expect to have no budget challenges during the first few years and thus did not see the benefit of pursuing it during that period.

Baseline Report – Evaluating MTW Flexibility for Smaller PHAs ▌pg. 27

Chapter 3: Motivation for Applying for MTW and Plans for Using Its Flexibility About a third of interviewed PHAs reported that they did not understand funding flexibility (as of March 2021) and wanted to learn more about the option before acting. 32 Some of these PHAs operate only one type of program and did not see how funding flexibility would help improve their operations. Other PHA staff reported that the MTW implementation process revealed that they did not yet understand all the options available to them. They expect to have a clearer idea in the future about whether they want to use funding flexibility.

3.2.4 New Funding Sources

PHAs reported that they expect participation in MTW to attract new funding options or increase existing partnerships, including new housing development partners and workforce development and job training partners. However, no PHA had identified new funding sources as of the March 2021 interviews.

3.3 Changes in Flexibility Cohort PHAs’ Plans over Time

Between completing the MTW Application Survey in 2018–19 and Part 2 of the MTW Application in late 2020, a few PHAs shifted their priorities for how they would use MTW flexibility. These changes did not reflect any major change to the underlying rationale for conducting MTW activities or to PHAs’ main programmatic or operational goals. Instead, PHAs were still largely motivated by the desire to reduce administrative burden, simplify program rules, promote self-sufficiency, increase affordable housing stock, and improve landlord participation in the HCV program.

Among the 31 Flexibility Cohort PHAs, improving cost-effectiveness was the most important statutory objective for 15 PHAs, followed by increasing self-sufficiency (9 PHAs) and improving housing choice (7 PHAs). PHAs often prioritized improving cost-effectiveness because it could serve a dual purpose by (potentially) reducing their costs and therefore freeing up resources to invest in other objectives such as providing additional support services to tenants or improving housing quality. Self-sufficiency remained the top priority for slightly less than one-third of PHAs.

Self-sufficiency was very important, but we knew to do that we had to reduce costs.

– PHA Executive Director

Between the time that the MTW Application Survey was fielded and PHAs developed their response to Part 2 of the MTW Application, HUD changed the final list of approved MTW waivers. The final Operations Notice included many of the initial waivers from the MTW Self-Sufficiency Program category into a new set of MTW waivers for PHAs already operating an FSS program. It eliminated waivers related to public housing leases, but added waivers related to housing quality standards, moving-on policy, de-concentration of poverty in public housing, public housing as an incentive for economic progress (such as by extending the amount of time households can remain in public housing while they are over income-eligibility requirements), and acquisition without prior HUD approval.

As a result of these changes and additional considerations about how to use their MTW flexibility, by the time Flexibility Cohort PHAs had drafted their responses to Part 2 of the MTW Application, all had made 32 Some of these PHAs intended to use funding flexibility when they completed the Part 1 of the MTW Application but were uncertain as to how they would do so at the time they completed Part 2.

Baseline Report – Evaluating MTW Flexibility for Smaller PHAs ▌pg. 28

Chapter 3: Motivation for Applying for MTW and Plans for Using Its Flexibility some adjustments to the waivers and activities they planned to implement. Twenty-eight PHAs dropped at least one of the waiver activities their initial application had indicated they would pursue. All 31 PHAs added at least one waiver or activity, including some of the additional waiver activities approved by HUD. Other changes to PHAs’ planned use of waivers reflected a narrowing of focus, once staff thought more about which activities would best suit the PHA’s goals and the needs of its tenants, reviewed the experience of Initial MTW PHAs, and assessed what they believed could be accomplished in the near term.

The COVID-19 pandemic also affected PHAs’ plans. A few PHAs noted that policies they had planned as of the MTW Application Survey—for example, changes to the rent rules that would increase tenant rent or making participation in their FSS program mandatory—did not seem feasible during the pandemic and were removed.

“The [baseline] survey was ‘I can have it all.’ The [MTW] Plan was the more manageable option and what we expect to actually be able to implement…. The survey was like the whole buffet, but … reality hits when you talk to your administrative staff. We brainstormed on what should work and how we should do it.… This first group [of activities] will test [our] theories.

– PHA Executive Director

One Flexibility Cohort PHA that originally planned to make participation in its FSS program mandatory later decided not to, at least not during its first year of MTW implementation. Another PHA originally planned to prioritize reducing administrative burden, but then shifted its priority to landlord engagement and recruitment. This PHA is in an area that is expected to see a lot of growth in the next few years, and it has already begun to see some positive returns from its pre-MTW efforts to engage landlords.

Few PHAs changed their intention to pursue funding flexibility. However, two PHAs that had provided detailed plans for using funding flexibility in their responses to Part 2 of the MTW Application, then reported in the PHA interviews that they no longer had plans to do so. For one of these PHAs, a staff change affected its plans for MTW.

COVID-19 Waivers

Between completion of the MTW Application Survey in 2019 and development of their responses to Part 2 of the MTW Application in late 2020, the COVID-19 pandemic emerged in the United States. As life drastically changed to ensure public safety and contain the spread of the virus, HUD issued all PHAs a number of new COVID-19 waivers. 33 These temporary waivers were issued to allow PHAs to continue operations while adhering to state and local stay-at-home orders and guidance around social distancing.

For example, under the COVID-19 waivers, PHAs may delay or in some cases waive annual income 33 The Coronavirus Aid, Relief, and Economic Security (CARES) Act provided HUD the authority to waive or modify statutory and regulatory requirements for the Public Housing and HCV programs. HUD implemented a number of modifications and waivers (which we refer to as “COVID-19 waivers”) beginning in April 2020 and continued to issue additional waivers throughout the pandemic. Many of these COVID-19 waivers are currently in place through the end of 2021, but they may be further extended. A full list of the COVID-19 waivers and alternative requirements for public housing and HCV programs can be found on HUD’s website:

https://www.hud.gov/sites/dfiles/PIH/documents/PIH2021-14.pdf

Baseline Report – Evaluating MTW Flexibility for Smaller PHAs ▌pg. 29

Chapter 3: Motivation for Applying for MTW and Plans for Using Its Flexibility recertifications or housing inspections or allow for some type of self-certification—for example, allowing landlords to self-certify that their rental units have no life-threatening deficiencies. Many of these COVID-19 waivers give flexibility similar to that permitted under the MTW waivers. However, they are time-limited and set to expire once pandemic-related restrictions on day-to-day life are eased.

During the interviews with Flexibility Cohort PHAs, the evaluation team discussed with PHAs whether they intended to seek HUD approval to continue to use any of the COVID-19 waivers after they are set to expire. Nine of the 31 PHAs (29 percent) said they were not using any of the COVID-19 waivers and thus had no plans to seek approval of these waivers in MTW. Among the 22 PHAs that were using the COVID-19 waivers, 13 (59 percent) planned to seek approval to continue using their MTW flexibility.

They reported liking the benefits of allowing self-inspections, inspection waivers, self-certification of income for reexaminations, conducting virtual/video inspections rather than in-person, and allowing for electronic signatures and other paperless efforts.

3.4 Control PHAs’ Plans for Using MTW Flexibility

The MTW Application Survey showed that as of 2018–19, the intentions of what would become the Control PHAs were very similar to the Treatment PHAs’ (exhibit 3-5). In their responses to the MTW

Application Survey, Control PHAs had similar desires for using funding flexibility as Treatment PHAs: the ability to distribute funds based on the specific needs of the PHA, such as shifting more funds to HCV

administration, expanding self-sufficiency programs and supports, and promoting affordable housing within the community.

Exhibit 3-5. RCT PHAs’ Intent to Use MTW Funding Flexibility

PHA Intends

to Use MTW

PHAs with HCV and

PHAs with Only

PHAs with Only PH

Funding

PH Programs

HCV Programs

Programs

All RCT PHAs

Flexibility

#

%

#

%

#

%

#

%

Treatment PHAs (n=33)

Yes

11

64.7

7

53.8

3

100.0

21

63.6

Maybe

4

23.5

2

15.4

0

0.0

6

18.2

No

2

11.8

4

30.8

0

0.0

6

18.2

Control PHAs (n=10)

Yes

3

60.0

2

66.7

1

50.0

6

60.0

Maybe

1

20.0

0

0.0

1

50.0

2

20.0

No

1

20.0

1

33.3

0

0.0

2

20.0

Notes: Among the 33 Treatment PHAs, 17 operated both an HCV and public housing program, 13 operated only an HCV program, and 3

operated only a public housing program. Of the 10 Control PHAs, 5 operated both an HCV and public housing program, 3 operated only an HCV program, and 2 operated only a public housing program.

Source: MTW Application Survey of 43 RCT PHAs, fielded in 2018–19

In the spring of 2021, we asked executive directors from the 10 Control PHAs and the two Treatment PHAs that did not move forward with Part 2 of the application to complete a short online survey. The survey asked the PHAs to describe their initial objectives in applying for MTW, whether they planned to apply for future cohorts of MTW, and to detail any activities they were undertaking or planned to undertake in the next year to support objectives in the areas of improving cost-effectiveness, increasing self-sufficiency, or improving housing choice.

Baseline Report – Evaluating MTW Flexibility for Smaller PHAs ▌pg. 30

Chapter 3: Motivation for Applying for MTW and Plans for Using Its Flexibility Eight of the 12 PHAs completed the survey: 7 of the 10 Control PHAs and one of the two Treatment PHAs. 34 Of those eight, four indicated that they were currently undertaking or planned to undertake activities in support of the three statutory objectives. We asked those four PHAs additional follow-up questions via phone or email; however, only three responded to the request. This section reports the findings from that effort.

The Control PHAs are not precluded from implementing changes to their programs or policies that are compliant with the 1937 Act, seeking approval from HUD to implement new policies, or applying for future cohorts of MTW. Some Control PHAs may implement policy changes in the coming year or apply for and receive MTW designation in a future cohort. Indeed, five of the eight PHAs that completed the survey indicated a sustained interest in applying for MTW in future. As part of the evaluation, we will monitor changes to activities that Control PHAs implement, as well as whether any of the Control PHAs are designated an MTW agency in a future cohort.

On the survey, many of the Control PHAs reported similar objectives as their Treatment PHA counterparts had when applying for MTW designation. Executive directors reported a desire to improve cost-effectiveness and self-sufficiency. Staff reported objectives such as reducing staff burden by simplifying tenant rent calculations, improving administrative functions, and fostering self-sufficiency among tenants. Control PHAs were also interested in the potential benefits that MTW funding flexibility could have on the efficiency of their operations and their ability to maintain administrative stability when faced with fluctuating revenue. These Control PHAs do not have MTW flexibility, but they can make changes to their programs and operations within the scope of the 1937 Act and associated rules and regulations. The scope of such changes would be more limited than what would be available to them under MTW, however.

Since submitting their initial applications for MTW, three Control PHAs have already undertaken activities within the traditional program rules and regulations to support their objectives. 35

Two of these PHAs have begun efforts to increase their PHA’s cost-effectiveness by reorganizing staff responsibilities, making changes to their waitlist management, or shifting from annual to biennial inspections for HCV tenants. The PHA that shifted to biennial inspections noted that it has already begun to see some cost savings from doing so (lower transportation, postage, and paper costs). However, similar to concerns among Treatment PHAs, this PHA is concerned that the cost savings could be reduced or eliminated in future years by the need for re-inspections if more units begin to fail inspection because of 34 The PHA that did not complete Part 2 of the application experienced a change in leadership between the first and second step of the MTW application and chose to focus its efforts on handling the COVID-19 pandemic before bringing on a new program that required additional changes to its operating procedures. The PHA remains open to pursuing MTW designation in the future.

35 The Housing Opportunity and Modernization Act of 2016 (HOTMA) provided some additional flexibilities for non-MTW PHAs, such as allowing: full reexaminations to be done every three years instead of annually for families on fixed incomes; easing move-in rules for non-life-threatening failures of HQS or for units that passed inspection in last two years; PBVs to exceed the 20 percent cap of vouchers if additional units are used for certain targeted populations; and reducing interim reexaminations for households whose earnings grow in certain situations (and removed Earned Income Disregard allowance). For HOTMA law, see:

https://www.congress.gov/114/plaws/publ201/PLAW-114publ201.pdf. For HUD-proposed rules following HOTMA, see: https://www.federalregister.gov/documents/2020/10/08/2020-21400/housing-opportunity-through-

modernization-act-of-2016-housing-choice-voucher-hcv-and-project-based

Baseline Report – Evaluating MTW Flexibility for Smaller PHAs ▌pg. 31

Chapter 3: Motivation for Applying for MTW and Plans for Using Its Flexibility the longer time between them. The third PHA is planning to increase its cost-effectiveness by improving internal controls around voucher sizes. To do this, the PHA will audit tenant files once a month to look for tenants who have had a decrease in household size but no corresponding reduction in their subsidy’s supported unit size. Fixing these administrative oversights will bring down the PHA’s HAP payments by increasing the voucher holders’ total tenant payments.

To increase cost-effectiveness and improve housing choice, one PHA plans to begin participating in the Rental Assistance Demonstration (RAD). RAD allows PHAs to convert existing public housing units into project-based Section 8 (HCV) programs, which allows PHAs to access more stable funding and gain access to additional lending sources they can use to perform needed rehabilitation work on the buildings, thus improving housing conditions for tenants.

To increase housing choice among residents in their community, another PHA began to administer a mainstream voucher program (for nonelderly individuals with a disability). 36 That same PHA is also hoping to improve housing choice for community residents by developing new partnerships with housing developers to increase the number of affordable housing units developed in its area. Another PHA hopes to increase choice by conducting more landlord outreach and providing landlord incentives to rent to HCV recipients. This latter effort also included reaching out to some of the larger apartment complexes in its area to negotiate rents that would be affordable to voucher holders.

One Control PHA reported plans to expand efforts to increase self-sufficiency by encouraging more participation in its FSS program.

36 Mainstream vouchers are not included in the MTW demonstration program. However, some aspects of MTW

flexibility can be applied to mainstream vouchers (MTW Operations Notice, sect. VI.9).

Baseline Report – Evaluating MTW Flexibility for Smaller PHAs ▌pg. 32

4. Baseline Values of Outcome Variables

This chapter presents the definitions and baseline values of outcome measures that will be used to estimate the impact of MTW designation in Annual Reports starting in 2022. Sections 4.1, 4.2, 4.3, and 4.4 define outcomes for cost-effectiveness, self-sufficiency, housing choice, and other tenant outcomes, respectively. These sections report the baseline values of their outcomes—that is, their values in 2020, prior to implementation of MTW flexibility by the Flexibility Cohort PHAs. We describe the outcome measures for both the confirmatory and the exploratory hypothesis tests.

To avoid the appearance of “cherry-picking” the evaluation’s results, the Research Design/Data Collection and Analysis Plan pre-specified one confirmatory hypothesis test for each statutory objective (see Turnham et al., 2021). 37 All other hypothesis tests will be exploratory.

• The confirmatory hypothesis test provides the best single measure of whether Treatment PHAs have achieved that statutory objective. A statistically significant finding from the confirmatory hypothesis test provides definitive evidence that MTW has a nonzero impact on that objective.

• The exploratory hypothesis tests provide a more complete picture of the potential impact of MTW

on the objective. Outcomes for the exploratory hypotheses can be alternative measures of the confirmatory outcome, outcomes measured for portions of PHAs’ programs or participants served, or components of a multi-part confirmatory outcome measure. A statistically significant finding from an exploratory hypothesis test provides suggestive, but not definitive, evidence that MTW has a nonzero impact on that objective.

All measures presented are means of PHA-level data. Some measures, such as program costs, are first calculated at the PHA level; others are initially calculated at the household level. For example, for earnings in the last four quarters, we first compute the mean earnings in the last four quarters for tenants served at a specific PHA. Next, we compute the mean across all the PHAs’ means. The reason for this analytic choice is that the PHA-level averages will be used as baseline covariates in the analysis. Even for outcomes measured at the individual level, such as earnings, the corresponding baseline covariate is always the PHA-level average of that outcome, because the sample of households will change as households enter and exit the PHA during the study period.

37 We select just one hypothesis test as confirmatory for each statutory objective (“outcome domain”) because testing more than one confirmatory outcome would require that we adjust our statistical tests to overcome the

“multiple comparisons” problem. Such adjusting is punishing in terms of statistical power: the more outcomes, the higher the statistical bar, and therefore the larger the sample size needed to detect an effect. Given the small sample size for this study, using just one confirmatory outcome per outcome domain maximizes statistical power.

We do not correct exploratory hypothesis tests for multiple comparisons bias, so we can examine multiple exploratory outcomes.

We use “confirmatory outcome” and “exploratory outcome” as shorthand to denote which impact estimates we will test using a confirmatory hypothesis test and which we will be test using an exploratory hypothesis test.

Always we are testing this hypothesis: What is the average impact of the offer of MTW designation on

[outcome]? ”

Baseline Report – Evaluating MTW Flexibility for Smaller PHAs ▌pg. 33

Chapter 4: Baseline Values of Outcome Variables

The baseline values are reported for the RCT PHAs (Treatment and Control PHAs). We do not report baseline values for the Comparison PHAs in the QED analysis, because the QED sample is designed to have the same characteristics as the Treatment PHAs.

Appendix C provides more details on the construction of each outcome measure. The appendix sections parallel those in this chapter (for example, the variables in exhibit 4-1 on cost-effectiveness are defined in detail in exhibit C-1, and so on).

4.1 PHA Cost-Effectiveness

What matters most for the cost-effectiveness objective is whether MTW PHAs can provide a similar level of housing assistance at a lower cost than do non-MTW PHAs, a question best evaluated by comparing PHA expenditures per household per month. The MTW statutory objective related to cost effectiveness is

“to reduce cost and achieve greater cost effectiveness in Federal expenditures” (1996 Act). The use of MTW waivers may allow PHAs to pursue increases in total revenue or cash reserves while also pursuing lower operating expenses and HAP expenditures. Using a comparative time series analysis of the original Moving to Work demonstration, Stacy et al. (2020) found no evidence that MTW status affected cost per assisted household, but did find evidence that MTW status was associated with an increase in federal funding and cash reserves.

For this evaluation, the annual reports will track one confirmatory outcome measure and six exploratory measures (exhibit 4-1). Our analysis will focus on expenditures on the housing programs included in MTW—public housing, HCV (excluding special voucher programs), and LNT programs—because funding for other programs such as special voucher programs cannot be flexibly applied, even for MTW

agencies unless the PHA receives explicit permission from HUD’s Voucher Office. 38 For each type of expenditure the evaluation will study, we specify the measure in terms of expenditures per unit per month, rather than on annual expenditures, because MTW gives PHAs flexibility in how and when they spend the funding they receive. It is important to note that the set of outcome measures selected for this statutory objective of the MTW program are comprised largely of measures that will assess “reduced costs” in federal expenditures as opposed to “the achievement of greater cost effectiveness” of federal expenditures, given the complexity of truly measuring cost effectiveness.

The confirmatory measure is total operating and administrative expenditures per household per month. 39

Operating and administrative expenditures are the PHA’s annual costs for HCV administrative services and HAP, plus the PHA’s annual public housing operating expenses. We divide the total expenditures by the total number of unit months leased for the PHA’s HCV and public housing programs. The data source 38 This baseline report does not include LNT expenses because none of the PHAs had received MTW designation prior to joining this study. HUD instructs MTW PHAs to report HAP expenses for HCV and LNT programs separately in the Voucher Management System (VMS), so we hope to be able to compare (in our exploratory analyses) the per unit per month HCV and LNT HAP expenses across MTW PHAs, as well as the per unit per month HCV HAP expenses between MTW and non-MTW PHAs.

39 The confirmatory measure excludes utility payments from public housing operating expenses. We exclude utilities from public housing operating costs to allow for a more consistent comparison between PHAs. Some PHAs pay utilities directly, so the cost shows up on FDS as operating expenses (these PHAs collect more total tenant [rent]

payment because there is no utility allowance for tenants); other PHAs do not pay these costs directly, so utility costs do not show up on FDS as operating expenses (but these PHAs collect less in total tenant payment because they provide a utility allowance).

Baseline Report – Evaluating MTW Flexibility for Smaller PHAs ▌pg. 34

Chapter 4: Baseline Values of Outcome Variables

for both total expenditures and the number of unit months leased is the annual PHA income statement in HUD’s Financial Data Schedule (FDS).

Exhibit 4-1. Baseline Levels of PHA Cost (per Household per Month)

Outcome

Mean ($)

Confirmatory

Total operating and administrative expenditures

630

Exploratory

Public housing operating expenses minus utilitiesPH

477

HCV and local, non-traditional programs’ administrative and tenant services expendituresHCV

74

HCV Housing Assistance Payment (HAP) expendituresHCV

617

Total federal funding of MTW-eligible programs

546

Total revenue of MTW-eligible programs

724

Cash reserves

158

Notes: Sample size is 43 RCT PHAs. The means reported are the average of PHA-level values. The confirmatory measure excludes utility payments from public housing operating expenses. We exclude utilities from public housing operating costs to allow for a more consistent comparison between PHAs. Some PHAs pay utilities directly, so the cost shows up on FDS as operating expenses (these PHAs collect more total tenant [rent] payment because there is no utility allowance for tenants); other PHAs do not pay these costs directly so it does not show up on FDS as operating expenses (but collect less in total tenant payment because provide a utility allowance). In versions of this exhibit in future Annual Reports, we will also report on local, non-traditional HAP expenditures. These are not reported at baseline because no PHA was eligible to have local, non-traditional HAP expenditures at baseline. Appendix C, Exhibit C-1 provides details on the construction of the measures in this exhibit.

PH Where indicated, means are estimated using only PHAs with public housing programs (27 PHAs).

HCV Where indicated, means are estimated using only PHAs with voucher programs (38 PHAs).

Sources: PHA Financial Data Schedule from most recent fiscal year (fiscal year varies by PHA); 2020 quarterly data from the Voucher Management System

We identified six exploratory outcome measures for the cost-effectiveness objective. The exploratory measures include the expenditures for the public housing programs, administrative and services expenditures for the voucher program, and housing assistance payments in the voucher program. By looking into these types of program costs separately, we may find information about how MTW agencies reallocate their budget and expenditures for these programs. The exploratory measures also include three measures of funding sources, including federal funding, total revenue, and cash reserves. By looking at these measures of available resources separately, we may find information about how MTW agencies find revenue opportunities or choose to build up (or spend down) cash reserves.

For any MTW PHA that implements an LNT program, we will review the MTW Supplement and interview PHA staff to understand the type and level of subsidy provided through the LNT program. LNT

activities could include shallow subsidies that do not compare to the assistance provided in the traditional programs, which could bias the cost per household downward relative to traditional PHAs if the PHA is providing shallow subsidies to a large number of households. LNT activities could also include housing assistance plus services, which could bias the cost estimate upward.

4.2 Household Self-Sufficiency

The MTW statutory objective related to self-sufficiency is “to give incentives to families with children where the head of household is working, seeking work, or is preparing for work by participating in job training, educational programs, or programs that assist people to obtain employment and become

Baseline Report – Evaluating MTW Flexibility for Smaller PHAs ▌pg. 35

Chapter 4: Baseline Values of Outcome Variables

economically self-sufficient” (1996 Act). A quasi-experimental analysis of the 39 Initial MTW PHAs found some evidence that MTW designation results in an increase in household income (Treskon et al., 2020). For purposes of this evaluation, in keeping with our understanding of the MTW waivers designed to encourage employment and earnings, our analysis will focus on self-sufficiency outcomes among persons in nonelderly, nondisabled households. Nonelderly, nondisabled households are households where neither the head of household nor the spouse or co-head is aged 62 or older or has a disability.

Exhibit 4-2 presents the baseline values of the confirmatory ( earnings in the last four quarters) and exploratory outcome measures for testing the impact of MTW on self-sufficiency. All the measures focus on earnings, not income, assets, or human capital measures. We chose to limit the analysis to earnings because work is the main goal of the statutory objective, and earnings growth is the main pathway for nonelderly, nondisabled adults to become economically self-sufficient. In addition, reliable administrative data do not exist for capturing the extent to which households living in HUD-assisted households are searching for work or participating in job training or education. 40

The data source for earnings is the National Directory of New Hires (NDNH), which provides quarterly wage information on individual employees from state workforce agency and federal agency records. HUD

requested person-level NDNH data for all nonelderly, nondisabled adults served by the PHAs in the study’s treatment, control, and comparison groups. Using household identifiers, we can compute household-level measures by merging the data of adults in the same household.

The timeframe for the confirmatory measure is the household’s most recent four quarters of earnings. We use the last four quarters of NDNH data available at the start of the analysis period each time we conduct the analysis. Capturing average earnings across four quarters (a full year) rather than in the most recent quarter shortens the window for observing change between baseline and the point of analysis, but it mitigates against any volatility in earnings over the course of the year. However, we also examine average earnings of households over the last quarter as an exploratory measure and the percentage of households with any earnings. In future reports we will also examine the percentage of households with earnings greater than 30 percent of area median income (AMI). Thirty percent of AMI is the threshold to determine whether any household’s earnings moved them above the income level that HUD requires non-MTW

PHAs to target for assistance. 41

The last two exploratory measures apply to people who have exited assistance by the year covered by the report. We define an “exit” as any household with an exit record in the calendar year (2020 for the baseline). Among nondisabled nonelderly adults in households that exited in the calendar year, we examine the average earnings of households in the most recent quarter after they exited housing assistance, and whether household earnings exceed 2.5 times the local FMR. Future reports will also include an outcome based on how many households have earnings above 30 percent of AMI.

40 The HUD Form 50058 collects some information on education and training needs of families participating in the Family Self-Sufficiency (FSS) program, but not all assisted households will be in the FSS program, and the data collected do not provide information on the type or intensity of the training provided.

41 The comparison to AMI is not included in this report because AMI was not included in pass-through file to use with NDNH data.

Baseline Report – Evaluating MTW Flexibility for Smaller PHAs ▌pg. 36

Chapter 4: Baseline Values of Outcome Variables

We examine earnings at the quarter of exit, in order to focus on whether households are poised to find housing affordable to them at time of exit. The 2.5 times the local FMR measure is based on the idea that a household with this level of income would be able to rent housing at the FMR without paying more than 40 percent of its income for rent. 42 To be consistent with the way the tenant share of rent is calculated in housing assistance programs, we would prefer to evaluate the affordability of private market housing using total household income rather than household earnings. However, the only income data have we will be from the household’s last recertification. We anticipate that the quarterly earnings data we obtain from NDNH will provide a much better indicator of the income of those exiting housing than will PIC

data on total income, because in many cases, PHAs will not have collected information on the household’s income at or near the time of exit. 43

Exhibit 4-2. Baseline Levels of Employment and Earnings among Adults in Nonelderly, Nondisabled Households

Outcome

Mean

Confirmatory

Household earnings in most recent year (Q1 2020 – Q4 2020)

$16,673

Exploratory

Percent of households with any earnings in Q3 of 2020

66.9%

Among households who exited assistance in 2020, annualized household earnings in quarter $24,626

after exit

Among households who exited assistance in 2020, percent with household earnings in quarter 37.2%

after exit equivalent to 2.5 times or more of the local FMR

Notes: Sample size is 43 RCT PHAs with 7,838 non-exiting households and 1,673 exiting households. The means reported are the average of PHA-level values. Appendix C, exhibit C-2 provides details on the construction of the measures in this exhibit.

Sources: National Directory of New Hires wage data from quarter 1 of 2020 through quarter 4 of 2020, HUD administrative data 4.3 Household Housing Choice

This section describes outcome measures related to the statutory objective to “increase housing choices for low-income families” (1996 Act). The statutory language does not define housing choices further, and our review of the literature indicates that MTW PHAs have interpreted the objective very broadly.

Khadduri et al. (2014) found that MTW PHAs have implemented three types of activities under the objective of increasing housing choice: activities to increase access to low-poverty neighborhoods, activities to increase the supply of quality affordable housing, and activities to assist hard-to-house or underserved households. Because these are quite different goals with different associated outcome measures, for the purposes of this evaluative effort, we will examine outcomes within three different facets of housing choice, as follows:

• What is the impact of MTW designation on households’ access to low-poverty neighborhoods?

• What is the impact of MTW designation on the supply of quality affordable housing?

42 For example, if the local FMR is $750 per month, a person earning 2.5 times the FMR earns $1,875 per month.

The monthly rent of $750 is equal to 40 percent of $1,875.

43 See Exhibit 4-7 for information on household income (measured at last recertification) relative to FMR for households exiting assistance.

Baseline Report – Evaluating MTW Flexibility for Smaller PHAs ▌pg. 37

Chapter 4: Baseline Values of Outcome Variables

• What is the impact of MTW designation on housing assistance for hard-to-house populations?

Each of these facets of housing choice is distinct and covers its own aspect of housing choice. Therefore, we designate a separate exploratory outcome for each. However, we designate only one confirmatory outcome for the housing choice statutory objective. The confirmatory outcome is in the access to opportunity neighborhoods sub-domain, which we judged was the primary focus of this MTW objective.

The three sub-sections discuss the outcome measures separately for each of the three areas of housing choice.

4.3.1 Opportunity Neighborhoods

The confirmatory measure for the access to opportunity neighborhood sub-domain (and the only confirmatory measure for the housing choice objective) is the percentage of HCV households living in low-poverty census tracts. There are many possible ways to define an opportunity neighborhood, and for the purpose of this evaluation, we chose to base the definition solely on neighborhood poverty rate because it is a readily available measure that is associated with children’s future outcomes (see, for example, Galster et al., 2007). PHAs with in-depth knowledge of their communities may be able to use a more refined definition of opportunity neighborhoods using additional information. However, using census tract poverty rates allow us to consistently define the outcome for all 142 PHA communities in the RCT and QED samples using American Community Survey (ACS) data).

For this study, we define a low-poverty census tract as a census tract where the poverty rate is in the lowest quartile of poverty rates within the PHA’s jurisdiction, or where the poverty rate is less than 10

percent. 44 The portion of the confirmatory measure that relates to the poverty distribution in the PHA’s jurisdiction reflects that PHAs primarily can influence where voucher holders locate within their jurisdiction. 45 The component of the confirmatory measure that relates to the absolute level of poverty in the census tract (< 10 percent) reflects that some PHA jurisdictions consist mainly of low-poverty neighborhoods, and movement to the areas of lowest quartile of poverty in that jurisdiction would not produce meaningful differences in opportunity from other low-poverty neighborhoods in the jurisdiction.

To estimate the PHA’s lowest quartile of poverty rate, we first identify the census tracts in its jurisdiction.

To select the lowest quartile of poverty rate, we do not give equal distributional weight to each census tract. Because we want to learn whether renters are in the lowest quartile of poverty rates for renters in that jurisdiction, we weight the distribution of the census tracts’ poverty rates by the number of households in that census tract that rent (rather than own) their residence. This choice results in a better 44 The PHA’s jurisdiction is defined as the census tracts in which at least 2 percent or five or more (whichever is smaller) of the PHA’s households reside, as well as abutting census tracts if the abutting census tracts are in the same state. Among the 31 Flexibility Cohort PHAs, we tested alternative constructions of this measure: (a) the jurisdiction told to us by PHA administrators in interviews, (b) the PHA’s jurisdiction that we would presume, based on information available on its website, (c) the entire county in which the PHA operates, (d) all ZIP codes where at least 2 percent of the PHA’s households reside, (e) all ZIP codes where at least 2 percent of the PHA’s households reside as well as all abutting ZIP codes as long as they are in the same state, (f) all census tracts where at least 2 percent of the PHA’s households reside as well as all abutting census tracts as long as they are in the same state. Method (a) is our preferred method, but it is not scalable to the full set of 142 PHAs in the RCT

and QED samples combined. Among the subset of PHAs examined, method (f) matched method (a) more often than the other methods.

45 Tenants using PBVs are part of the HCV program, and thus are considered throughout as “voucher” households.

Baseline Report – Evaluating MTW Flexibility for Smaller PHAs ▌pg. 38

Chapter 4: Baseline Values of Outcome Variables

reflection of the neighborhood choices for renters for that jurisdiction. Appendix exhibit C-3 contains more details on the data sources and construction of measures of low-poverty neighborhoods.

Although the confirmatory analysis is for all HCV households, exploratory measures focus on households with children. The research literature suggests that children are the main beneficiaries of moves from high to low-poverty neighborhoods, so it is important to consider whether households with children are more or less likely than households overall to live in low-poverty census tracts.

We will also examine the percentage of public housing households (overall and with children) living in low-poverty neighborhoods. Although public housing locations are immobile, PHAs can influence the income mix of their public housing developments and manage their development to make living in the neighborhood more desirable, both of which could affect the neighborhood poverty rate. PHAs also could indirectly influence neighborhood poverty rates through neighborhood revitalization activities with partners and by encouraging local government investments.

One limitation with these measures is that households that use their vouchers to port out46 of the PHA’s jurisdiction are not part of the calculation, so these measures may understate access to opportunity for PHAs with high port-out rates to low-poverty neighborhoods. In future reports we will examine the percentage of HCV households porting out to low-poverty census tracts relative to the sending PHA’s jurisdiction. We do not include this measure in the baseline report because of a data limitation that makes it difficult to determine from which PHA a household has “ported out” from.

Exhibit 4-3 shows the baseline values of the opportunity neighborhood measures that we will ultimately use for estimating the impact of MTW designation.

46 Porting out means using the portability measures of the HCV program to use their voucher assistance outside the PHA’s jurisdiction.

Baseline Report – Evaluating MTW Flexibility for Smaller PHAs ▌pg. 39

Chapter 4: Baseline Values of Outcome Variables

Exhibit 4-3. Households’ Housing Choice: Low-Poverty Neighborhood Location at Baseline Outcome

Mean (%)

Confirmatory

Percentage of HCV households living in low-poverty census tractsHCV

34.7

Exploratory

Percentage of HCV households living in a census tract with a poverty rate below the median for the PHA’s jurisdictionHCV

53.5

Percentage of HCV households with children living in low-poverty census tractsHCV

35.5

Percentage of HCV households with children living in a census tract with a poverty rate below the median for the PHA’s jurisdictionHCV

55.0

Percentage of public housing households living in low-poverty census tractsPH

20.6

Percentage of public housing households with children living in low-poverty census tractsPH

16.3

Notes: Sample size is 43 RCT PHAs. The means reported are the average of PHA-level values. A “low-poverty” census tract is defined as a census tract with a poverty rate of less than 10% or a poverty rate in the lowest quartile of census tract poverty rates for the PHA’s jurisdiction.

This differs from the estimate of 19.7 in exhibit 2-1 because it was measured in a different time period (tenants as of December 2020 instead of at tenants at time of random assignment) using a different data source (PIC rather than Picture of Subsidized Housing) and because we use the definition of poverty rate given in appendix C rather than what is used to construct the variables in the Picture of Subsidized Housing.

appendix C, exhibit C-3 provides details on the construction of the measures in this exhibit.

PH Means are estimated using only PHAs with public housing programs (20 PHAs, because 7 of the 27 PHAs with public housing programs had missing data on tenant census tract location).

HCV Means are estimated using only PHAs with voucher programs (38 PHAs).

Source: HUD administrative data, American Communities Survey

4.3.2 Affordable Housing

A second way of expanding housing choice is by increasing the number of housing units available and affordable for the population served by the PHA. This could include Flexibility Cohort PHAs being able to assist more households, preventing decreases in supply through preservation of units at risk of becoming unaffordable, development activities that help create new affordable units, or efforts to increase the number of landlords accepting vouchers. Exhibit 4-4 shows the baseline values of the outcome measures, all of which are exploratory, which we will use for testing the impact of MTW on the supply of quality, affordable housing. They are a combination of counts of the number of households served in the voucher and public housing programs compared to baseline and utilization or occupancy rates that measure the number of households served by program type against the PHA’s normal (non-MTW) capacity for that program.

The first measure in Exhibit 4-4 is the baseline level of the combined number of HCV and public housing households served in an average month. Growth in the number of households served would reflect the overall change in housing supply provided directly by PHA programs’ net of the movement between the programs allowed by MTW flexibility. In future impact reports, we will also include the number of households served by LNT programs. A limitation of the measure is that no standard method currently exists for calculating LNT households served using a unit-months method comparable to the measure used for HCV and public housing based on administrative data. We will use the information from the process study to convert the LNT households served to unit-month measures and document the calculations.

We will also examine HCV unit utilization, HCV budget utilization, and public housing occupancy rates.

They are commonly used performance measures (Buron et al., 2017; GAO, 2018) and provide a standardized, cross-PHA comparable measure of the quantity of households served relative to capacity—

that is, the resources (units or budget) available.

Baseline Report – Evaluating MTW Flexibility for Smaller PHAs ▌pg. 40

Chapter 4: Baseline Values of Outcome Variables

Ideally, we would also study how readily voucher households can lease housing—the HCV success rate and the average number of days between voucher issuance and lease-up. However, this information was not available at the time of this analysis. However, we analyze the number of unique HCV landlords per 100 households as a measure of landlord acceptance of voucher holders, an indicator of how readily voucher holders can lease housing.

The last two exploratory measures in exhibit 4-4 relate to public housing. We will examine the impact of MTW designation on the quality of a PHA’s public housing, as measured by the percentage of public housing units in developments scoring 90 or above on their most recent HUD inspection. In addition, we will study PHAs’ investment in public housing using the concept of economic useful life and data from HUD’s FDS. The economic useful life of a public housing development is 100 percent minus the accumulated depreciation for the development divided by the gross cost of assets for the development. 47

The result of the ratio is a percentage. The higher the percentage, the greater the economic useful life is remaining at the development. If a PHA’s investments extend the economic useful life of its public housing, that could be evidence of preserving the supply of affordable housing.

Exhibit 4-4. Households’ Housing Choice: Supply of Quality, Affordable Housing Levels at Baseline Outcome

Mean

Exploratory

Total number of households served through HCV and public housing programs in average month

539.6

HCV unit utilization rateHCV (%)

87.7

HCV budget utilization rateHCV (%)

98.4

Public housing occupancy ratePH (%)

93.8

Number of unique HCV landlords per 100 voucher householdsHCV

34.4

Public housing units scoring 90 or above on most recent physical inspectionPH (%) 84.5

Economic useful life of public housing units (measured as ratio of total capital assets net of depreciation (excluding land) and total capital assets including accumulated depreciation)PH

24.3

Notes: Sample size is 43 RCT PHAs. The means reported are the average of PHA-level values. Appendix C, exhibit C-4 provides details on the construction of the measures in this exhibit.

PH Means are estimated using only PHAs with public housing programs (27 PHAs).

HCV Means are estimated using only PHAs with voucher programs (38 PHAs).

Source: HUD administrative data, American Communities Survey

One limitation is that economic useful life is not the same as actual useful life. That is, a household could continue to live in a unit deemed to have no economic useful life. Also, if assessed over time, we may need to adjust this indicator to take into consideration public housing repositioning activity, such as Rental Assistance Demonstration conversions. Disposal or demolition of old public housing units would be reflected as an improvement in this indicator, so we may need to make adjustments to reduce the sensitivity of this measure to large-scale disposal or demolition activities where new investment is not involved.

47 PHAs report the accumulated depreciation and gross cost of assets, by public housing development, in the FDS.

Baseline Report – Evaluating MTW Flexibility for Smaller PHAs ▌pg. 41

Chapter 4: Baseline Values of Outcome Variables

4.3.3 Hard-to-House Populations

We identified three exploratory measures for assessing how Flexibility Cohort PHAs are using MTW to assist hard-to-house households. Exhibit 4-5 shows the baseline values of these three measures. The first two measures track the percentage of households served that have a nonelderly family member with a disability and the percentage of households served that have three or more minors. The third exploratory measure is the percentage of households served that were homeless at entry, based on question 4c in the Family 50058 Form (the “homeless at admission” field). This data source is not ideal, because a survey of PHAs conducted by Abt (described in Khadduri et al., 2014) found that the homeless at admission field is often not verified or well maintained. It is likely that this field will be better maintained by PHAs that have implemented a preference for people experiencing homelessness or participate in the coordination of local homelessness services.

Exhibit 4-5. Households’ Housing Choice: Hard-to-House Population Levels at Baseline Outcome

Mean (%)

Exploratory

Percentage of households served that have a nonelderly family member with a disability 30.5

Percentage of households served that have three or more minors

5.0

Percentage of households served that were homeless at the time of admission 5.3

Notes: Sample size is 43 RCT PHAs. The means reported are the average of PHA-level values. Appendix C, exhibit C-5 provides details on the construction of the measures in this exhibit.

Source: HUD administrative data

4.4 Other Tenant Outcomes

This section presents other important outcome measures that do not fit in one of the three MTW

objectives but are important measures of housing programs to track in the MTW expansion: measures of households served, exits, and rent burden.

4.4.1 Households Served and Household Exits

To round out the picture of how PHAs used their MTW designation, we will analyze the extent to which MTW designation affects aspects of PHA programs and operations not directly related to the statutory objectives. Exhibit 4-6 displays baseline values of the outcomes related to the type of household served by the PHAs, by program. In particular, we will focus on the income level of admitted households, as well as changes over time in the number of households served, the proportion of households with children, and the proportion of households where the head, co-head, or spouse is aged 61 or younger and has a disability.

Baseline Report – Evaluating MTW Flexibility for Smaller PHAs ▌pg. 42

Chapter 4: Baseline Values of Outcome Variables

Exhibit 4-6. Other Tenant Outcomes: Households Served Levels at Baseline Outcome

Mean

Proportion of newly admitted households with income at or below the greater of 30% of AMI or the federal poverty level

Overall

89.4

Public housingPH

89.8

Housing choice vouchersHCV

90.3

Number of households served in MTW-eligible programs

Overall

539.6

Public housingPH

158.8

Housing choice vouchersHCV

380.8

Percentage of households with at least one child aged <18

Overall

38.6

Public housingPH

34.9

Housing choice vouchersHCV

39.7

Percentage of households whose head of household, spouse, or co-head is aged <62 and has a disability Overall

28.6

Public housingPH

25.1

Housing choice vouchersHCV

30.5

Notes: Sample size is 43 RCT PHAs. The means reported are the average of PHA-level values. Appendix C, exhibit C-6 provides details on the construction of the measures in this exhibit.

PH Means are estimated using only PHAs with public housing programs (27 PHAs).

HCV Means are estimated using only PHAs with voucher programs (38 PHAs).

Source: HUD administrative data, American Community Survey

MTW waivers may affect household exit. For example, success at improving self-sufficiency may increase the rate of exit as households become financially independent. Exit rates may also increase if households react negatively to changes in program rules. For example, the introduction of work requirements may increase the rate of exit. We will examine the exit rate over time as well as the financial self-sufficiency of households that leave the housing subsidy program, as measured by whether household income is more than 2.5 times higher than FMRs at time of exit. Earnings at this level suggest that the household would need to pay no more than 40 percent of its income to rent a reasonably priced housing unit.

Exhibit 4-7 displays the baseline exit levels, showing that 13.6 percent of all public housing households served in 2020 exited the program, and 8.4 percent of all HCV households served in 2020 exited the HCV

program. Of the households that exited the program, 1.7 percent had total household incomes more than the annual equivalent of 2.5 times FMR at the time of the most recent (last) income certification. This low percentage contrasts with the estimated 23 percent of exiting households with quarterly earnings exceeding the quarter-equivalent of 2.5 times FMR in the last quarter for which earnings data are available (exhibit 4-2). The large difference is likely due to three factors: (1) PHAs do not require income recertification at the time of exit and thus are likely to have outdated income records, (2) the earnings exit data do not align with time of exit but rather the most recent calendar quarter, so households may have increased earnings after exiting, and (3) earnings may be underreported to PHAs. We will also provide descriptive statistics (averages and frequencies of variables in the PIC dataset) on the reason for exit, which will be available for the Flexibility Cohort PHAs only.

Baseline Report – Evaluating MTW Flexibility for Smaller PHAs ▌pg. 43

Chapter 4: Baseline Values of Outcome Variables

Exhibit 4-7. Other Tenant Outcomes: Households Exit Levels at Baseline

Outcome

Mean (%)

Percentage of households ending participation in past 12 months

Overall

10.3

Public housingPH

11.9

Housing choice vouchersHCV

9.1

Percentage of households ending participation in the past 12 months with household income exceeding 2.5 times FMR at last recertification

Overall

2.0

Public housingPH

1.9

Housing choice vouchersHCV

2.0

Notes: Sample size is 43 RCT PHAs. The percentages are percentages of any household assisted in the calendar year 2020. The means reported are the average of PHA-level values. Appendix C, exhibit C-7 provides details on the construction of the measures in this exhibit.

PH Where indicated, means are estimated only using PHAs with public housing programs (27 PHAs).

HCV Where indicated, means are estimated only using PHAs with voucher programs (38 PHAs).

Source: HUD administrative data, American Communities Survey

4.4.2 Rent Burden

Several MTW waivers could affect tenant rent contributions as well as the tenant’s household income. We will estimate the impact of MTW designation on the changes in rent burden over time for all PHAs, and separately for households in public housing programs and households in the HCV program. Exhibits 4-8

and 4-9 show baseline levels of these outcomes. We define rent burden as the ratio of 12 times the total tenant payment divided by the household’s unadjusted annual income.

Baseline Report – Evaluating MTW Flexibility for Smaller PHAs ▌pg. 44

Chapter 4: Baseline Values of Outcome Variables

Exhibit 4-8. Other Outcomes: Rent Burden in Public Housing at Baseline

Outcome

Mean

Average rent burden in public housinga

Overall

32.0

Household with children

34.2

Households headed by person aged ≥62

28.1

Households headed by person aged <62 with a disability

28.7

Percentage of households in public housing with rent burden above 30%

Overall

12.5b

Household with children

13.0

Households headed by person aged ≥62

0.8

Households headed by person aged <62 with a disability

1.1

Percentage of households in public housing with rent burden above 40%

Overall

6.8

Household with children

11.6

Households headed by person aged ≥62

0.5

Households headed by person aged <62 with a disability

0.9

Percentage of households in public housing with rent burden above 50%

Overall

6.0

Household with children

10.3

Households headed by person aged ≥62

0.5

Households headed by person aged <62 with a disability

0.8

Notes: Sample size is 43 RCT PHAs. The means reported are the average of PHA-level values.

a Rent burden is calculated as the ratio of 12 times the total tenant payment to the household’s unadjusted (gross) annual income, top-coded at 1 (100%). Appendix C, exhibit C-8 provides details on the construction of the measures in this exhibit.

b Two-thirds of household with a rent burden above 30 percent pay $50 or less in rent, thus are likely paying the minimum rent.

PH Means are estimated using only PHAs with public housing programs (27 PHAs).

HCV Means are estimated using only PHAs with voucher programs (38 PHAs).

Source: HUD administrative data

Baseline Report – Evaluating MTW Flexibility for Smaller PHAs ▌pg. 45

Chapter 4: Baseline Values of Outcome Variables

Exhibit 4-9. Other Outcomes: Rent Burden in Housing Choice Voucher Programs at Baseline Outcome

Mean

Average rent burden in the HCV programa

Overall

31.2

Household with children

31.6

Households headed by person aged ≥62

28.5

Households headed by person aged <62 with a disability

29.0

Percentage of households in the HCV program with rent burden above 30%

Overall

9.4

Household with children

8.4

Households headed by person aged ≥62

1.1

Households headed by person aged <62 with a disability

1.6

Percentage of households in the HCV program with rent burden above 40%

Overall

5.2

Household with children

7.2

Households headed by person aged ≥62

0.6

Households headed by person <62 with a disability

1.4

Percentage of households in the HCV program with rent burden above 50%

Overall

4.8

Household with children

6.4

Households headed by person aged ≥62

0.5

Households headed by person aged <62 with a disability

1.3

Note: Sample size is 43 RCT PHAs.

a Rent burden is calculated as the ratio of 12 times the total tenant payment to unadjusted household income, top-coded at 1 (100%).

Appendix C, exhibit C-8 provides details on the construction of the measures in this exhibit.

PH Means are estimated using only PHAs with public housing programs (27 PHAs).

HCV Means are estimated using only PHAs with voucher programs (38 PHAs).

Source: HUD administrative data

Baseline Report – Evaluating MTW Flexibility for Smaller PHAs ▌pg. 46

5. Summary

This baseline report for the evaluation of first cohort of the MTW expansion provides a foundation for the evaluation’s future work. It does so by explaining how the study PHAs were selected to support a randomized controlled trial (RCT) and summarizing the design of the evaluation, which evolved to include a comparison sample to support a quasi-experimental design (QED). The report also documents how Flexibility Cohort PHAs plan to use their MTW flexibility and presents baseline values of the pre-specified outcome measures that will be used in future impact analyses.

The baseline characteristics of the Treatment and Control PHAs are very similar, with no more differences in their baseline characteristics than one would expect due to chance. Given the study’s random assignment procedures for the treatment and control groups and their similarity in baseline characteristics, any impacts found will be definitively attributed as caused by MTW designation. The evaluation team was able to select the QED sample to be very similar to the RCT sample. As a result, we also have a high degree of confidence that the QED-estimated impacts will be interpretable as being caused by MTW designation. Nevertheless, because there may be unobservable differences in PHAs that did and did not apply for the first cohort of MTW, the RCT analysis will be considered the primary source of impact findings. QED impact findings for the whole sample will need to be consistent with the RCT

findings for us to have confidence in subgroup analysis that uses the Comparison PHAs.

We also compared the 43 RCT PHAs to all smaller PHAs that were eligible to apply for the first cohort of MTW to judge the extent to which impact findings from this evaluation would generalize to a world in which all smaller PHAs were offered MTW flexibility. The RCT PHAs are more often in metropolitan areas than are all smaller PHAs. This finding is correlated with other differences: the RCT PHAs are larger and are in jurisdictions with higher fair market rents. These differences need to be kept in mind when considering how the findings would apply if considering allowing all high-performing, smaller PHAs to have the flexibility allowed in MTW .

Flexibility Cohort PHAs were asked about their plans to use MTW flexibility in the initial and second parts of the application process and in interviews conducted in March 2021, about 2 months after they were offered MTW designation. The most important goal of PHAs is improving cost-effectiveness. PHAs report hoping to reduce administrative burden and simplify program rules and use any cost savings for serving more households, investing in affordable housing, or providing additional supportive services to tenants. The specific waivers most frequently cited would alter the reexamination process for determining rent and change the rent calculation procedures. More than half of the PHAs plan to use their MTW

flexibility to add landlord incentives, implement work requirements, or make changes to their FSS

programs.

Most Flexibility Cohort PHAs also are planning to take advantage of the ability to use their funding streams flexibly. Though most PHAs did not yet have definite plans for using this funding flexibility, it was an important draw for becoming an MTW agency.

The impact analysis will include more than 30 outcome measures relating to PHA cost and cost-effectiveness, to tenant self-sufficiency, to housing choice, and to other tenant outcomes. This report presents the baseline values of these measure as of 2020. The baseline measures for the three confirmatory tests selected by HUD and the evaluation team are: (1) total operating and administrative

Baseline Report – Evaluating MTW Flexibility for Smaller PHAs ▌pg. 47

Chapter 5: Summary

expenditure per household per month (baseline average of $644); (2) annual earnings of adults in nonelderly, nondisabled households (baseline average of $14,030); and (3) percentage of HCV

households living in low-poverty census tracts (baseline average of 19.4 percent).

This report covers the period before and just after the first cohort of MTW expansion PHAs were offered the opportunity to become an MTW agency. Future Annual Reports will cover the first year after the offer of MTW designation (2021) and each year through the fourth year after MTW designation (2022, 2023, and 2024). These reports will document the actual MTW activities that the Flexibility Cohort PHAs implement, how they adapt these activities over time, and the impact of MTW on PHA and tenant outcomes.

Baseline Report – Evaluating MTW Flexibility for Smaller PHAs ▌pg. 48

Appendix A. List and Map of Study PHAs

Exhibit A-1. List of 33 Treatment PHAs

PHA

PHA

#

Code

PHA Name

#

Code

PHA Name

1

AL050

Auburn Housing Authority

18 MO212 Ripley County Housing Authority

Robeson County Housing

2

AL068

Sheffield Housing Authority

19 NC084

Authority

AL073

Housing Authority of the City of

3

Ozark

20 NE175

South Sioux City Housing Agency

Combined PHA comprising

AR097 (Housing Authority of the

City of Fayetteville) and AR181

4

ARX01 (Fayetteville Housing Authority)

21 NH003

Dover Housing Authority

Housing Authority of the City of

5

CA123

Pomona

22 NJ048

Neptune Housing Authority

County of Solano Housing

6

CA131

Authority

23 NJ059

Pleasantville Housing Authority

Housing Authority of the City of

7

CO019 Brighton

24 SC031

Housing Authority of Cheraw

Housing Authority of New Smyrna

8

FL022

Beach

25 SC036

Housing Authority of Fort Mill

Housing Authority of the City of

9

GA095

Newnan

26 TN065

Maryville Housing Authority

Housing Authority of the City of

Housing Authority of Travis

10 ID005

Pocatello

27 TX480

County

Housing Authority of the City of

11 LA054

Housing Authority of Ruston

28 TX483

Rosenberg

Bristol Redevelopment & Housing

12 MA107 North Andover Housing Authority 29 VA002

Authority

Harrisonburg Redevelopment &

13 MD007 Rockville Housing Enterprises

30 VA014

Housing Authority

14 MN004 The HRA of Hibbing

31 VT002

Brattleboro Housing Authority

15 MN168 Kandiyohi County HRA

32 WA020 HA City of Kelso

Housing Authority of Randolph

16 MN203 McLeod County HRA

33 WV045 County

Washington County Community

17 MN212 Development Agency

Key: HRA=Housing and Redevelopment Authority

Source: HUD communications

Baseline Report – Evaluating MTW Flexibility for Smaller PHAs ▌pg. 49

Appendix A. List and Map of Study PHAs

Exhibit A-2. List of 10 Control PHAs

PHA

PHA

#

Code

PHA Name

#

Code

PHA Name

Bossier Parish Section 8 Housing

1

AZ006

Flagstaff Housing Authority

6

LA190

Authority

Housing Authority of the City of

2

CO095 Garfield County Housing Authority 7

OK146

Stillwater

Housing Authority of the Town of

Housing Authority of the County

3

CT039

West Hartford

8

PA079

of Warren

Greater Metro. Area Housing

Housing Authority of the County

4

IL010

Authority of Rock Island County

9

PA086

of Clarion

Housing Authority of Bowling

5

KY063

Green

10 TN081

Erwin Housing Authority

Source: HUD communications

Baseline Report – Evaluating MTW Flexibility for Smaller PHAs ▌pg. 50

Appendix A. List and Map of Study PHAs

Exhibit A-3. List of 99 Comparison PHAs

PHA

PHA

#

Code

PHA Name

#

Code

PHA Name

1

AL052

Housing Authority of Cullman

51 MN171 Le Sueur County HRA

Housing Authority of Hamilton

2

AL053

Alabama

52 MN179 Morrison County HRA

Scott County Community

3

AL054

Florence Housing Authority

53 MN184 Development Agency

Scotland County Public Housing

4

AL061

Housing Authority of Opelika

54 MO200 Agency

5

AL072

Housing Authority of Columbiana 55 MO207 Pulaski County Housing Authority

Jasper County Public Housing

6

AL075

Boaz Housing Authority

56 MO215 Agency

Housing Authority of the City of

Housing Authority of the City of

7

AL105

Talladega

57 NC008

Concord

Housing Authority of the Town of

8

AL118

Eufaula Housing Authority

58 NC023

Mount Airy

9

AL152

Housing Authority of Northport

59 NC027

Hendersonville Housing Authority

10 AL177

Housing Authority of Troy

60 NC066

Burlington Housing Authority

Town of East Spencer Housing

11 AR068 Hope Housing Authority

61 NC134

Authority

12 AR117 Polk County Housing Authority

62 NE150

Hastings Housing Authority

Housing Authority of the City of

13 AR175 Benton

63 NE157

Norfolk Housing Agency

14 CA066 Suisun City Housing Authority

64 NH004

Portsmouth Housing Authority

Housing Authority of the Borough

15 CA110 Culver City Housing Authority

65 NJ011

of Lodi

Housing Authority of the City of

16 CA119 South Gate

66 NJ035

South Amboy Housing Authority

17 CA126

Hawthorne Housing

67 NJ047

Carteret Housing Authority

Housing Authority of the City of

18 CA136

Hawaiian Gardens

68 NJ056

Berkeley Housing Authority

County of El Dorado Housing

19 CA151

Authority

69 NJ066

Housing Authority City of Linden

Housing Authority of the

20 CO034 Loveland Housing Authority

70 NJ081

Township of Middletown

Town of Huntington Housing

21 CO050 Arvada Housing Authority

71 NY035

Authority

22 CO090 Housing Authority of Weld County 72 NY045 Kingston Housing Authority

23 CT042

Hamden Housing Authority

73 NY050

Housing Authority of Long Beach

24 DE003

Newark Housing Authority

74 NY061

Hudson Housing Authority

Town of Union Community

25 FL024

Ormond Beach Housing Authority 75 NY505

Development

26 FL034

Plant City Housing Authority

76 OR017 Klamath Housing Authority

27 FL035

Housing Authority of Springfield

77 OR020 Coos-Curry Housing Authority

28 FL060

Punta Gorda Housing Authority

78 PA088

Centre County Housing Authority

Baseline Report – Evaluating MTW Flexibility for Smaller PHAs ▌pg. 51

Appendix A. List and Map of Study PHAs

PHA

PHA

#

Code

PHA Name

#

Code

PHA Name

Lancaster County Housing

29 FL141

Collier County Housing Authority

79 PA090

Authority

Housing Authority of the City of

30 GA078 East Point

80 SC005

Housing Authority of Darlington

31 ID021

Ada County Housing Authority

81 SC020

Housing Authority of Chester

32 IL084

Fulton County Housing Authority

82 SC029

Housing Authority of Hartsville

Lyon County/Eckan Housing

33 KS159

Authority

83 TN024

Tullahoma Housing Authority

Housing Authority of the City of

34 LA042

Bossier City

84 TN066

Bristol Housing

Housing Authority of the City of

35 LA044

Thibodaux

85 TN079

Dickson Housing Authority

Housing Authority of the Town of

36 LA124

Olla

86 TX349

Weatherford Housing Authority

Grayson County Housing

37 MA043 Dracut Housing Authority

87 TX458

Authority

Brazoria County Housing

38 MA047 Falmouth Housing Authority

88 TX484

Authority

39 MA069 Milford Housing Authority

89 TX485

Big Spring Housing Authority

40 MA096 Greenfield Housing Authority

90 TX493

Terrell Housing Authority

Housing Authority of Montgomery

41 MA117 Stoughton Housing Authority

91 TX560

County

Waynesboro Redevelopment &

42 MA139 Tewksbury Housing Authority

92 VA022

Housing Authority

Lee County Redevelopment &

43 MN006 HRA of Winona

93 VA034

Housing Authority

County of Albemarle/Office of

44 MN010 HRA of the City of South St. Paul 94 VA036

Housing

HRA in and for the City of

Accomack-Northampton Regional

45 MN032 Brainerd

95 VA040

Housing Auth

Cambridge Economic

46 MN067 Development Authority

96 VT003

Rutland Housing Authority

Housing Authority of Grant

47 MN070 HRA of Litchfield

97 WA014 County

Housing Authority of Asotin

48 MN085 HRA of Austin

98 WA017 County

Housing Authority of St. Louis

Housing Authority of Island

49 MN144 Park

99 WA024 County

50 MN166 Swift County HRA

Key: HRA=Housing and Redevelopment Authority.

Source: Abt selected using Pictures of Subsidized Housing data from 2019

Baseline Report – Evaluating MTW Flexibility for Smaller PHAs ▌pg. 52

Appendix A. List and Map of Study PHAs

Exhibit A-4. Map of PHAs in the Study

Source: Location of PHAs is from Affordable Housing Online (https://affordablehousingonline.com/housing-authority) accessed in November 2020

Baseline Report – Evaluating MTW Flexibility for Smaller PHAs ▌pg. 53

Appendix B. Comparison of Treatment PHAs to Control and

Comparison PHAs

This appendix compares the Treatment PHAs to the Control and Comparison PHAs to rule out the possibility that any future differences between the groups’ outcomes could be explained by underlying differences rather than the intervention. Section B.1 considers the RCT, comparing the Treatment PHAs to the Control PHAs. Section B.2 considers the QED, comparing the Treatment PHAs to the Comparison PHAs.

B.1 Baseline Characteristics and Balance of the RCT PHAs

Baseline differences between the Treatment and Control PHAs are due only to chance. That said, it is standard practice in experimental evaluations to assess the extent of those baseline differences. Those differences being infrequent and/or small provides reassurance that impacts are solely attributable to the intervention rather than to something else (such as something that relates to some underlying characteristic that, by chance, differs between the groups). Exhibit B-1 compares average PHA and household characteristics for the PHAs in the two groups using 2019 data. As expected from an evaluation design that uses random assignment, there are few statistically significant differences between the Treatment and Control PHAs. Among the 20 variables that we examined, one difference appears as statistically significant, which is no more than would be expected by chance. 48 Also, as is standard practice in experimental evaluations, we will include baseline covariates in our analysis to control for chance and small differences. Doing so improves the precision of the impact estimates.

Exhibit B-1. Comparison of PHA and Household Characteristics at Treatment and Control PHAs Treatment

Control

PHAs

PHAs

Difference

Characteristics

(n=33)

(n=10)

(T-C)

p-Value

PHA Characteristics

Percentage (number) of PHAs with HCV and public

housing

51.5 (17)

50.0 (5)

1.5 p.p.

.615a

Percentage (number) of PHAs with only HCV

39.4 (13)

30.0 (3)

9.4 p.p.

Percentage (number) of PHAs with only public housing

9.2 (3)

20.0 (2)

−10.9 p.p.

Number of public housing units (excluding PHAs with

no public housing units)

241

261

−20

.763

Number of HCV units (excluding PHAs with no HCV

units)

397

499

−102

.277

Number of units (HCV and/or public housing)

507

582

−75

.389

Occupancy rate (all programs)b

88.9

93.0

−4.1 p.p.

.190

Months since move-in (all programs)

87.9

82.3

5.6

.496

Average HUD expenditure per unit per month

$689

$552

$138

.101c

48 That specific variable is the share of households in special voucher programs: the Treatment PHAs have a lower share than the Control PHAs (5.3 percent compared to 13.7 percent). We do not expect this variable to have a strong association with most outcome measures. Still, in the analysis, the evaluation team will control for baseline values of the outcome measures to improve the precision of the estimates (see Turner et al., 2021, for more details).

Baseline Report – Evaluating MTW Flexibility for Smaller PHAs ▌pg. 54

Appendix B: Comparison of Treatment PHAs to Control and Comparison PHAs Treatment

Control

PHAs

PHAs

Difference

Characteristics

(n=33)

(n=10)

(T-C)

p-Value

PHA headquarters is in a metropolitan area

72.7

60.0

12.7 p.p.

.455

Fair Market Rent (FMR) for 2-bedroom unit

$1,041

$899

$143

.246

80% of Area Median Income (AMI) for 4-person

household

$59,329

$54,570

$4,759

.256

Ratio of 2-bedroom FMR to 80% of AMI for 4-person

household

0.206

0.197

0.010

.449

Average weekly wage for local government workers in

the county where the PHA is located

$886

$676

$210

.053

Characteristics of Assisted Households (All Programs)

Months on the waiting list

19.8

22.1

−2.3

.686

Months since move-in

87.9

82.3

5.6

.496

Percentage of households where head or co-head is

aged 62 or older OR has a disability

56.7

59.3

−2.6 p.p.

.548

Average earnings of nondisabled adults aged 18-61

$18,004

$16,968

$1,036

.493

Percentage of HCV households living in low-poverty

census tracts

19.3

21.3

−2.0 p.p.

.497

Percentage of households in special voucher programs

5.3

13.7

−8.5 p.p.

.026

Notes: The abbreviation “p.p.” indicates “percentage points.” A p-value less than 0.10 means that if we drew two random samples from the same population, such a difference would occur less than 10 percent of the time. Two PHAs in the RCT were missing data for all 2019 POSH

metrics except for public housing and HCV unit counts; 2018 POSH data were substituted for these metrics for these two PHAs.

a Indicates that the p- value is from a Chi-square test. Unless indicated otherwise, p-values come from t tests of simple comparison of means, using a pooled variance estimate.

b Occupancy rate is the number of occupied units divided by the number of units available.

c t test uses Satterthwaite variance estimate due to unequal variance.

Sources: HUD 2018 and 2019 Picture of Subsidized Households (POSH), HUD December 2019 Inventory Management System/PIH

Information Center (PIC), HUD FY20 Fair Market Rents, HUD December 2019 Voucher Management System (VMS), HUD FY2019 Section 8

administrative data, Bureau of Labor Statistics 2019 Q3 local government wages for all industries, counties, and establishment sizes The MTW application that the 43 RCT PHAs completed in 2018–19 asked which of 16 possible waivers the PHA planned to use if they became an MTW agency. The PHAs’ intentions might have changed between that application and becoming an MTW agency in early 2021 (see Chapter 3) and might be different between what the PHAs end up implementing when they have a chance to use their MTW

flexibility in 2021 and later. Regardless, the RCT PHAs’ initial applications provide another comparison for how well-balanced the Treatment and Control PHAs are prior to random assignment.

As can be seen in exhibit B-2, Treatment and Control PHAs’ plans to use MTW waivers were similar.

There were no statistically significant differences in plans for using the 16 waivers included in the MTW

Application. 49 Overall, the similarity of the planned use of waivers is reassuring because it is another indication that suggests that the Control PHAs offer a quality counterfactual for the outcomes Treatment PHAs would have if they were not designated as MTW.

49 That said, the magnitude of some of the differences might appear to indicate some qualitative differences worth discussing. For example, a seemingly greater share of treatment than Control PHAs indicate plans for using project-based voucher (PBV) waivers, such as eliminating the competitive PBV process or increasing the rent to owner. Given the number of PBV waivers asked about on the survey and the small samples, we do not judge these differences to be meaningful.

Baseline Report – Evaluating MTW Flexibility for Smaller PHAs ▌pg. 55

Appendix B: Comparison of Treatment PHAs to Control and Comparison PHAs Exhibit B-2. Comparison of Plans to Use MTW Waivers at Treatment and Control PHAs Percentage

% of

% of

Point

Treatment Control

Difference

Waiver

P HAs

PHAs

(T-C)

p-Value

Waivers for Either HCV or PH Program

(n=33)

(n=10)

Reexaminations

78.8

80.0

-1.2

.999

Tenant Rent Policies

90.9

80.0

10.9

.292

MTW Self-Sufficiency Program

72.7

70.0

2.7

.999

Work Requirements

66.7

70.0

-3.3

.999

Short-Term Assistance

45.5

40.0

5.5

.527

Term-Limited Assistance

48.5

40.0

8.5

.527

Waivers for HCV Programs

(n=30)

(n=8)

Payment Standards and Rent Reasonableness

50.0

50.0

0.0

.999

Voucher (Landlord) Leasing Incentivesa

53.3

37.5

15.8

.480

PBV – Elimination of Competitive Process

26.7

12.5

14.2

.414

PBV – Alternate Competitive Process

10.0

25.0

-15.0

.285

Increase Total PBV Cap

20.0

25.0

-5.0

.999

Increase PBV Development Cap

16.7

25.0

-8.3

.285

PBV – Unit Types – Shared Housing

16.7

25.0

-8.3

.285

Increase PBV Rent to Owner

16.7

0.0

16.7

.285

Increase Elderly Age

13.3

25.0

-11.7

.285

Waiver for Public Housing Program

(n=20)

(n=7)

Public Housing Leasesb

35.0

14.3

20.7

.127

Notes: All the waivers specifically asked about in the MTW Application Survey are included in this exhibit. A PHA is reported as intending to use a waiver if their answer to whether they planned to use it is “yes” rather than “maybe” or “no”). The waivers applicable to either the HCV or PH program were asked about each program separately on the survey, but are reported here if the PHA marked that they intended to use the waiver in either program. P values are those associated with the chi-square statistic.

a Voucher Leasing Incentives was a proposed MTW Waiver at the time the Initial MTW Application Survey was fielded. This waiver was revised during the finalization of the MTW Operating Notice to become Landlord Leasing Incentives.

b Public Housing Leases was a proposed MTW Waiver at the time the Initial MTW Application Survey was fielded. This waiver was removed from the list of approved MTW Waivers during finalization of the MTW Operating Notice.

Source: Initial MTW Application Survey of 43 RCT PHAs, fielded in 2018–19

B.2 Baseline Characteristics and Balance of the QED PHAs

As with the RCT sample, for the QED sample we examine balance in the baseline characteristics to assess the extent to which the treatment and comparison groups are well matched. Of course, we can only consider that balance on variables for which we observe. Unobservable characteristics that led one PHA to apply for MTW and another not to apply may correlate with outcomes, which is why there is potential for impacts estimated in the QED analysis to be biased. Although we cannot control for unobservable characteristics about a PHA that might correlate with both the decision to apply for MTW designation and the outcomes of interest, we mitigate the risk of bias by ensuring that the Comparison PHAs closely match the treatment group on important, observable characteristics as measured in HUD administrative data.

Baseline Report – Evaluating MTW Flexibility for Smaller PHAs ▌pg. 56

Appendix B: Comparison of Treatment PHAs to Control and Comparison PHAs Our analysis of that balance leads to the conclusion that the selected 99 Comparison PHAs are well matched to the Treatment PHAs. There are no statistically significant differences between the groups, as shown in exhibit B-3.

Exhibit B-3. Comparison of Treatment and Comparison PHAs

Treatment

Comparison

PHAs

PHAs

Difference

Characteristic

(n=33)

(n=99)

(T-C)

p-Value

PHA Region

Percentage (number) in Northeast

27.3 (9)

27.3 (27)

0.0 p.p.

Percentage (number) in Southeast

27.3 (9)

27.3 (27)

0.0 p.p.

Percentage (number) in Midwest

12.1 (4)

12.1 (12)

0.0 p.p.

1.000 a

Percentage (number) in Southwest

18.2 (6)

18.2 (18)

0.0 p.p.

Percentage (number) in West

15.2 (5)

15.2 (15)

0.0 p.p.

Program Type

Percentage (number) of PHAs with HCV and public

housing

51.5 (17)

51.5 (51)

0.0 p.p.

1.000a

Percentage (number) of PHAs with only HCV

39.4 (13)

39.4 (39)

0.0 p.p.

Percentage (number) of PHAs with only public

housing

9.1 (3)

9.1 (9)

0.0 p.p.

Other Characteristics

Number of units (HCV and/or public housing)

507

461

47

.286

PHA headquarters is in a metropolitan area

72.7

66.7

6.1 p.p.

.521

Average earnings of nondisabled adults aged 18-61

$18,004

$17,323

$681

.418

Percentage of HCV households living in low-poverty

census tracts

19.3

17.6

1.6 p.p.

.227

Average HUD expenditure per unit per monthb

$689

$620

$69

.257

Percentage of households where head or co-head is

aged 62 or over OR has a disability

56.7

60.7

−4.0 p.p.

.155

Fair Market Rent (FMR) for 2-bedroom unit

$1,041

$1,075

−$32

.681

80% of Area Median Income (AMI) for 4-person

household

$59,329

$60,354

−$1,025

.698

Ratio of 2-bedroom FMR to 80% of AMI for 4-

person household

0.206

0.209

−0.003

.719

Average weekly wage for local government workers

in the county where the PHA is located

$886

$894

−$7

.889

Percentage of households in special voucher

programs

5.3

3.7

1.5 p.p.

.373

Notes: The abbreviation “p.p.” indicates “percentage points.” A p-value less than 0.10 means that if we drew two random samples from the same population, such a difference would occur less than 10 percent of the time. Two PHAs in the RCT were missing data for all 2019 POSH

metrics except for public housing and HCV unit counts. 2018 POSH data were substituted for these metrics for these two PHAs.

a Indicates that the p-value is from a Chi-square test. Unless indicated otherwise, p-values come from t tests of simple comparison of means, using a pooled variance estimate.

b t test uses Satterthwaite variance estimate due to unequal variance.

Sources: HUD 2018 and 2019 Picture of Subsidized Households (POSH), HUD December 2019 Inventory Management System/PIH

Information Center (PIC), HUD FY20 Fair Market Rents, HUD December 2019 Voucher Management System (VMS), HUD FY2019 Section 8

administrative data, Bureau of Labor Statistics 2019 Q3 local government wages for all industries, counties, and establishment sizes

Baseline Report – Evaluating MTW Flexibility for Smaller PHAs ▌pg. 57

Appendix C. Outcome Measures’ Definitions & Construction

This appendix provides definitions and construction details of the outcome variables. Bold font and shading distinguish the confirmatory outcomes.

Exhibit C-1. PHA Cost and Cost-Effectiveness (per Household per Month)

Outcome

Construction Details

Total operating and

Source: FDS. We use the most recent annual report for a PHA on file, as of administrative expenditure

December 31st of the analysis year (2020 for this baseline report). We exclude CARES Act funding because it is a one-time funding source; for both

treatment and control, that funding will be almost entirely spent before any treatment PHA has MTW designation and funding flexibility.

Numerator is the sum of:

a) HCV and LNT expenditures for the year (FDS Line 96900: Total

Operating Expenses, FDS Line 97300, 97350: HAP expenditures); plus

b) HCV and LNT HAP expenditures

c) Public housing operating expenses minus utilities (FDS Line 96900:

Total Operating Expenses minus Line 93000: Total Utilities, Sum of all

PH projects). This measure excludes utilities from public housing operating costs to allow for a more consistent comparison between PHAs. Some PHAs pay utilities directly, so the cost shows up on FDS (these PHAs collect more total tenant [rent] payment because there is no utility allowance for tenants); other PHAs do not pay these costs directly so it does not show up on FDS (but collect less in total tenant payment because provide a utility allowance).

Denominator is the sum of:

a) Total HCV and LNT unit months leased (FDS 11210: Number of Unit

Months Leased); plus

b) Total public housing unit months leased (FDS 11210: Number of Unit

Months Leased)

Public housing operating

Source: FDS. We use the most recent annual report for a PHA on file, as of expenses minus utilitiesPH

December 31st of the analysis year (2020 for this baseline report). We exclude CARES Act funding.

Numerator is public housing operating expenses minus utilities (FDS Line

96900: Total Operating Expenses minus Line 93000: Total Utilities, Sum of all PH projects).

Denominator is the total public housing unit months leased (FDS 11210:

Number of Unit Months Leased)

HCV and local, non-traditional

Source: FDS. We use the most recent annual report for a PHA on file, as of programs’ administrative and

December 31st of the analysis year (2020 for this baseline report). We exclude tenant services expendituresHCV

CARES Act funding.

Numerator is the HCV and LNT administrative and tenant services

expenditures for the year (FDS Line 96900: Total Operating Expenses).

Denominator is the total HCV and LNT unit months leased (FDS 11210:

Number of Unit Months Leased)

HCV Housing Assistance

Source: VMS (HUD Voucher Data Dashboard). We sum the 2020 monthly

Payment (HAP) expendituresHCV

HAP payments (January through December) and divide by the sum of the

unit-months leased (January through December).

Total federal funding of MTW-

Source: FDS. We use the most recent annual report for a PHA on file, as of eligible programs

December 31st of the analysis year (2020 for this baseline report). We exclude CARES Act funding.

Numerator is the sum across PH and HCV projects of:

a) HUD PHA Operating Grants (FDS Line 70600); plus

b) Capital Grants (FDS Line 70610)

Baseline Report – Evaluating MTW Flexibility for Smaller PHAs ▌pg. 58

Appendix C: Outcome Measures’ Definitions

Outcome

Construction Details

Denominator is the sum across PH and HCV projects of unit months leased

(FDS 11210)

Total revenue of MTW-eligible

Source: FDS. We use the most recent annual report for a PHA on file, as of programs

December 31st of the analysis year (2020 for this baseline report). We exclude CARES Act funding.

Numerator is the sum across PH and HCV projects of Total Revenue (FDS

Line 70000).

Denominator is the sum across PH and HCV projects of unit months leased

(FDS 11210)

Cash reserves

Cash reserves is defined as total current assets minus net current liabilities.

Total current assets is equal to the sum of:

a) Cash–unrestricted (FDS 111)

b) Cash–tenant security deposits (FDS 114)

c) Cash–restricted for payment of current liabilities (FDS 115)

d) Total receivables (FDS 120)

e) Instruments–unrestricted (FDS 131)

f) Instruments–restricted for payment of current liabilities (FDS 135)

g) Pre-paid expenses and other assets (FDS 142)

h) Interprogram Due From (FDS 144)

i) Assets held for sale (FDS 145)

Net current liabilities is equal to total current liabilities (FDS 310) minus current proportion of LTD-capital projects (FDS 343-10)

Key: CARES=Coronavirus Aid, Relief, and Economic Security. FDS=Financial Data Schedule. HAP=Housing Assistance Payment.

HCV=Housing Choice Voucher program. LNT=local, non-traditional. LTD= long-term debt. PH=public housing. VMS=Voucher Management System.

Source: Abt-derived definitions

Baseline Report – Evaluating MTW Flexibility for Smaller PHAs ▌pg. 59

Appendix C: Outcome Measures’ Definitions

Exhibit C-2. Self-Sufficiency on Nonelderly, Nondisabled Households

Outcome

Specification

Household earnings in the last

Sum of most recent four quarters of earnings from the NDNH data.

four quarters

Universe includes household members age 14 and over who do not

have a disability and who reside in a nonelderly, nondisabled household.

Nonelderly, nondisabled households are those where the head and

spouse, (or co-head) are under age 62 and do not have a disability. The

sample is restricted to nonelderly, nondisabled households who resided

in PHAs as of December 31st of the calendar year of interest (in this

report: 2020). We exclude households who exited the PHA before

December 31, 2020. Individual-level data are merged to the household

level using a household identifier.

Household employment (has

Most recent quarter of earnings from the NDNH data. Universe includes

earnings) in last quarter

household members age 14 and over who do not have a disability and

who reside in a nonelderly, nondisabled household. Nonelderly,

nondisabled households are those where the head and spouse (or co-

head) are under age 62 and do not have a disability. The sample is

restricted to nonelderly, nondisabled households who resided in PHAs

as of December 31st of the calendar year of interest (in this report:

2020). We exclude adults in households with an exit record in 2020.

Average household earnings after

Earnings in quarter after exit. Universe includes household members

exit, in most recent quarter after exit

age 14 and over who do not have a disability and who reside in a

nonelderly, nondisabled household. with an exit record in 2020.

Nonelderly, nondisabled households are those where the head and

spouse (or co-head) are under age 62 and do not have a disability.

Percentage of heads of nonelderly,

Equal to one (zero) if earnings in quarter after exit exceed (do not

nondisabled households exiting

exceed) 2.5 times the local FMR. Universe includes household

housing with earnings equivalent to

members age 14 and over who do not have a disability and who were

2.5 times the local FMR

member in nonelderly, nondisabled households with an exit record in

2020. Nonelderly, nondisabled households are those where the head

and spouse (or co-head) are under age 62 and do not have a disability.

Key: AMI=Area Median Income. FMR=Fair Market Rent. HCV=Housing Choice Voucher. NDNH=National Directory of New Hires. PIC= PIH

Information Center.

Source: Abt-derived definitions

Baseline Report – Evaluating MTW Flexibility for Smaller PHAs ▌pg. 60

Appendix C: Outcome Measures’ Definitions

Exhibit C-3. Households’ Housing Choice: Opportunity Neighborhoods

Outcome

Specification

Percentage of HCV households living in

We define a census tract as low poverty if it is in the first quartile

low-poverty census tractsHCV

of the PHA’s jurisdiction’s distribution of poverty rates OR a

poverty rate of less than 10 percent (the national poverty rate in

2019 was 10.5 percent).

We define the distribution of the poverty rate in a PHA’s

jurisdiction as the population poverty rate of each of its census

tracts, (5-year ACS, 2019: S1701_C03_001E) weighted by the

number of renter-occupied units in that census tract (5-year ACS,

2019: DP04_0047E). The PHA’s jurisdiction includes census

tracts where at least 2 percent or five or more (whichever is

smaller) of its tenants live, and all abutting census tracts. Voucher

households only.

Percentage of HCV households with children We consider households with a member less than age 18 as of living in low-poverty census tractsHCV

the last day of the calendar year under study (December 31st,

2020) to be a “household with children”. Voucher households only.

Percentage of HCV households with children Similar to the two items above, except that we use the median living in a census tract with a poverty rate

poverty rate of the jurisdiction instead of the first quartile. Voucher

below the median for the PHA’s

households only.

jurisdictionHCV

Percentage of public housing households

Same poverty definition as confirmatory outcome. Public housing

living in low-poverty census tractsPH

households only.

Percentage of public housing households

Same poverty definition as confirmatory outcome. We consider

with children living in low-poverty census

households with a member less than age 18 as of the last day of

tractsPH

the calendar year under study (December 31st, 2020) to be a

“household with children.” Public housing households only.

Key: ACS=American Community Survey. HCV=Housing Choice Voucher program. PH=public housing program.

Source: Abt-derived definitions

Baseline Report – Evaluating MTW Flexibility for Smaller PHAs ▌pg. 61

Appendix C: Outcome Measures’ Definitions

Exhibit C-4. Households’ Housing Choice: Supply of Quality, Affordable Housing Outcome

Specification

Total number of households served through HCV and

Sum of: (a) Average of “unit months leased” in each

public housing programs in average month

month of calendar year from VMS. (b) Number of public

housing households housed in the last month of the

year.

HCV unit utilization rateHCV (%)

The sum of unit months leased over the calendar year

divided by the sum of unit months available over the

calendar year, both in the VMS dataset.

HCV budget utilization rate HCV (%)

Sum of HAP payments in the twelve months of the

calendar year (2020) divided by the sum of HAP budget

authority in the twelve months of the calendar year

(2020), both from the VMS dataset.

Public housing occupancy rate PH (%)

Number of households in the public housing program as

of December 31, 2020 (available from PIC) divided by

the number of units in that PHA’s public housing projects

(from HUD’s Open Data website).

Number of unique HCV landlords per 100 voucher

The number of unique TINs at a PHA (as observed in

households HCV

PIC data) divided by the number of voucher households

(as observed in the PIC data), then multiplied by 100.

The same landlord can use different TINs at different

properties, but this is thought to be relatively rare.

Public housing units scoring 90 or above on most

Average of REAC scores for each public housing

recent physical inspection (%)PH

property, weighted by the number of units in that

property (from HUD’s Open Data website).

Economic useful life of public housing units PH

The ratio of total capital assets net of depreciation

(excluding land) and total capital assets including

accumulated depreciation. This data is available from

FDS, we use the PHA’s most recent fiscal year reporting.

(ITEM_160 - ITEM_161)/(ITEM_160 - ITEM_161 -

ITEM_166)

We compute this ratio for each public housing project,

and then average the ratios, weighting by the number of

public housing unit-months for that project in its most

recent fiscal year.

Key: HAP=Housing Assistance Payment. HCV=Housing Choice Voucher. PH=public housing. PIC= PIH Information Center. REAC=Real Estate Assessment Center. TIN=Taxpayer Identification Number. VMS= Voucher Management System.

Source: Abt-derived definitions

Baseline Report – Evaluating MTW Flexibility for Smaller PHAs ▌pg. 62

Appendix C: Outcome Measures’ Definitions

Exhibit C-5. Households’ Housing Choice: Hard-to-House Populations

Outcome

Mean

Percentage of households served that have a

This measure includes households in public housing and

nonelderly family member with a disability

households in the HCV program as of December 31,

2020. We use the disability indicator in the member-level

file of the PIC data.

Percentage of households served that have three or

This measure includes households in public housing and

more minors

households in the HCV program as of December 31,

2020. This indicator variable is equal to one if the

household includes at least three members under the

age of 18 as of December 31, 2020.

Percentage of households served that were homeless This measure includes households in public housing and at the time of admission

households in the HCV program as of December 31,

2020, who have non-missing information about their

homelessness status at time of admission.

Source: Abt-derived definitions

Baseline Report – Evaluating MTW Flexibility for Smaller PHAs ▌pg. 63

Appendix C: Outcome Measures’ Definitions

Exhibit C-6. Other Tenant Outcomes: Households Served

Outcome

Specification

Proportion of newly admitted households with income at or below the greater of 30% of Area Median income or the federal poverty level

Overall Using PIC records, we define this measure for all

households with a program “entry” record in 2020. We

Public housing PH use total unadjusted household income and compare it

to the PHA’s jurisdiction’s area median income. We

define the PHA’s jurisdiction in the same fashion

described in Exhibit C-3. We use the 5-year ACS (2019)

median income data at the census tract level

Housing choice vouchers HCV (S1901_C01_012E), weighted by the number of renter-occupied units in that census tract. We adjust the 2019

ACS data using the CPI to 2020 dollars. We use the

federal poverty level in 2020 for a family of four: $26,200.

Number of households served in MTW-eligible programs

Overall This information is the same as presented in Exhibit 4-

4/C-4.

Public housing PH

Housing choice vouchers HCV

Percentage of households with at least one child aged < 18

Overall Using PIC records, we define this measure for all

households served by the program as of December 31st,

Public housing PH 2020. We consider households with a member less than

age 18 as of the last day of the calendar year under

Housing choice vouchers HCV study (December 31st, 2020) to be a “household with children”

Percentage of households whose head of household, spouse, or co-head is aged <62 and has a disability Overall Using PIC records, we define this measure for all

households served by the program as of December 31st,

Public housing PH 2020, computing the head of household’s, spouse, and

Housing choice vouchers HCV co-head’s age as of December 31st, 2020.

Source: Abt-derived definitions

Baseline Report – Evaluating MTW Flexibility for Smaller PHAs ▌pg. 64

Appendix C: Outcome Measures’ Definitions

Exhibit C-7. Other Tenant Outcomes: Households Exits

Outcome

Specification

Percentage of households ending participation in past 12 months

Overall We use PIC transaction records to flag

households with an “exit” record. We also

Public housing PH consider households with no PIC entry in the last

15 months as an “presumed exit in the last

twelve months”. The denominator is equal to the

Housing choice vouchers HCV number of households served in an average

month (HCV) or as of December 31st, 2020

(public housing).

Percent of households ending participation in the past 12 months with annual earnings exceeding 2.5 times FMR

Overall Conditional on having an exit record or

presumed exit (see above), we calculate

Public housing PH whether total household income (unadjusted) is

Housing choice vouchers HCV greater than 2.5 times the PHA’s FMR.

Source: Abt-derived definitions

Exhibit C-8. Other Outcomes: Rent Burden

Outcome

Mean

Average rent burden in the HCV program

Overall Rent burden is calculated as the ratio of the

total tenant payment (“TTP” in PIC) divided by

Household with children total household income (unadjusted). We

Households headed by person aged 62+ define this for the subgroups named in Exhibits 4-8 and 4-9 using program indicator variables,

member disability information, and age as of

December 31, 2020. All these fields are

Households headed by person younger than 62+ with a calculated from data in PIC. Values of rent disability burden are capped at one: i.e., if a household

has zero income or if the ratio of total tenant

payment to income is great than one, we set

rent burden equal to 1.

Source: Abt-derived definitions

Baseline Report – Evaluating MTW Flexibility for Smaller PHAs ▌pg. 65

References

Abravanel, M. D., R. E. Smith, M.A. Turner, E. Cove, L. Harris, and C. Manjarrez. 2004. Testing Public Housing Deregulation: A Summary Assessment of HUD’s “Moving to Work” Demonstration. Report prepared by the Urban Institute for the U.S. Department of Housing & Urban Development.

Buron, L., M. Vandawalker, and T. Morrill (with J. Khadduri, J. Lubell, and A. Shivji). 2017. Testing Performance Measures for the MTW Program. Report prepared by Abt Associates for the Public and Affordable Housing Research Corporation, a HAI Group Company.

Fischer, W., and B. Sard. (2016) . Sweeping Deregulation of Small Public Housing Agencies Would Do More Harm Than Good. Center on Budget and Policy Priorities.

Galster, G., D. Marcotte, M. Mandell, H. Wolman, and N. Augustine. 2007. “The Influence of Neighborhood Poverty During Childhood on Fertility, Education, and Earnings Outcomes,” Housing Studies 22:5,723–751. DOI:10.1080/02673030701474669.

Galvez, M., J. Simington, and M. Treskon. 2018. Moving to Work and Neighborhood Opportunity: A Scan of Mobility Initiatives by Moving to Work Public Housing Authorities. The Urban Institute.

Galvez, M., D. Teles, A. Oneto, and M. Gerken. 2021. Moving to Work Agencies’ Use of Project-Based Voucher Assistance. A report prepared by the Urban Institute for the U.S. Department of Housing and Urban Development.

Greenlee, A.J., H.B. Lee, and P.E. McNamara. 2018. “Performance of Small Public Housing Authorities: Opportunities and Barriers,” Cityscape, 20(3), 257–275.

Guzman, G., K.G. Posey, A. Bishaw, and C. Benson. 2018. Poverty Rates Higher, Median Household Income Lower in Rural Counties Than Urban Areas. U.S. Department of Commerce.

https://www.census.gov/library/stories/2018/12/differences-in-income-growth-across-united-states-

counties.html

Khadduri, J., M. Vandawalker, R. Cohen, and J. Lubell (with L. Buron, L. Freiman, and E. Kean). 2014.

Innovations in the Moving to Work demonstration. Report prepared by Abt Associates for the Public and Affordable Housing Research Corporation, a HAI Group Company.

Levy, D., L. Edmonds, and D. Long. 2020. “Moving to Work Agencies’ Use of Funding Flexibility,”

Cityscape, 22(3), 29–54.

Stacy, C., J. Leopold, D. Teles, A. Oneto, Y. Su, M. Gerken, and R. Gourevitch. 2020. “Fund More, Serve More, Save More: Moving to Work and Cost-Effectiveness,” Cityscape, 22(3), 193–217.

Treskon, M., M. Gerken, and M. Galvez. 2021. Housing Choice and Self-Sufficiency Outcomes at Moving to Work Agencies. Report produced by the Urban Institute for the U.S. Department of Housing and Urban Development.

Treskon, M, M. Gerken, and M. Galvez. 2020. “Can Diverse Activities Have a Combined Impact?

Examining the Effects of the Moving to Work Demonstration on Housing Choice and Self-Sufficiency Outcomes,” Cityscape, 22(3), 163–192.

Baseline Report – Evaluating MTW Flexibility for Smaller PHAs ▌pg. 66

References

Turnham, J., M. Finkel, L. Buron, M. Vandawalker, B. Kaul, K. Hathaway, and C. Kubacki. 2015.

Housing Choice Voucher Program Administrative Fee Study. Report prepared by Abt Associates for the

U.S. Department of Housing and Urban Development.

Turnham, J., L. Buron, J. Geyer, T. de Sousa, and L. Peck. 2021. Research Design/Data Collection and Analysis Plan: Evaluation of Cohort 1 of the Moving to Work Demonstration Program Expansion. A report prepared by Abt Associates for the U.S. Department of Housing and Urban Development.

United States Department of Agriculture (USDA). 2018. Rural Employment and Unemployment.

https://www.ers.usda.gov/topics/rural-economy-population/employment-education/rural-employment-

and-unemployment/

United States Government Accountability Office (GAO). 2012. Housing Choice Vouchers: Options Exist to Increase Program Efficiencies (GAO Publication No. 12-300).

United States Government Accountability Office (GAO). 2018. Rental Housing: Improvements Needed to Better Monitor the Moving to Work Demonstration, Including Effects on Tenants (GAO Publication No.

18-150).

Webb, M. D., K. Frescoln, and W. Rohe. 2015. Innovation in Public Housing: The Moving to Work Demonstration. Center for Urban and Regional Studies, University of North Carolina at Chapel Hill.

Baseline Report – Evaluating MTW Flexibility for Smaller PHAs ▌pg. 67

References

Key Terms Regarding PHAs

Minimum rent hardship exemption: PHA tenants may request a minimum rent hardship exemption under certain circumstances. This includes when there is a decrease or loss of income, a death in the family, or when the family may be evicted because of inability to pay.

Housing assistance payment (HAP): The amount a PHA pays directly to the landlord on behalf of an HCV tenant. It is typically the difference between the unit rent and the total tenant payment.

Inspection: Under the housing quality standards (HQS), PHAs inspect HCV units (typically on an annual basis) to ensure units comply with the requirements set to ensure safe, decent, and sanitary housing.

Local, non-traditional (LNT): Allowable activities that are outside of the scope of traditional HCV and public housing programs. Examples of LNT activities include shallow subsidy programs, self-sufficiency programs, and housing development programs.

Minimum rent: The minimum monthly rent charged to tenants. For public housing and the Housing Choice Voucher Program, PHAs can establish a minimum rent of up to $50.

Payment standards: The HUD-defined standard rent for a moderately-priced housing unit (which varies by family size). It is based on the fair market rent (FMR) for the area (based on the metropolitan area or nonmetropolitan county (standard FMR) or zip code (Small Area FMR)) and is used to determine maximum rent subsidy levels for HCV tenants.

Reexamination: Tenants’ income must be reexamined to determine their continued eligibility for the program and required contribution to rent. Reexaminations occur on a standard schedule, such as annually but can also occur on an interim basis (i.e., done ad hoc when earnings change).

Rent reasonableness: PHAs must validate that rent charged to HCV tenants are reasonable. To do so, the PHA must compare the rent for the voucher unit to rents for similar market rental units.

Total tenant payment (TTP or “tenant rent” ): The amount a household is required to pay in rent for their public housing or HCV unit. TTP is based on the household’s anticipated gross annual income, sometimes reduced by deductions or other adjustments.

Utility allowance: The Total Tenant Payment may include an adjustment for tenant-paid utilities. The utility allowance is the amount a PHA determines is necessary to cover the resident's reasonable utility costs such as electricity, heating, water and sewage service, and garbage collection.

Baseline Report – Evaluating MTW Flexibility for Smaller PHAs ▌pg. 68

U.S. Department of Housing and Urban Development

Office of Policy Development and Research

Washington, DC 20410-6000

May 2023

Document Outline

Evaluating MTW Flexibility for Smaller PHAs Baseline Report

DISCLAIMER

Foreword

Contents

List of Exhibits

Executive Summary

The First Cohort of MTW Expansion

Evaluation Design

Process Study

Impact Study

Key to PHA Sample Names

Key Findings from this Baseline Report

The most important objective motivating PHAs to apply for MTW was improving cost-effectiveness.

Nearly all Flexibility Cohort PHAs plan to use MTW flexibility to change their reexamination process or tenant rent policies.

Most Flexibility Cohort PHAs narrowed their planned use of waivers during the application process.

Many Flexibility Cohort PHAs do not yet have definitive plans for using their funding flexibility, but nearly all intend to use this flexibility at some point.

Analysis of baseline data shows that the impact study will provide unbiased estimates of the effect of MTW designation.

The 43 RCT PHAs are more urban than are all PHAs eligible to participate in this cohort.

In anticipation of the impact study, the evaluation team has pre-specified more than 30 outcome measures of PHA cost-effectiveness, tenant self-sufficiency, housing choice, and other tenant outcomes.

1. Introduction to the MTW Flexibility for Smaller PHAs Evaluation

1.1 Core Components of MTW

1.1.1 Statutory Objectives

1.1.2 Statutory Requirements

1.1.3 Funding Flexibility

1.1.4 Statutory and Regulatory Waivers

1.2 Overview of the First Cohort of MTW Expansion Agencies

1.2.1 Eligibility and Requirements for Flexibility Cohort

1.2.2 Implementation of Flexibility Cohort Demonstration to Date

1.3 Overview of the Evaluation Design

Process Study Design

Impact Study Design

2. The First MTW Expansion Cohort in Context

2.1 Prior Research on MTW Outcomes

2.2 Flexibility Cohort PHAs Compared to All Smaller PHAs

3. Motivation for Applying for MTW and Plans for Using Its Flexibility

3.1 Motivations for Applying for the MTW Demonstration

3.1.1 Factors Motivating Application

3.1.2 Most Important Statutory Objectives in PHAs’ Decision to Apply

3.2 Flexibility Cohort PHAs’ Plans for Using MTW Flexibility

3.2.1 Developing a Plan for Part 2 of the MTW Application

3.2.2 Current Plans for MTW Implementation

3.2.3 MTW Funding Flexibility

3.2.4 New Funding Sources

3.3 Changes in Flexibility Cohort PHAs’ Plans over Time

COVID-19 Waivers

3.4 Control PHAs’ Plans for Using MTW Flexibility

4. Baseline Values of Outcome Variables

4.1 PHA Cost-Effectiveness

4.2 Household Self-Sufficiency

4.3 Household Housing Choice

4.3.1 Opportunity Neighborhoods

4.3.2 Affordable Housing

4.3.3 Hard-to-House Populations

4.4 Other Tenant Outcomes

4.4.1 Households Served and Household Exits

4.4.2 Rent Burden

5. Summary

Appendix A. List and Map of Study PHAs

Appendix B. Comparison of Treatment PHAs to Control and Comparison PHAs

B.1 Baseline Characteristics and Balance of the RCT PHAs

B.2 Baseline Characteristics and Balance of the QED PHAs

Appendix C. Outcome Measures’ Definitions & Construction

References

Key Terms Regarding PHAs