The Rent Reform

Demonstration: Early

Effects on Employment

and Housing Subsidies

Prepared by:

James Riccio

Victoria Deitch

May 2019

Acknowledgments

This report reflects contributions from many people. 1 We are especially grateful to the families who are participating in the Rent Reform Demonstration at the Lexington-Fayette Urban County Housing Authority, Louisville Metro Housing Authority, San Antonio Housing Authority, and District of Columbia Housing Authority whose experiences have informed this report. We also want to give a heartfelt thanks to the staff members of these agencies, who have been operating the new rent policy and conducting recertifications. They are too numerous to list by name, but the study depends greatly on their efforts to implement the alternative rent policy with care. In addition, we appreciate the assistance of agency staff members who were deeply involved in designing the policy and in determining how best to implement it (and the study procedures) or who have supported their agencies’ continued engagement in the demonstration. These include, in Lexington, Executive Director Austin Simms, with Andrea Wilson, Aldean Pleasant, and Tonya Christopher; in Louisville, former Executive Director Tim Barry, current Executive Director Lisa Osanka, with Deborah Gilbert and Tracy Holmes-Bel ; in San Antonio, Executive Director David Nisivoccia, with Richard Milk, Pat Ortega, Brandee Perez, Gary Baxter, Priscilla Salazar, and Valerie Ochoa; and in Washington, D.C., current Executive Director Tyrone Garrett, former Executive Director Adrianne Todman, with Kimberley Cole, Ronald McCoy, Anissa Jones, Lori Parham, Jannie Lebby, Fatima Koroma, and Rani Joseph.

We are grateful to many staff members at the U.S. Department of Housing and Urban Development (HUD) for their valuable feedback on the evaluation and for reviewing drafts of this report. These include, from the Office of Public and Indian Housing, Dominque Blom, Marianne Nazzaro, Alison Christensen, John Concannon, and Ebony Gayles; and from the Office of Policy Development and Research, Todd Richardson, Mark Shroder, Carol Star, Lynn Rodgers (who was particularly helpful on the use and interpretation of HUD data), Paul Joice, Meena Bavan, George Carter, Nathan Bossie, and especial y Marina Myhre. Marina Myhre serves as our HUD Contracting Officer’s Representative and has been a source of steady guidance and a careful and insightful reviewer of all technical assistance and research activities, including the production of this report.

We thank those reviewers who serve on our expert panel and have given us comments on the study and this report: Nicole Barrett and Emily Warren of the Council of Large Public Housing Authorities; Janelle Beverly of the Fairfax County Redevelopment and Housing Authority; Elayne Weiss and Ed Gramlich of the National Low-Income Housing Coalition; Tushar Gurjal of the National Association of Housing and Redevelopment Officials; Lawrence Katz of Harvard University; Noelle Porter of the National Housing Law Project; Andrew Lofton of the Seattle Housing Authority; Larry Orr (consultant); Greg Russ of the Minneapolis Public Housing Authority; Barbara Sard of the Center on Budget and Policy Priorities; and Michael Wiseman of George Washington University.

1The Acknowledgments list the institutions with which individuals were affiliated at the time they contributed to the work reflected in this report.

iii

We acknowledge our research partners from other organizations who played a crucial role in the design of the new rent policy and the evaluation, worked directly with the housing agencies to implement the new rent rules and research procedures, or provided feedback on the evaluation: John Goering of the City University of New York, Don Davis of the Bronner Group, Ingrid Gould Ellen of New York University, Roberta Graham and Jessica Porter of Quadel, Marty Abravanel and Diane Levy of the Urban Institute, and Barbara Fink (consultant).

At MDRC, Nandita Verma provided careful review and guidance on all aspects of the analysis and on each draft of the report. Jonathan Bigelow and Keith Olejniczak worked with the housing agencies to monitor their implementation of the new rent rules. Gordon Berlin, Charles Michalopoulos, and Howard Bloom provided important feedback on the analysis, and Cynthia Miller and Ingrid Gould Ellen reviewed drafts of the report. Josh Vermette and Audrey Yu processed and analyzed the quantitative data, Katerina Galkin assisted in data checking, and Andrew Rock prepared the exhibits and assisted with fact-checking. Alice Tufel edited an earlier draft of the report, and Ann Kottner helped prepare it for HUD’s editor. Finally, we appreciate the support of our resource managers, Crystal Ganges-Reid and Stephanie Rubino.

DISCLAIMER

The contents of this report are the views of the contractor and do not necessarily reflect the views or policies of the U.S. Department of Housing and Urban Development or the U.S. Government.

iv

Contents

ACKNOWLEDGEMENTS ...................................................................................................... iii

FOREWORD ........................................................................................................................ x

EXECUTIVE SUMMARY .................................................................................................... ES-1

HUD’S TRADITIONAL RENT POLICY ............................................................................................. ES-2

OVERVIEW OF THE NEW RENT POLICY ......................................................................................... ES-3

EVALUATION DESIGN, SAMPLE CHARACTERISTICS, AND DATA SOURCES ............................................. ES-5

CHARACTERISTICS OF ENROLLED FAMILIES .................................................................................... ES-6

EARLY FINDINGS ON HOUSEHOLD HEADS’ EMPLOYMENT AND EARNINGS .......................................... ES-6

EARLY IMPACTS ON OUTCOMES RELATED TO HOUSING SUBSIDIES .................................................. ES-12

CONCLUSION ......................................................................................................................... ES-19

INTRODUCTION .................................................................................................................. 1

HUD’S TRADITIONAL RENT POLICY .................................................................................................. 3

OVERVIEW OF THE NEW RENT POLICY .............................................................................................. 4

ADMINISTERING THE NEW RENT POLICY ............................................................................................ 8

TTPS AND RENT SUBSIDIES AT THE START OF THE NEW RENT POLICY ................................................... 10

THE SCOPE OF THIS REPORT .......................................................................................................... 11

THE STUDY SAMPLE AND ANALYSIS METHODS .................................................................. 12

BUILDING THE RESEARCH SAMPLE .................................................................................................. 12

CHARACTERISTICS OF ENROLLED FAMILIES ....................................................................................... 15

DATA SOURCES AND FOLLOWUP PERIOD ......................................................................................... 21

COMPARABILITY OF THE TWO RESEARCH GROUPS ............................................................................ 23

ANALYSIS ISSUES AND STRATEGIES .................................................................................................. 27

LOUISVILLE OPT-OUTS ................................................................................................................. 30

EARLY IMPACTS ON EMPLOYMENT AND EARNINGS ........................................................... 31

EARLY IMPACTS FOR HEADS OF HOUSEHOLD .................................................................................... 32

IMPACTS FOR OTHER ADULTS AND ALL ADULTS IN THE HOUSEHOLD .................................................... 57

CONCLUSION .............................................................................................................................. 58

EARLY IMPACTS ON HOUSING-RELATED OUTCOMES .......................................................... 59

CALCULATING FAMILIES’ CONTRIBUTIONS TO THEIR HOUSING COSTS ................................................... 59

EARLY IMPACTS ON FAMILIES’ HOUSING EXPENDITURES AND SUBSIDIES ............................................... 62

PAYING THE MINIMUM TTP .......................................................................................................... 76

USE OF SAFEGUARDS BY THE NEW RENT RULES GROUP ..................................................................... 78

INTERIM RECERTIFICATIONS .......................................................................................................... 80

HARDSHIP REMEDIES ................................................................................................................... 80

IMPACTS ON PHA ACTIONS FOR FAMILIES ....................................................................................... 82

CONCLUSION .............................................................................................................................. 89

SUMMARY AND NEXT STEPS IN THE RENT REFORM DEMONSTRATION............................... 91

APPENDIX A ...................................................................................................................... 96

APPENDIX B ...................................................................................................................... 97

STRATEGY FOR ADDRESSING MULTIPLE OUTCOME MEASURES IN THE RENT REFORM DEMONSTRATION .... 97

APPENDIX C ..................................................................................................................... 108

REFERENCES .................................................................................................................... 116

v

List of Exhibits

Exhibit ES.1. Early Impacts on Employment and Earnings: Heads of Households .................. ES-9

Exhibit ES.2. Early Impacts on Employment and Earnings, by Public Housing Agency

(PHA), Heads of Households ............................................................................ ES-10

Exhibit ES.3. Early Impacts on Families’ Subsidy Receipt and Housing Costs ....................... ES-14

Exhibit ES.4. Early Impacts on Families’ Subsidy Receipt and Housing Costs,

by Public Housing Agency (PHA) ...................................................................... ES-17

Exhibit 1.1. Comparison of Traditional and New Rent Policies .................................................. 5

Exhibit 1.2. Total Tenant Payment (TTP) at Initial Study Recertification, Al Public

Housing Agencies (PHAs) ...................................................................................... 10

Exhibit 2.1. Enrol ment Flow Chart .......................................................................................... 14

Exhibit 2.2. Characteristics of Families in the Impact Sample, by Public Housing

Agency (PHA) ........................................................................................................ 18

Exhibit 2.3. Characteristics of Heads of Households in the Impact Sample,

by Public Housing Agency (PHA) ........................................................................... 18

Exhibit 2.4. Characteristics of Adults Who Are Not Heads of Households,

by Public Housing Agency (PHA) ........................................................................... 19

Exhibit 2.5. Data Sources for This Report ................................................................................ 21

Exhibit 2.6. Simplified Depiction of Random Assignment and Fol owup Period ...................... 22

Exhibit 2.7. Random Assignment Period, New Rent Effective Dates, and Year 1

End Dates, by Public Housing Agency (PHA) ......................................................... 24

Exhibit 2.8. Employment and Earnings in the Three Quarters Before Random

Assignment, by Household Head Status ............................................................... 26

Exhibit 3.1. Quarterly Impacts on Household Heads’ Employment and Earnings,

All Public Housing Agencies (PHAs) ....................................................................... 33

Exhibit 3.2. Early Impacts on Employment and Earnings: Heads of Households ..................... 35

Exhibit 3.3. How to Read the Impact Tables in This Report ..................................................... 39

Exhibit 3.4. Quarterly Impacts on Household Heads’ Employment and Earnings,

by Public Housing Agency (PHA) ........................................................................... 40

Exhibit 3.5. Early Impacts on Employment and Earnings, by Public Housing Agency

(PHA): Heads of Households ................................................................................. 44

Exhibit 3.6. Early Impacts on Employment and Earnings, by Employment Status

in the Quarter Before Random Assignment: Heads of Households ...................... 50

vii

Exhibit 3.7. Early Impacts on Employment and Earnings, by Employment Status

at Random Assignment and Public Housing Agency (PHA): Heads of

Households ........................................................................................................... 53

Exhibit 3.8. Early Impacts on Employment and Earnings, by Age of Youngest Child

in the Household at the Time of Random Assignment: Heads of Households ..... 56

Exhibit 4.1

Total Tenant Payment and Family Share .............................................................. 61

Exhibit 4.2. Impacts on Families’ Housing Costs and Subsidies in Year 1, Al

Public Housing Agencies (PHAs) ........................................................................... 63

Exhibit 4.3. Selected Impacts on Tenants’ Housing Costs and Subsidies in Year 1,

All Public Housing Agencies (PHAs) Except Washington, D.C. .............................. 65

Exhibit 4.4. Impacts on Families’ Housing Costs and Subsidies in Year 1, by

Public Housing Agency (PHA) ................................................................................ 69

Exhibit 4.5. Total Tenant Payment (TTP) Relative to the Local Minimum TTP and

Use of Safeguards in Year 1, by Public Housing Agency (PHA), New

Rent Rules Group Only ......................................................................................... 77

Exhibit 4.6. Safeguards Built into the New Rent Rules Policy .................................................. 79

Exhibit 4.7. San Antonio Housing Authority’s Rent Reform Flyer ............................................ 80

Exhibit 4.8. Public Housing Agency (PHA) Actions in Year 1, All PHAs ..................................... 84

Exhibit 4.9. Public Housing Agency (PHA) Actions in Year 1, by PHA ....................................... 87

Exhibit 5.1. Summary of Early Impacts on Employment, Earnings, and Housing

Subsidy-Related Outcomes ................................................................................... 92

Exhibit A.1. Existing Rent Policies of Public Housing Agencies (PHAs) Participating

in the Rent Reform Demonstration ...................................................................... 96

Exhibit B.1. Employment and Earnings in the Three Quarters Before Random

Assignment, by Public Housing Agency (PHA): Heads of Households ................. 104

Exhibit B.2. Employment and Earnings in the Three Quarters Before Random

Assignment, by Public Housing Agency (PHA): Adults Who Were Not

Heads of Households .......................................................................................... 106

Exhibit C.1. Treatment-on-Treated (TOT) Early Impacts for Selected Outcomes,

Louisvil e ............................................................................................................. 108

Exhibit C.2. Early Impacts on Employment and Earnings: Adults Who Were Not

Heads of Households .......................................................................................... 109

Exhibit C.3. Early Impacts on Employment and Earnings, by Public Housing Agency

(PHA): Adults Who Were Not Heads of Households ........................................... 109

viii

Exhibit C.4. Early Impacts on Employment and Earnings: Household Heads and

Other Adults ....................................................................................................... 111

Exhibit C.5. Impacts on Employment and Earnings, by Public Housing Agency

(PHA): Household Heads and Other Adults ........................................................ 113

Exhibit C.6. Average Earnings by Quarter Relative to the Quarter of Random

Assignment, by Public Housing Agency (PHA) Control Group: Heads

of Households Randomly Assigned in Quarter 2 of 2015 ................................... 114

ix

Foreword

In 1969, a landmark piece of national affordable housing legislation, sponsored and subsequently named for U.S. Senator Edward Brooke, was enacted into law. This law, the Brooke Amendment, established limitations on the rents charged to families and individuals in federally assisted housing. Initially, the Brooke Amendment limited rent charges to 25 percent of an assisted family’s income. Over time, numerous changes were made to the basic rent setting policy, raising the threshold to 30 percent (enacted in 1981) and adding numerous adjustments, exclusions and deductions, as well as adding minimum and ceiling rent options.

Over the last few decades, critics have suggested that the Brooke Amendment, in its pursuit of safeguarding affordability, creates a disincentive to work by dampening tenant motivation to earn more income. In response, The Department of Housing and Urban Development (HUD) has undertaken a Rent Reform Demonstration to comprehensively test alternatives to the current rent-setting requirements for one of its key, and largest, housing assistance programs: Housing Choice Vouchers (HCV). The Demonstration has three key goals it is testing. They are specifically, how to:

• Incentivize employment for work-eligible individuals

• Reduce the complexity and administrative burden for PHAs

• Avoid unnecessary hardship on assisted families

This demonstration has been underway since 2015 and in that time over 6,600 families have been randomly assigned to either the alternative rent rules or a control group subject to the existing rules at the four Public Housing Agencies (PHAs) that are participating in the demonstration. This report, “Early Effects”, is one of two reports being released simultaneously on the Rent Reform Demonstration. It presents findings on the new rent policy’s effects, or “impacts,”

on household heads’ labor market and housing-related outcomes from the first 12 to 18 months after the alternative rent model went into effect for the treatment group. The other report, “Interim Findings,” presents results from the second fol owup period, 27 to 30 months after the alternative rent model was implemented for the treatment group.

The early results indicate that when the findings for all four PHAs are combined, the new policy generated a small statistically significant increase in heads of household’s quarterly employment rate. When findings for all of the PHAs except Washington, DC are combined, there is a small statistically significant increase in both quarterly employment rates and Year 1 average annual earnings. The story, however, varied substantial y across locations. There were some positive effects on earnings in Lexington, on earnings and employment in San Antonio, but not in Louisvil e and Washington, D.C. The report also presents other early effects indicating larger housing subsidies and longer tenure in the voucher program, which are expected short-term results related to the use of triennial income recertification to establish tenant rent contributions, increased use of hardship remedies, decreased PHA administrative actions, and some preliminary subgroup findings.

x

Given the early nature of this review, it is too soon to draw firm conclusions on the triennial recertification (the element of the model most expected to influence employment and earnings) and whether it is having its intended effect. However, the findings indicate that a new minimum rent does not seem to have any short-term impact on employment or earnings, based on the lack of employment or earnings effects in Washington, DC where the minimum rent increased from $0 to $75.

Future reports wil assess the impact of the alternative rent model, including any long-term effects, through a fol owup survey. There wil be an additional interim report, expected in 2021, with the final report gathering 6 years of data (covering two triennial recertifications), scheduled for 2023.

Seth D. Appleton

Assistant Secretary for Policy Development and Research

U.S. Department of Housing and Urban Development

xi

Executive Summary

For many years, housing subsidies for people who receive rental assistance vouchers have, according to critics, created a disincentive for tenants to work. Because voucher holders pay more toward their rent and utilities as their incomes rise, they face an implicit marginal “tax” on increased earnings (approximately 30 percent). Critics of this traditional rent policy also believe that it imposes a substantial and costly administrative burden on public housing agencies (PHAs), in part because it requires them to adjust subsidies up or down, as families’ incomes fall or rise, and to apply complicated rules in determining subsidy levels.

Strong evidence is lacking on whether any alternative rent policies substantially improve tenants’ employment outcomes, or on what effects they might have on families’ receipt of housing subsidies or PHAs’ administrative burden and costs. In a step toward promoting innovative rent subsidy policies and building better evidence on “what works,” the U.S.

Department of Housing and Urban Development (HUD) launched the Rent Reform Demonstration. The demonstration, which began enrolling voucher holders in 2015, focuses on recipients of tenant-based housing choice vouchers in four cities and PHAs: Lexington-Fayette Urban County Housing Authority (generally referred to as the Lexington Housing Authority), in Lexington, KY; Louisvil e Metropolitan Housing Authority, in Louisvil e, KY; San Antonio Housing Authority, in San Antonio, TX; and District of Columbia Housing Authority, in Washington, D.C. These housing agencies are a subset of 39 PHAs that, at the time the project was launched, were part of HUD’s Moving to Work demonstration program, which allows selected PHAs more administrative flexibility in operating their housing assistance programs. HUD selected MDRC and its partners to lead the initiative, 2 working closely with HUD and the four PHAs to design and evaluate an alternative rent policy using a randomized control ed trial.

This report is the second in a series of five reports on the Rent Reform Demonstration. MDRC prepared the initial “Baseline” report in the series. Published by HUD in 2017, the Baseline report describes the origins of the Rent Reform Demonstration, the selection of PHAs, the features of the new policy, the rationale behind each of its main elements, the PHA’s initial implementation experiences, and the manner in which the policy is being evaluated (Riccio, Deitch, and Verma, 2017).

This current "Early Effects" report presents early findings on the effects of the alternative rent policy on the employment and earnings of low-income adults who receive rental assistance through the federal Housing Choice Voucher (HCV) program and on a variety of outcomes related to their housing subsidies. The report provides the first look at the new rent policy’s effects, or “impacts,” on families’ labor market and housing-related outcomes. It covers 2The study team includes the Urban Institute, the Bronner Group, Quadel Consulting & Training, and professors Ingrid Gould Ellen (New York University) and John Goering (City University of New York).

ES-1

a followup period for each eligible family of approximately 12 to 18 months after the new policy took effect.

Next in the series is the "Interim Findings" report. It presents results from the first 27-30 months of followup. The fourth report (expected in 2021) will provide findings from the long-term followup survey and the first triennial recertification, about 36 months after the new rent policy effective-date. The fifth and final report (expected in 2023) will present findings after the second triennial recertification, about 72 months after the new rent policy effective date.

The 12- to 18-month followup timeframe of this report is too short for a ful assessment of the new policy’s effects, in part because a critical feature of the policy—new rules governing families’ contributions to their rent and utilities—lasts 3 years. The report, nevertheless, does show whether or not the new policy began to effect change in the labor market and in housing-related outcomes relatively soon after implementation.

The results indicate that when the findings for all four PHAs are combined, the new policy did not generate statistically significant increases in tenants’ average earnings over the first 18 months of followup. The story varied substantially across locations, however, with some early positive effects on earnings in Lexington and San Antonio but not in Louisvil e and Washington, D.C. Across al four agencies, the new policy reduced certain types of PHA transactions with families during the first 12 months of followup, general y somewhat reduced families’ expenditures for rent and utilities, and reduced their likelihood of exiting the voucher program. Because their rent and utilities costs were capped and they were somewhat less likely than control group families to exit the voucher program, those in the new rent rules group received more in housing subsidies during this period than they would have received under existing rules, as intended by the new policy’s goal of reducing work disincentives.

HUD’s Traditional Rent Policy

Currently, most families who receive housing choice vouchers are expected to contribute 30

percent of pretax income (after certain adjustments)―known as the “total tenant payment”

(TTP) ―toward their rent and utilities. 3 The rules for calculating a family’s TTP allow several deductions from gross income (including a deduction for some childcare costs for working parents), yielding an “adjusted income” estimate. The calculation looks forward in time, basing the adjusted income estimate on the amount of income a family currently receives and anticipates receiving during the coming year (“current/anticipated” income in this report). The PHA pays the difference between the family’s TTP and the maximum combined rent and utilities cost that the PHA wil allow for rental units (depending largely on family size), called a 3Throughout this report, HUD’s “current” or “traditional” rent policy for voucher holders refers to the national rent policy in effect for non-Moving to Work PHAs before the passage of the Housing Opportunity Through Modernization Act of 2016.

ES-2

“payment standard.” PHAs are permit ed to establish a minimum TTP, or “minimum rent,” of up to $50 per month, although not al have done so.

This traditional “percentage of adjusted income” approach builds a strong safety-net feature into the rent subsidy system: If a family’s income falls, the family pays less toward its housing costs. This approach also implicitly “taxes” tenants for increasing their earnings, however, which some experts contend reduces their work effort. 4 The traditional rent policy also requires PHAs to review each family’s income and recertify their continued eligibility for the voucher program at least annually and to adjust TTPs and housing subsidies if families’

incomes change. The complex rules governing the calculation of income and rent are commonly considered to be administratively burdensome and prone to errors that can lead to improper payments. The new rent policy attempts to address those problems.

Overview of the New Rent Policy

The new rent policy developed for the Rent Reform Demonstration substantially alters the traditional rent subsidy approach for voucher holders. Its core features include a combination of elements that are intended to achieve a balance between increasing the financial incentives for tenants to try to increase their earnings, protecting families from excessive rent burden, reducing the PHAs’ administrative burden of operating a rent policy system, and containing subsidy and administrative costs. Those features include the following components:

• Recertifying families’ continued eligibility for the voucher program and recomputing their TTPs every 3 years rather than every year

o Under the triennial recertification schedule, if a family increases its earnings during the 3 years, its TTP wil not be raised, and its housing subsidy wil not be reduced until the end of Year 3.

• Changes in the formula for calculating a family’s TTP and subsidy o Eliminate al deductions from income, so that gross income, rather than adjusted income, is the basis for calculating a family’s TTP.

o Set a family’s TTP at 28 percent of gross income over the prior 12

months (referred to as “retrospective income”), rather than 30 percent of current/anticipated adjusted income.

o Ignore a family’s income from assets when the total value of its assets is less than $25,000 (and not require documentation of those assets).

4This is on top of possible reductions in other means-tested benefits families might be receiving (such as welfare or food stamps), making their combined marginal tax on increased earnings more than 30 percent.

ES-3

o Simplify the policy for determining utility al owances to a streamlined standard schedule based primarily on unit size, with some adjustments for more expensive utilities.

o Establish a minimum TTP of at least $50 per month and require

families to pay at least the specified minimum TTP directly to their

landlords.

• Safeguards for families (in addition to interim recertifications) o A TTP grace period at the start of the 3-year period, allowing for a temporary (6-month) TTP reduction when a family’s current/anticipated gross income is less than its retrospective gross income by more than 10

percent.

o Al owing one interim recertification per year if a family’s retrospective income falls by more than 10 percent before the next required triennial review.

o A hardship policy that covers a standard set of conditions and includes a standard set of remedies that permit TTP reductions for families meeting those conditions at any time during the 3-year period—to protect families from excessive rent burdens.

The PHAs participating in the demonstration helped to develop and support this common framework; however, they also saw a need to adapt the model in response to local considerations, or they had to accommodate some policy changes that they had already implemented. For example, the PHAs set their minimum TTPs at different levels, ranging from $50 to $150 per month. Two of the four PHAs—in Louisville and Washington, D.C.—

introduced a minimum TTP for the first time ($50 and $75, respectively), whereas San Antonio, which already had a minimum TTP, increased it for the demonstration from $50 to $100. Lexington had already introduced a $150 minimum TTP before the start of the demonstration, and it continued that policy for both the new rent rules group (the program group) and the existing rent rules group (the control group). The process for determining hardship remedies also varies, although the general conditions defining a hardship and the remedies themselves do not. 5

Of all the features of the new rent policy, the 3-year recertification is the main one intended to improve labor market outcomes because it eliminates the implicit “tax” on earnings during the 3-year period. The introduction of a minimum TTP, or the increase in an existing one, might also increase work effort because some tenants may need to increase their earnings 5 Lexington generally does not permit any reductions in TTPs below the minimum in its application of the demonstration’s hardship policy. The other three PHAs generally require families with zero income to report their family expenditures regularly to the PHA, but the ways they adjust TTPs for those families under the hardship remedies are the same.

ES-4

to have enough income to meet the new minimum. Various features of the new policy are intended to reduce the administrative burden on PHAs and to protect families from excessive rent burdens.

Evaluation Design, Sample Characteristics, and Data Sources

At the beginning of the study, the PHAs and MDRC identified existing voucher holders who would soon be scheduled for an annual recertification meeting to calculate their new TTPs and rental subsidies. Random assignment was then used to allocate families deemed eligible for the Rent Reform Demonstration to either a new rent rules group that would be subject to the new policy for the duration of the demonstration or to a control group that would continue to be subject to the existing rent rules. (Certain types of families, including those defined as elderly or disabled according to HUD criteria, were excluded from the demonstration.) 6

PHA staff calculated families’ TTPs and housing subsidies according to the rules of the rent policy group to which they were assigned. In Louisvil e, an opt-out option was offered to families assigned to the new rules group, meaning that they could choose to continue having their TTPs calculated according to the existing rent policy. By the end of the enrollment period, about 22 percent of the eligible families in Louisvil e’s new rent rules group chose to opt out of the new policy, although they did not opt out of the evaluation. 7 To minimize selection bias, the evaluation treats the opt-out families as members of the new rent rules group (rather than the existing rules group), even though they are subject to the existing rent rules. This decision means that the estimated effects of the new rent policy may be somewhat diluted in Louisvil e because not all members of the new rent rules group were exposed to the new policy.

Preexisting policies in several of the other PHAs also have implications for the evaluation. As mentioned previously, Lexington’s $150 minimum TTP applies to both research groups (and allows few hardship exemptions from the minimum TTP). Because this feature does not differ between the Lexington program and control groups, effects estimated for the Lexington sample on any labor market and housing-related outcomes wil not reflect any effects that the minimum TTP (as opposed to the policy’s other features) may have had on those outcomes. In contrast, Washington, D.C., had instituted a biennial recertification schedule for working-age, nondisabled families before joining the Rent Reform Demonstration. 8 This means that both the new rent rules group and the control group had their TTPs capped during the early portion of the study’s followup period. Consequently, when examining labor market outcomes for that PHA, it is reasonable to view this early period primarily as an opportunity to assess the 6For more details on this process, see Riccio, Deitch, and Verma (2017).

7In all four PHAs, families could refuse to allow their individually identified data to be shared with the researchers; however, only 14 families (0.2 percent of the pooled sample) chose to do so.

8At the time of site selection, the biennial policy applied only to families whose anticipated incomes increased by a small or modest amount (less than $10,000 per year from a single income source); those with income increases above that threshold were to continue with an annual recertification schedule. In June 2016, during the demonstration’s first followup year, the PHA removed the income threshold so that even those control group families (and other families who were not in the demonstration) with income increases above the threshold were switched to a biennial schedule.

ES-5

effects of its new $75 minimum TTP on those outcomes. Across the PHAs, differences such as these create opportunities for learning more about certain features of the new rent policy; however, they also mean that the “pooled” impact estimates (with all four PHAs combined) reflect the summary results of somewhat different tests in four locations and need to be interpreted with that in mind.

Characteristics of Enrolled Families

Across the four PHAs, a total of 6,665 families are included in the impact analysis. Al were randomly assigned between February 2015 and November 2015 in approximately equal numbers to either the new rent rules group or the control group. The average household had just over three family members at the time of study enrol ment. In addition, only about one-third of families had more than one adult living in the household, most of whom were the young adult children of the household heads. Nearly all (94 percent) of the household heads in the study sample were women and, on average, household heads were about 39 years old when they entered the study. In Lexington, Louisvil e, and Washington, D.C., most heads of household are Black/African-American, whereas the majority (75 percent) in San Antonio is Hispanic/Latino.

Data Sources and Followup Period

This report uses two main sources of quantitative data: PHA administrative records and unemployment insurance (UI) wage records obtained through the National Directory of New Hires (NDNH), which capture employer-reported employment and earnings. For considerations having to do with the structure of these two data sources, the “first year of fol owup” for the purposes of this report is defined as the period that begins after a family’s new TTP took effect, which is roughly the third quarter after families were randomly assigned.

Early Findings on Household Heads’ Employment and Earnings

The first fol owup year was a time when the new rent rules were stil new to families, limiting the potential effects of those rules. Of course, some tenants may seek to work, or not to work, for entirely unrelated reasons. For those whose decisions might be influenced by rent policies, however, the new policy’s implementation timeframe may matter. For example, it may have taken time for some families to understand how the policy supported work. Some adults who may have been inspired by the new rules to try to find work or increase their hours of work may have needed time to achieve those goals. Some may have needed time to overcome common types of impediments—for example, finding suitable job openings given their skil levels or arranging transportation or childcare. Thus, a fair assessment of the effects of the new policy must wait for longer-term data to become available; however, two additional quarters of employment and earnings data beyond the first fol owup year (Quarters 7 and 8) offer a peek into that longer-term period.

In examining the new policy’s effects on tenants’ earnings, the study focuses primarily on the household heads. This is because most of the non-heads of household were the young adult children of the household heads, many of whom (about 21 percent in the first year) were ES-6

no longer on the lease, possibly having moved away. This rate is expected to grow over time, limiting their exposure to the new or existing rent policies. 9

The evaluation design includes several “confirmatory” outcome measures related to tenants’ earnings, housing subsidies, and material hardships. These confirmatory outcomes reflect the most important variables for judging the intervention’s effectiveness. Given their primacy, impact findings on those outcomes were subjected to further statistical adjustments that hold them to a higher standard of evidence. Those adjustments account for the likelihood that in a study using many outcome variables, some impact estimates may emerge as statistically significant simply by chance and do not reflect true intervention effects. One can have more confidence in any confirmatory impact estimates that remain statistically significant after adjusting for the total number of confirmatory outcome measures. The current report treats first-year pooled impact estimates for household heads’ earnings and families’ housing subsidy receipt as preliminary confirmatory measures and subjects them to adjustment for multiple outcomes. The concluding evaluation report wil present the final confirmatory impact estimates and adjustments, using longer-term data on these measures, approximately 72 months after the new rent policy effective-date. It wil also include a survey-based family hardship scale as a confirmatory outcome measure (Benjamini and Hochberg, 1995).

• The results for the pooled sample show that heads of household in the new rent rules group were slightly more likely than those in the control group to work during Year 1.

Among household heads in the control group, with al four PHAs combined, a majority (66.8 percent) worked in a UI-covered job at some point during Year 1 (defined for this report as Quarter 3 through Quarter 6 after random assignment). The rate for the new rent rules groups was 68.2 percent, but the gain is not statistically significant; however, exhibit ES.1 shows that the new policy increased the proportion of household heads working in an average quarter by a statistically significant 1.6 percentage points (from 54.4 percent for the control group to 56

percent for the new rent rules group). Differences in Year 1 earnings were small and not statistically significant.

As mentioned previously, the pooled results of the new rent policy can be difficult to interpret because of differences across the PHAs in their minimum TTPs and control group conditions. Particularly important is the biennial recertification policy in effect for the control group in Washington, D.C. Because of that policy, the new rules group wil experience no meaningful advantage in terms of the triennial recertification until the third year of fol owup.

For this reason, in examining early results, it is useful to consider labor market impacts for the pooled sample without Washington, D.C., so that all families in the pooled control group are subject to an annual recertification policy. The second panel of exhibit ES.1 presents those results. It shows somewhat larger impacts of the new rent policy than the pooled estimates with 9Nonetheless, Appendix C in the full report presents findings for the effects on other adults as well as all adults combined (heads and non-heads of households).

ES-7

Washington, D.C, including a statistically significant increase in Year 1 average earnings of $466, a gain of 4.8 percent above the control group average of $9,660 per household head. 10

(This estimate remains statistically significant after adjustment for multiple hypothesis testing.)

• The new rent rules produced more positive results in Lexington and San Antonio than in Louisvil e and Washington, D.C.

In Lexington, the effects of the new policy on tenants’ earnings began to grow midway through Year 1 and were statistically significant in Quarters 7 and 8 (as shown in exhibit ES.2).

In Quarter 8, the average earnings for the new rules group were $352 higher than those of the existing rules group, a gain of 14 percent. 11 This earnings gain was achieved with little effect on employment rates, suggesting that it resulted largely from an increase in hours worked or in hourly wages.

In San Antonio, the new rent policy had statistically significant and positive effects on employment as well as on earnings outcomes. For example, the average quarterly employment rate during Year 1 increased by 3.2 percentage points relative to the control group rate. The impact on average earnings for Year 1 was $916 (an increase of 10 percent).

10Average earnings are based on all sample members in each group and include zero earnings for individuals who were not employed.

11 The variation across the four PHAs in estimated impacts on earnings is statistically significant at the 10-percent level.

ES-8

Exhibit ES.1. Early Impacts on Employment and Earnings: Heads of Households New Rent

Existing Difference

Outcomes

Rules Rent Rules

(Impact)

P-Value

All PHAs

Average quarterly employment, Year 1a

(%)

56.0

54.4

1.6 **

0.046

Total earnings ($)

Year 1 (quarters 3-6)

10,185

9,922

263

0.182

Quarter 7

2,944

2,865

79

0.246

Quarter 8

2,949

2,833

116

0.101

Sample size (total = 6,665)

3,312

3,353

All PHAs except Washington, D.C.

Average quarterly employment, Year 1a

(%)

61.6

59.7

1.9 **

0.038

Total earnings ($)

Year 1 (quarters 3-6)

10,126

9,660

466 **

0.035

Quarter 7

2,767

2,684

83

0.274

Quarter 8

2,822

2,660

162 **

0.041

Sample size (total = 4,756)

2,368

2,388

PHA = public housing agency.

aAverage quarterly employment is calculated as total number of quarters with employment divided by total number quarters of followup, expressed as a percentage. Year 1 is defined as the period that begins after a family's new TTP takes effect. For NDNH data, which are only available on a quarterly basis, that is the third quarter after families were randomly assigned.

* = 10 percent. ** = 5 percent. *** = 1 percent.

Notes: Estimates were regression adjusted using ordinary least squares, controlling for pre-random assignment characteristics of sample members. Rounding may cause slight discrepancies in calculating sums and differences. A two-tailed t-test was applied to differences between research groups. The p-value indicates the likelihood that the difference between the new rent rules group and the existing rent rules group arose by chance. Sample sizes for specific outcomes may vary because of missing values.

Confirmatory outcomes were tested for multiple hypothesis testing using the Benjamini-Hochberg procedure. The adjusted p-value = .182 for the impact on total Year 1 earnings for all four PHAs combined. The adjusted p-value = .035 for the impact on total Year 1 earnings for all PHAs combined excluding Washington, D.C.

Source: MDRC calculations, using quarterly wage data from the National Directory of New Hires

ES-9

Exhibit ES.2. Early Impacts on Employment and Earnings, by Public Housing Agency (PHA), Heads of Households

New

Existing Difference

Outcomes

Rent Rules Rent Rules

(Impact)

P-Value

Lexington

Average quarterly employment, Year 1a (%)

65.7

64.0

1.7

0.395

Total earnings ($)

Year 1 (quarters 3-6)

10,387

9,921

466

0.315

Quarter 7

2,881

2,583

298 *

0.065

Quarter 8

2,869

2,517

352 **

0.028

Sample size (total = 979)

486

493

Louisville

Average quarterly employment, Year 1a (%)

60.7

59.8

0.9

0.544

Total earnings ($)

Year 1 (quarters 3-6)

10,113

10,083

29

0.936

Quarter 7

2,775

3,027

-252 **

0.047

Quarter 8

2,822

3,003

-181

0.181

Sample size (total = 1,908)

947

961

San Antonio

Average quarterly employment, Year 1a (%)

60.5

57.3

3.2 **

0.037

Total earnings ($)

Year 1 (quarters 3-6)

10,003

9,086

916 ***

0.010

Quarter 7

2,704

2,379

324 ***

0.007

Quarter 8

2,786

2,395

392 ***

0.002

Sample size (total = 1,869)

935

934

Washington, D.C.

Average quarterly employment, Year 1a (%)

41.7

41.6

0.2

0.899

Total earnings ($)

Year 1 (quarters 3-6)

10,266

10,634

-368

0.379

Quarter 7

3,387

3,315

72

0.622

Quarter 8

3,264

3,264

0

0.999

Sample size (total = 1,909)

944

965

(continued)

ES-10

Exhibit ES.2 (continued)

PHA = public housing agency.

aAverage quarterly employment is calculated as total number of quarters with employment divided by total number quarters of followup, expressed as a percentage. Year 1 is defined as the period that begins after a family's new TTP takes effect. For NDNH data, which are only available on a quarterly basis, that is the third quarter after families were randomly assigned.

* = 10 percent. ** = 5 percent. *** = 1 percent.

Notes: Estimates were regression adjusted using ordinary least squares, controlling for pre-random assignment characteristics of sample members. Rounding may cause slight discrepancies in calculating sums and differences. A two-tailed t-test was applied to differences between research groups. The p-value indicates the likelihood that the difference between the new rent rules group and the existing rent rules group arose by chance. Sample sizes for specific outcomes may vary because of missing values. The variation across the four PHAs in estimated impacts on total earnings in Year 1 is statistically significant at the 10 percent level based on an H-statistic test.

Source: MDRC calculations, using quarterly wage data from the National Directory of New Hires

The results for Louisvil e and Washington, D.C., were less positive. In fact, in Louisvil e, the effects on earnings turned somewhat negative at the end of the first year. This was not because those in the new rules group did not experience growth in earnings over time.

Indeed, the earnings trends are positive for both research groups; the trends are just less positive for the new rules group, resulting in a statistically significant negative impact in Quarter 7; however, the negative effect attenuated and was no longer statistically significant by Quarter 8.

Longer-term fol owup data wil show whether this negative effect is part of a longer-term pattern or an aberration. The new rent rules had lit le effect on average quarterly employment rates among household heads in Louisvil e during the followup period.

In Washington, D.C., few statistically significant differences in employment and earnings outcomes are evident.

• Evidence from Lexington suggests that the new rent policy’s triennial recertification feature alone can have a positive early effect on

household heads’ earnings.

Because Lexington’s preexisting $150 minimum TTP, which might have created a stronger financial incentive to work, applied to both research groups, it cannot help explain the new policy’s growing positive effects on earnings in that PHA. Rather, the new rent policy’s other main (and more important) work incentive feature, the extended recertification, likely accounts for most, if not all, of Lexington’s early impacts on earnings. This finding provides at least some evidence to suggest that, by itself, substituting triennial for annual recertifications holds some potential to improve tenants’ earnings.

• In Washington, D.C., the new minimum TTP appears not to have affected household heads’ employment or earnings.

ES-11

The adoption of the triennial recertification policy is not expected to affect tenants’

earnings in Year 1 in Washington, D.C., because, as already mentioned, the control group there is subject to a biennial recertification policy. Thus, neither research group faced the normal percentage-of-income work disincentive during the early followup period.

More important in the short term is Washington, D.C.’s new $75 minimum TTP (compared with no minimum TTP for the existing rules group). The evaluation’s finding of no positive short-term impacts on earnings for tenants in this PHA suggests that a $75 minimum TTP, in the absence of a distinctively more favorable recertification schedule, and with hardship remedies available may not necessarily promote greater work effort.

• By the end of Quarter 8, no consistent or definite pattern had emerged with regard to the new rent policy’s short-term impacts on the employment and earnings of any subgroup of household heads.

Different types of voucher holders might respond differently to the new rent policy’s financial incentives to increase earnings because of differences in capacities, skil s, or personal or family circumstances that may make it easier or harder to take advantage of the rent incentives. Some tenants might also be inclined to increase their work effort and earnings even in the absence of those incentives. This report examines initial responses to the new rent policy, primarily for subgroups of household heads, defined by their employment status in the quarter before random assignment and by the age of the youngest child in the household at the time of random assignment.

The results show no clear evidence that the new rent policy has had differential effects on tenants’ short-term labor market outcomes according to their initial employment status or age of the youngest child, although some early pat erns are noteworthy. For example, in the two PHAs (Lexington and San Antonio) that show some full-sample positive impacts on labor market outcomes, statistically significant earnings gains emerge more consistently among household heads already employed in the quarter before random assignment than among those not already employed. In addition, with all PHAs combined, statistically significant earnings gains appear more consistently among household heads whose youngest child in the household was a teenager at the time of random assignment. However, the dif erences in impacts across the relevant subgroup categories were not always themselves statistically significant, indicating considerable uncertainty in the patterns. The longer-term findings wil show whether sharper statistically significant patterns emerge over time.

Early Impacts on Outcomes Related to Housing Subsidies

The new rent policy substantially changed the rules for measuring family income used in determining TTPs, the rules for calculating the share of income that families must contribute toward rent and utilities, and the requirements for reporting income changes and adjusting TTPs over time. Those changes have had some short-term consequences for families’ housing subsidies and PHAs’ experiences in administering the voucher program.

ES-12

• The new rent rules modestly increased the likelihood that families would stil be using their vouchers by the end of Year 1.

Exhibit ES.3 shows that, with all four PHAs combined, 88.2 percent of the existing rules group were stil in the voucher program and “leased up” (that is, were using their rent subsidies) at the end of Year 1. In contrast, 92.4 percent of the new rent rules group remained in the voucher program and leased up—a statistically significant increase of 4.3 percentage points above the existing rules group rate.

This effect varied widely across the four PHAs. For example, lit le effect was evident in Washington, D.C., possibly because of the biennial recertification for the existing rules group in that PHA, which meant families in neither research group could become ineligible for the voucher program by increasing their earnings within this early period, nor would income growth reduce their housing subsidies before Year 3. Among the other PHAs, the new rent rules increased the first-year voucher retention and lease-up rate (above the existing rules group mean) by a low of 3.6 percentage points in Lexington to a high of 8.0 percentage points in San Antonio. When the results for the three PHAs other than Washington, D.C. are combined, the averaged pooled impact is higher at 5.6 percentage points (as seen in exhibit ES.3). Formal exits from the voucher program for the new rules group were lower than for the existing rules group by 3.1 to 3.5 percentage points across these three PHAs (not shown).

• On average, families who were subject to the new rent policy paid somewhat less toward their housing costs, compared with the control group, while in the voucher program.

Over the course of the first year, the new rent rules group paid an average monthly TTP

of $289 while in the voucher program, or $14 less than families in the existing rules group paid while they were stil receiving vouchers (see exhibit ES.3). 12 At the end of Year 1, voucher holders in the new rent rules group were less likely than those in the existing rules group to be paying a very low TTP ($0 to $50), owing to the minimum TTP policy. They were also somewhat less likely to be paying a very high TTP (over $700).

The average monthly family share (which includes payments by tenants above their obligated TTP contribution) was also lower for the new rules group than for the existing rules group ($335 versus $351) while families were stil in the voucher program. A generally similar pattern is evident in each of the four PHAs.

12Exhibit ES.3 does not present impact estimates on these measures because families in the two research groups exited the voucher program at different rates during Year 1. Consequently, the types of families receiving vouchers in any given month, and for whom TTPs could be calculated in each month, may have differed, potentially biasing estimates of the policy’s impacts on average TTPs.

ES-13

Exhibit ES.3. Early Impacts on Families’ Subsidy Receipt and Housing Costs New

Existing Difference

Outcome

Rent Rules Rent Rules

(Impact)

P-Value

All PHAs

Currently Enrolled in HCV Program and

Leased Up at the End of Year 1 (%)

92.4

88.2

4.3 ***

0.000

Average Monthly TTP in Months

Received HCVa ($)

289

303

--

--

Total Housing Subsidy in Year 1b ($)

9,977

9,719

258 ***

0.008

Sample Size (total = 6,665)

3,312

3,353

All PHAs except Washington, D.C.

Currently Enrolled in HCV Program and

Leased Up at the End of Year 1 (%)

90.4

84.8

5.6 ***

0.000

Average Monthly TTP in Months

Received HCVa ($)

255

273

--

--

Total Housing Subsidy in Year 1b ($)

7,505

7,188

316 ***

0.000

Sample Size (total = 4,756)

2,368

2,388

HCV = Housing Choice Voucher. PHA = public housing agency. TTP = total tenant payment.

aTotal tenant payment is the amount a family must contribute to its rent and utilities, regardless of the unit selected. Under the new rent rules, TTP is 28 percent of prior-year gross income, and under existing rent rules TTP is 30 percent of adjusted income.

bHousing subsidy (housing assistance payment) is the full subsidy amount paid by the housing agency, and includes any utility allowance payments made to the tenant in addition to rent paid to the owner by the housing agency.

* = 10 percent. ** = 5 percent. *** = 1 percent.

Notes: Estimates were regression adjusted using ordinary least squares, controlling for pre-random assignment characteristics of sample members. Rounding may cause slight discrepancies in calculating sums and differences. A two-tailed t-test was applied to differences between research groups. The p-value indicates the likelihood that the difference between the new rent rules group and the existing rent rules group arose by chance. Italic type indicates comparisons that are nonexperimental. Statistical tests were not performed. Confirmatory outcomes were tested for multiple hypothesis testing using the Benjamini-Hochberg procedure. The adjusted p-value = .016 for the impact on the total Year 1 housing subsidy for all four PHAs combined. The adjusted p-value = .000 for the impact on the total Year 1 housing subsidy for all PHAs combined excluding Washington, D.C.

Source: MDRC calculations, using PHA data

ES-14

Most families in the new rent rules group paid more than the minimum TTP during Year 1. For al four PHAs combined, 77.6 percent had paid more than the minimum TTP

sometime during Year 1, whereas 29.3 percent had paid exactly the minimum TTP. Only 5.7

percent of families in the new rent rules group had ever paid less than the minimum TTP.

• Compared with the control group, families in the new rent rules group received more in rental subsidies during Year 1, an expected short-term result of the policy.

The lower average TTP for the new rules group combined with a longer duration in the voucher program means that families in that group were receiving a somewhat higher housing subsidy (housing assistance payment, or HAP) than they would have received in the absence of the new policy (represented by the control group’s subsidy amount). As exhibit ES.3 shows, on average for all four PHAs combined, the new rules group received $9,977 in housing subsidies during the first followup year, which is $258 (or 2.7 percent) more than the control group average ($9,719). The difference is somewhat higher—$316, or 4.4 percent—when the findings from all PHAs except Washington, D.C., are pooled (both pooled estimates of the effects on housing subsidy amounts remain statistically significant after adjustment for multiple outcomes). A generally similar pattern is evident when the results are examined by PHA, as exhibit ES.4 shows, and the variation in PHAs’ impacts on subsidies is not statistically significant.

This general pattern of results—somewhat longer tenure on the voucher program, lower TTPs, and more in subsidies relative to the control group—is to be expected during the 3-year period until the next recertification under the new rent rules. Families in the new rent rules group, unlike those in the control group (except in the Washington, D.C. control group), were not required to report any increases in their earnings to the PHAs until their triennial recertification. Consequently, those in the new rules group whose earnings grew over that period did not have their TTPs raised and subsidies reduced, and they could not earn their way off the voucher program during that time. This was intended by the policy design so that families would experience the benefits of their increased work effort during the 3 years between recertifications. Of course, for the PHAs, this means a short-term increase in expenditures on housing assistance. An important open question is whether forgone subsidy reductions wil be recouped after the triennial recertifications are completed.

• A smal portion of families used the new rent policy’s hardship remedies.

The new policy offers potential relief to families whose TTPs exceed 40 percent of their current/anticipated gross incomes. Such families are considered to have excessive rent burdens and are generally eligible to request a hardship remedy. These remedies, which are renewable, include setting the TTP at the minimum level, or at 28 percent of current income, for up to 6

months at a time. Families in Lexington are eligible for a hardship remedy only if they are paying TTPs that exceed the PHA’s $150 minimum and stil meet the 40-percent threshold. In other words, Lexington’s hardship policy allows families who meet the hardship criteria to ES-15

reduce their TTP to the $150 minimum but not below (except in cases in which a household becomes defined as disabled according to HUD).

ES-16

Exhibit ES.4. Early Impacts on Families’ Subsidy Receipt and Housing Costs, by Public Housing Agency (PHA)

New

Existing Difference

Outcome Rent Rules Rent Rules

(Impact) P-Value

Lexington

Currently Enrolled in HCV Program and Leased Up at

the End of Year 1 (%)

90.7

87.1

3.6 *

0.078

Average Monthly TTP in Months Received HCVa ($)

276

308

- -

- -

Total Housing Subsidy in Year 1b ($)

6,777

6,418

359 **

0.029

Sample Size (total = 979)

486

493

Louisville

Currently Enrolled in HCV Program and Leased Up at

the End of Year 1 (%)

93.7

89.7

4.1 ***

0.001

Average Monthly TTP in Months Received HCVa ($)

228

239

- -

- -

Total Housing Subsidy in Year 1b ($)

7,898

7,659

239 *

0.066

Sample Size (total = 1,908)

947

961

San Antonio

Currently Enrolled in HCV Program and Leased Up at

the End of Year 1 (%)

86.8

78.8

8.0 ***

0.000

Average Monthly TTP in Months Received HCVa ($)

271

291

- -

- -

Total Housing Subsidy in Year 1b ($)

7,507

7,088

419 ***

0.002

Sample Size (total = 1,869)

935

934

Washington, DC

Currently Enrolled in HCV Program and Leased Up at

the End of Year 1 (%)

97.5

96.5

1.0

0.206

Average Monthly TTP in Months Received HCVa ($)

371

378

- -

- -

Total Housing Subsidy in Year 1b ($)

16,211

15,953

258

0.319

Sample Size (total = 1,909)

944

965

(continued)

ES-17

Exhibit ES.4 (continued)

HCV = Housing Choice Voucher. PHA = public housing agency. TTP = total tenant payment.

aTotal Tenant Payment is the amount a family must contribute to its rent and utilities, regardless of the unit selected. Under the new rent rules TTP is 28 percent of prior-year gross income, and under existing rent rules TTP is 30 percent of adjusted income.

bHousing subsidy (Housing Assistance Payment) is the full subsidy amount paid by the housing agency and includes any utility allowance payments made to the tenant in addition to rent paid to the owner by the housing agency.

* = 10 percent. ** = 5 percent. *** = 1 percent.

Notes: Estimates were regression-adjusted using ordinary least squares, controlling for pre-random assignment characteristics of sample members. Rounding may cause slight discrepancies in calculating sums and differences. A two-tailed t-test was applied to differences between research groups. The p-value indicates the likelihood that the difference between the New Rent Rules group and the Existing Rent Rules group arose by chance. Italic type indicates comparisons that are nonexperimental. Statistical tests were not performed. The variation across the four PHAs in estimated impacts on average annual housing subsidy in Year 1 is not statistically significant based on an H-statistic test.

Source: MDRC calculations, using PHA data

Hardship remedies can be issued to qualifying families at any time during the 3-year period, but families must request them. Across all PHAs, about 8.4 percent of families requested and received a hardship remedy in Year 1. The pooled rate is higher than the rate at the time of initial recertification when only 0.5 percent of families across the four PHAs received a hardship remedy (Riccio, Deitch, and Verma, 2017). The small increase over time may reflect the possibility that a growing proportion of families experienced a post-recertification loss of income during Year 1. It could also reflect a growing awareness among already-qualifying families of the availability of this safeguard and wil ingness to request it. 13

• The new rent policy substantial y reduced the likelihood of PHA actions, with or on behalf of families, related to changes in families’

circumstances.

This report examines whether the new policy has already begun to affect the number of actions to address changes in families’ circumstances, which translates into administrative burdens and costs for PHAs over time. Those actions could include annual recertifications, interim recertifications needed because of changes in income, and actions related to household composition changes or changes in contract rents, moves, or other circumstances.

For all PHAs combined, almost three-fourths (73.6 percent) of the control group in Year 1 had a circumstance that required action on the part of PHA staff. This rate ranged from about 45 percent in Washington, D.C., (on the low end due largely to the biennial recertification 13 The PHAs issued several notices to remind families of this provision of the new rent policy, although it is possible that some qualifying families were still not fully aware of it, did not believe they qualified, or chose not to apply to avoid further interactions with the PHA. Future reports will explore this issue further.

ES-18

schedule) to more than 91 percent in Lexington. Several factors account for why actions for the control group, particularly annual recertifications, were less than 100 percent in each of the four PHAs. For example, although existing rules require all families (except in Washington, D.C.) to complete an annual recertification, some families exited the voucher program or transferred (“ported out”) to another housing agency before any further actions were required. In addition, families moving to a new unit must undergo a full income review, and depending on when the move occurs, that review can be completed in lieu of an annual recertification. For some other families, annual recertifications were delayed for various reasons, including delays in submit ing required documentation to the PHA.

Among families in the new rules group, about one-half (49.6 percent) had a transaction with PHA staff—a reduction of 23.9 percentage points compared with the control group level.

When Washington, D.C., is excluded, the reduction increases to 28.9 percentage points. The main factors contributing to that reduction were the elimination of annual recertifications and the reduction in interim recertifications for increases or decreases in family income. Those actions were generally the most time-consuming actions for staff because they required recalculating TTPs and subsidies.

The patterns varied substantially across the four PHAs, driven in part by the different circumstances affecting the control group in each location. Lexington stands out with the largest effects on most indicators. In that PHA (where all control group families remained subject to the traditional HUD interim recertification requirements, including the requirement that families report all income changes between annual recertifications), the proportion of families in the new rules group for whom a staff action was required during Year 1 was 44.7 percentage points lower than the 91.2-percent rate for the control group. That reduction included a 75.8-percentage-point reduction in annual recertifications, a 16.0-percentage-point reduction in interim recertifications for declining income, and a 20.8-percentage-point reduction in interim recertifications for increased income.

Conclusion

The early impact findings discussed in this report offer an initial but incomplete assessment of the new rent policy designed as part of HUD’s Rent Reform Demonstration. Nonetheless, some important patterns have begun to emerge on a variety of critical outcomes. Overall, for the pooled sample with all four PHAs combined, the policy has not produced statistically significant gains in earnings within the first 18 months of fol owup; however, the picture is more mixed when the results are examined site by site. For example, the new policy has led to some earnings gains in two of the PHAs (Lexington and San Antonio) for heads of household but not in the other two (Louisvil e and Washington, D.C.). The new policy has also begun to reduce the most time-consuming transactions (annual recertifications and interim adjustments in TTPs and subsidies, required under traditional HUD rules as families’ incomes changed). Those patterns wil be important to reexamine as the longer-term data become available—and especially once families in the new rent rules group have completed their triennial recertifications. At that point, those who have benefited from sustained higher earnings in the 12

ES-19

months leading up to that recertification point and who are stil receiving vouchers wil begin paying higher TTPs and receiving smaller housing subsidies (their new TTPs wil then be capped for another 3 years).

Future reports wil examine the new rent policy’s effects on these same outcomes over a longer followup period and on a wider array of outcomes. Those effects include impacts on families’ receipt of other government benefits (including Temporary Assistance for Needy Families, Supplemental Nutrition Assistance Program, and homelessness assistance) and on survey-based measures related to families’ housing stability, material hardships, and additional quality-of-life indicators. These future reports (scheduled to be completed in 2021 and 2023) wil also cover the PHAs’ continuing experiences in implementing the new policy over the next several years. They wil include an assessment of whether or not the new policy has helped reduce the administrative burden and costs for PHAs compared with the traditional rent policy.

ES-20

Chapter 1

Introduction

The federal Housing Choice Voucher program (formerly known as Section 8) provides critical rental subsidies to low-income families living in privately owned housing units. 14 Most public housing agencies (PHAs), which administer the voucher program, follow a common set of federal rules in determining how much tenants must contribute of their own income toward their rent and utilities and how much of a housing subsidy they will receive. The traditional way in which those subsidies have been calculated has been widely criticized for creating a disincentive to work while imposing a substantial and costly burden on PHAs to administer the subsidy system. 15 That system requires families to report changes in income at least annually and for the PHAs to adjust the subsidies up or down as the families’ income falls or rises.

Although this system provides a strong safety net for families by providing more rental assistance for families whose needs are greater because of lower or falling incomes, it also creates an implicit marginal “tax” on increased earnings (approximately 30 percent) because families pay more toward their rent and utilities as their incomes rise, possibly discouraging efforts to increase earnings. This implicit tax is on top of possible reductions in other means-tested benefits families might be receiving (such as welfare or food stamps), making their combined marginal tax on increased earnings more than 30 percent.

As one step toward trying to find a better way, the U.S. Department of Housing and Urban Development (HUD) launched the Rent Reform Demonstration, an initiative to design and careful y evaluate an alternative rent-subsidy policy for recipients of tenant-based housing choice vouchers. In setting guidelines for the demonstration, HUD sought a policy that would simplify the rent system to reduce PHAs’ administrative burden and costs, create a stronger financial incentive for families to increase their earned income, continue to provide a safety net to families who cannot readily increase their earnings, and reduce or at least minimize increases in PHAs’ average housing subsidy expenditures per family over time. HUD selected MDRC

and its partners to lead the effort to design and evaluate the new policy, working closely with HUD and the following four PHAs:16

14Tenant-based housing choice vouchers are portable, meaning that families can use the vouchers with private landlords of their own choosing if the housing unit meets the PHA’s quality standards, and they can take the vouchers with them to a new landlord if they choose to move. These vouchers differ from Project-Based Section 8 assistance, in which a subsidy is attached to a particular housing unit through a contract between the PHA and a private landlord.

15 These and other criticisms are described in Abt Associates, Inc., the Urban Institute, and Applied Real Estate Analysis, Inc. (2010); Government Accountability Office (2012); and Public Housing Authorities Directors Association (2005). See also Riccio, Deitch, and Verma (2017) for a summary of these perspectives and relevant prior evidence on housing assistances and labor force participation.

16The study team includes the Urban Institute, the Bronner Group, Quadel Consulting & Training, and professors Ingrid Gould Ellen (New York University) and John Goering (City University of New York) and research consultant Barbara Fink.

1

• Lexington-Fayet e Urban County Housing Authority in Lexington, KY

(generally referred to as the Lexington Housing Authority)

• Louisvil e Metropolitan Housing Authority in Louisville, KY

• San Antonio Housing Authority in San Antonio, TX

• District of Columbia Housing Authority in Washington, D.C.

The four PHAs participating in the Rent Reform Demonstration are implementing the new rent policy alongside the traditional policy to help determine its effects. They are a subset of 39 PHAs that, at the time the project was launched, were part of HUD’s Moving to Work demonstration. Congress has granted Moving to Work agencies more flexibility than other PHAs to change housing policies, provided they notify the public and receive approval from HUD and from their boards of directors. They are permitted to change certain policies that would otherwise require changes in legislation or regulations, and this administrative flexibility extends to rent rules. 17

The centerpiece of the evaluation is a two-group randomized control ed trial to test the effects of the alternative rent policy on voucher holders’ labor market outcomes, use of housing subsidies and other government programs, material hardship, well-being, PHA costs and administrative burden, and other outcomes. The random assignment process was incorporated into the regular recertification process through which PHA staff members normally review whether families continue to meet the voucher program’s income and other requirements, calculate how much each family is expected to contribute to its rent and utilities, and determine how much of a housing subsidy it wil receive. For the Rent Reform Demonstration, eligible voucher holders who were scheduled for recertification between February 2015 and November 2015 were enrolled in the study. 18 They were randomly assigned either to a program group that was subject to the new rent policy or to a control group that remained subject to the existing rent rules (exhibits in this report refer to the study groups as the new rent rules group and the existing rent rules group, respectively). The recomputed cost for rent and utilities obligations and housing subsidies for families in each research group took effect between June 2015 and March 2016 (the exact dates varied among families and the four PHAs, as shown in Chapter 2).

MDRC prepared an initial or “baseline” report on the demonstration, published by HUD in 2017, that describes the origins of the Rent Reform Demonstration, the features of the new policy, the rationale behind each of its main elements, and the manner in which it is being evaluated (Riccio, Deitch, and Verma, 2017). That report also describes in more detail the process for identifying and enrol ing families into the study, the background characteristics of 17 According to the Moving to Work Agreement, Moving to Work agencies have the authority to adopt and implement any reasonable policies to calculate tenants’ contributions toward their rents that differ from the program requirements as mandated in the United States Housing Act of 1937and its current implementing regulations. The four PHAs in the Rent Reform Demonstration were still largely following HUD’s traditional rent policy at the start of the demonstration, with some exceptions that are discussed later in this chapter.

18 The demonstration is slated to conclude in 2021.

2

those families, the amounts the families initially began paying for their rent and utilities under the new rent rules compared with the existing rules at the beginning of the study, and the housing subsidies they received initially.

The current report provides the first look at the effect, or “impact,” of this policy on families’ labor market and housing-related outcomes. It covers a fol owup period for each eligible family of approximately 12 to 18 months after the new policy took effect (depending on the outcome measure). This timeframe is too short for drawing firm conclusions about the new policy’s effects, in part because, as described below, a critical element of the new policy is a cap on families’ expected contributions to their rent and utilities over a 3-year period (as opposed to the 1-year period imposed by HUD’s traditional rent rules). A ful assessment must therefore wait until after that 3-year period has passed. Nonetheless, the early results show whether the new policy begins to change labor market and housing-related outcomes even in the short term.

As wil be seen, the new policy increased tenants’ earnings in two of the four PHAs within the first 18 months of fol owup and, generally (although to varying degrees) in each agency, reduced families’ housing costs, delayed their exits from the voucher program, increased their annual housing subsidies, and reduced certain types of PHA transactions with families during the first year of followup.

Future reports wil examine the PHAs’ ongoing experiences in implementing the new policy, including PHAs’ administrative burden and costs relative to operating the existing rent policy. Those reports wil also present findings on the new policy’s longer-term effects—

covering 3 or more years of followup—on families’ employment, earnings, contributions toward their rent and utilities, housing subsidies, receipt of other government benefits (including Temporary Assistance for Needy Families and Supplemental Nutrition Assistance Program benefits), and a variety of other outcomes, including housing stability, material hardships, and additional quality-of-life indicators. Some of those impact measures wil be based on data from administrative records, and others wil be based on a survey administered to the household heads in each research group.

HUD’s Traditional Rent Policy

National y, HUD funds 2,243 PHAs to provide approximately 2.2 million low-income households across the country with tenant-based housing choice vouchers. Currently, most housing choice voucher families are expected to contribute 30 percent of pretax income (after certain adjustments) toward their housing costs. 19 A family’s rent contribution and utility payments are referred to as its “total tenant payment” (TTP). The rules for calculating a family’s TTP allow several deductions from gross income (including a deduction for some childcare costs for working parents), yielding an “adjusted income” estimate. The calculation also looks forward in time, basing the adjusted income estimate on the amount of income a family currently receives and anticipates receiving in a typical month during the coming year 19 Throughout this report, HUD’s “current” or “traditional” rent policy for voucher holders refers to the national rent policy in effect for non-Moving to Work PHAs before the passage of the Housing Opportunity Through Modernization Act of 2016.

3

(which this report refers to as “current/anticipated” income). The PHA provides a subsidy for the difference between the family’s rental payment and the maximum allowable rent, called a

“payment standard,” which takes account of local area fair market rents. Al PHAs are permit ed to establish a minimum TTP, commonly referred to as a “minimum rent,” of up to $50 per month, although not al have done so. 20 (MTW agencies have more flexibility to establish higher minimum TTPs and make other adjustments in the rent policy.) This existing “percentage of adjusted income” approach builds a strong safety-net feature into the rent subsidy system: If a family’s income falls, the family pays less toward its housing costs; however, this approach also implicitly “taxes” tenants for increasing their earnings (which some experts contend discourages work) and requires PHAs to make continuous and administratively burdensome readjustments in TTPs and housing subsidies as a family’s income changes. The complex rules governing the calculation of “adjusted income,”

rent, and utility allowances are considered by critics of the existing policy to be administratively burdensome and prone to errors that can lead to improper payments. The new rent policy attempts to address those problems.

Overview of the New Rent Policy

The new rent policy applies only to working-age, nondisabled voucher recipients whose vouchers were administered under the Moving to Work demonstration. 21 The policy includes the core features described below, which are also summarized in exhibit 1.1. 22

Changes in rules for recertifying families’ continued eligibility for the voucher program and recomputing their TTPs:

• Replacing the annual recertification schedule with a triennial schedule so that a family is required to review its income with the PHA only every 3 years.

This change means that if a family increases its earnings during that period, it need not report the increase to the PHA, and its TTP wil not be raised, until the end of the 3-year period.

20For a full explanation of the HUD’s existing rent rules, see HUD (2001).

21Non-Moving to Work Vouchers (that is, Veterans Assisted Special Housing, Moderate Rehabilitation, and Shelter Plus Care), Enhanced Vouchers, and Section 8 Project-Based Vouchers were excluded from the study. In addition, the study did not include elderly households, disabled households, and households headed by people older than 56 years of age (who would become seniors during the long-term study). Households participating in Family Self-Sufficiency and homeownership programs before sample enrollment began were also excluded from the study, as were families who held vouchers but were receiving no housing subsidy.

22See Riccio, Deitch, and Verma (2017) for further details.

4

Exhibit 1.1. Comparison of Traditional and New Rent Policies

Component

Traditional HUD Policy

New Rent Policy

Total Tenant

30 percent of adjusted monthly

28 percent of gross monthly retrospective

Payment (TTP)

income (that is, total countable

income (that is, gross monthly income over

current/anticipated income, minus

the previous 12 months), with no deductions

deductions) or 10 percent of gross

or allowances.

Countable income estimate for set ing a

income, whichever is higher.

family’s TTP and housing subsidy is based on

12-month retrospective income.

Minimum TTP

Up to $50 per month, at public

$50 to $150 per month, depending on the

housing agency (PHA) discretion.

PHA.

Al families pay a minimum amount of rent

directly to their landlords, to mirror the

landlord-tenant relationship in the

unsubsidized rental market.

Assets

Family income from assets is

Family income from assets is ignored when

counted in determining a family’s

total asset value is less than $25,000, and

TTP.

families do not need to document those assets.

Recertification

Annual recertifications.

Triennial recertifications.

Period

Interim

At an agency’s discretion, families

Earnings gains do not increase TTP for 3

Recertifications

report any income increases when

years (that is, until the next triennial

When Income

they occur, before the next

recertification).

Changes

scheduled recertification. Families

Interim recertifications are limited to a

may request interim recertifications

maximum of one per year and only when a

whenever their incomes fall by any

family’s average gross income over the most

amount.

recent 12 months drops by more than 10

percent from the retrospective estimate that

was used to establish the TTP currently in

effect.

Utilities

Where the contract rent does not

A simplified utilities policy that is tailored to a

include utilities, a utility allowance

standard base rate for utility costs and that

is provided based on a detailed

varies according to the voucher amount, with

schedule that takes into

additional payments available to families

consideration voucher size (the

paying higher costs related to the type of

number of bedrooms covered by a

heating (for example, electric or oil heat) and

family’s voucher) and various other water and sewer charges.

aspects of the type of housing unit.

(continued)

5

Exhibit 1.1 (continued)

Component

Traditional HUD Policy

New Rent Policy

Hardship Policy

If the PHA has a minimum TTP,

Families qualify for consideration of a

it must suspend that minimum TTP

hardship-based remedy if—

for families who are unable to pay

it because of specified financial

• The family’s monthly TTP exceeds 40

hardships. Short-term hardships

percent of its current or anticipated monthly

(lasting 90 days or less) require the

gross income.

suspended minimum to be

• The hardship cannot be remedied by the one

reinstated after the hardship period

interim recertification permit ed each year.

ends and to be repaid according to

a reasonable payment plan.

• The family faces eviction for not paying rent

or utilities.

• The family meets other criteria determined

by the PHA.

Hardship remedy options include the

following standardized list:

• Allowing an additional interim

recertification beyond the normal one per

year.

• Setting the family’s TTP at the minimum

level for up to 180 days (this remedy can be

renewed at the end of that period if the

hardship persists).

• Setting the family’s TTP at 28 percent of its

current gross income (which may be less

than the minimum TTP) for up to 180 days

(except in Lexington) (this remedy can be

renewed at the end of that period if the

hardship persists).

• Offering a “transfer voucher” to support a

move to a more affordable unit.

Grace Period

Not applicable. TTP is always

At the triennial recertification, if a family’s

based on current income.

current gross income is more than 10

percent lower than its average gross

retrospective income over the past 12

months, the family will have its TTP

calculated at that time based on current

income rather than retrospective income,

and this TTP wil remain in effect for 6

months. During this grace period, families

can stil qualify for a hardship-based

remedy.

Notes: The Traditional HUD Policy column shows the national policy in existence for the non-Moving to Work tenant-based housing choice voucher population before the enactment of the Housing Opportunity Through Modernization Act of 2016. With a few exceptions, the PHAs participating in the Rent Reform Demonstration have continued to implement these policies. For each of the four demonstration PHAs, details on its existing policy and how it varies from the traditional HUD policy are available in exhibit A.1.

6

Changes in the formula for calculating a family’s TTP and subsidy:

• Eliminating all deductions from countable income so that gross income, rather than adjusted income, is the basis for calculating a family’s TTP (as a step toward simplifying that calculation). 23

• Calculating TTP at 28 percent of gross income rather the normal 30 percent of adjusted income (to help offset the elimination of income deductions).

• Using a family’s gross income over the previous 12 months (referred to as

“retrospective income”) in setting its TTP and housing subsidy rather than the traditional practice of using the adjusted income a family currently has and expects to have in the coming year.

• Ignoring a family’s income from assets when the total value of its assets is less than $25,000 (and not requiring documentation of those assets).

• Simplifying the policy for determining utility al owances to a streamlined standard schedule based primarily on unit size, with some adjustments.

• Establishing a minimum TTP of at least $50 per month and requiring families to pay at least the specified minimum TTP directly to their

landlords. This change means that all tenants have rent-paying relationships with their landlords, as they would in the unsubsidized rental market. 24

Safeguards for families (in addition to interim recertifications):

• A TTP “grace period” at the start of the 3-year period, al owing for a temporary (6-month) TTP reduction when a family’s current/anticipated gross income is more than 10 percent lower than its average monthly

retrospective gross income over the past year.

• Al owing one interim recertification per year if a family’s retrospective income falls by more than 10 percent before the next required triennial review. This change is intended to limit the volume of TTP adjustments the PHA makes (because normally they must be made whenever families report 23Countable income refers to income that, according to HUD guidelines, is taken into account when determining a family’s TTP. It excludes certain types of income, such as earnings of a child under the age of 18, health insurance payments, and nonrecurring income from gifts). The new rent policy uses the same types of income inclusions and exclusions in TTP calculations that apply under HUD’s traditional rent policy. However, nonwage income that is set to expire by the end of the look-back period, such as Temporary Assistance for Needy Families or unemployment insurance benefits, is not counted when calculating base income because a family would not be able to depend on such income going forward.

24Although most voucher holders pay some rent directly to their landlords, in some cases the housing authority pays the entire amount to the landlord. Requiring all families in the new rent rules group to pay at least some amount to their landlords was perceived by some HUD officials as a way of helping to prepare those families for the arrangement they would face if they increased their incomes and received lower housing subsidies or moved and were no longer receiving housing subsidies.

7

income gains or losses of any amount) while stil protecting families when their incomes drop substantially (the new policy does not restrict interim recertifications required for other reasons, such as a change in household composition or a move to a new unit).

• A hardship policy that covers a standard set of conditions (particularly when a family’s TTP exceeds 40 percent of its current income) and includes a standard set of remedies that permit TTP reductions at any time during the 3-year period to protect households from excessive rent burdens.

The PHAs participating in the demonstration helped to develop and support this common framework; however, they also saw a need to adapt the model in some ways in response to local considerations or to accommodate some preexisting policy changes they had already implemented (see exhibit A.1 for a summary of the existing rent policies across the four PHAs, which apply to the control groups in the demonstration). For example, reflecting local considerations, TTP levels vary among the PHAs from $50 to $150 per month. Two of the four PHAs—in Louisvil e and Washington, D.C.—introduced a minimum TTP for the first time ($50 and $75, respectively), and San Antonio increased its existing $50 minimum TTP to $100. Lexington had already introduced a $150 minimum TTP before the demonstration began, and it continued that policy for both the new rules group and the existing rules (control) group.

The process for determining hardship remedies also varies across the PHAs, although the conditions defining a hardship and the remedies themselves do not. Washington, D.C., had already instituted a simplified approach for calculating families’ utilities costs, a version of which each of the other PHAs in the demonstration adopted for the new rules group.

Of all the features of the new rent policy, the 3-year recertification is the main one intended to improve labor market outcomes because it eliminates the implicit “tax” on earnings during the 3-year period. The introduction of a minimum TTP or the increase in an existing one might also increase work effort unless that minimum is waived because some tenants may need to increase their earnings to have enough income to meet the new requirement. Various features of the new policy were also intended to reduce the administrative burden on PHAs or offer families protections from excessive rent burdens.

Administering the New Rent Policy

Although some of the changes introduced by the new rent rules may simplify the process of determining a family’s TTP (for example, by eliminating childcare and other deductions and streamlining the utilities policy), other changes could be burdensome to implement with some families (for example, computing and verifying retrospective income when a family’s income is volatile and not well documented or captured by the administrative records on families’ income that the PHAs can access from other government sources). 25 Although adopting a 3-year 25“Administrative records” comprise data that are collected in the course of administering a program.

These data are available to PHAs through the HUD Enterprise Income Verification system, which 8

recertification period is intended to reduce the overall burden on PHAs and families by reducing the volume of TTP recalculations and the number of contacts families have with the PHA over several years, whether those outcomes are actually achieved depends on the frequency with which hardship remedies and interim recertifications are requested and approved. It remains to be seen, therefore, whether the new rent rules— taken as a whole over several years—achieve the goals of simplification and reduced administrative costs. The evaluation wil attempt to answer this important question in future reports.

The new rent rules also impose extra communication responsibilities on the PHAs in at least two ways. First, if families are to respond to the work incentive built into the new rules, they must be aware that such an incentive exists and understand how it functions. Second, if the safeguards built into the new policy are to have their intended protective effects, families must be aware of those safeguards, understand how they work, and take advantage of them when needed. (The PHAs must also implement them properly). To implement the new rent policy, therefore, PHAs must undertake regular and active communication efforts beyond the initial explanations they offer to families at the time of recertification. To that end, with MDRC’s guidance and HUD’s support, the PHAs are sending additional mailings approximately twice each year to remind families of the new policy’s work incentive and safeguards and to invite them to contact a housing specialist if they believe they may qualify for a TTP reduction.

As the overall managers and evaluators of the demonstration, MDRC and its partners worked closely with the four PHAs to specify the processes required to implement the new rent policy. The MDRC team helped the agencies think through their staffing needs and software modifications, how they would integrate research procedures into recertification meetings, and how staff members would be trained in the procedures for calculating rent and utilities using a new set of rules. The team prepared a manual for each PHA describing those procedures and helped train housing specialists and their supervisors to apply them. In addition, the team observed recertification meetings, monitored implementation practices, and provided refresher training sessions on the use of interim recertifications and hardship remedies. Since that initial launch phase, the team has continued to conduct regularly scheduled check-in meetings with managers at each PHA to discuss any challenges that the PHA is facing in implementing the new rent policy and, in 2018, conducted “refresher” training sessions for staff at each location as they began to conduct triennial recertifications under the new policy. MDRC has had no direct operational role in the administration of the new rent rules, however.

provides information such as earnings reflected in unemployment insurance wage records, unemployment insurance compensation, and Social Security and Supplemental Security Income benefits. One known issue with the Enterprise Income Verification system is that it is not considered complete—or current—

because of reporting lags in some of its data sources. The National Directory of New Hires, for example, has a 6-month reporting lag.

9

Exhibit 1.2. Total Tenant Payment (TTP) at Initial Study Recertification, Al Public Housing Agencies (PHAs)

New

Existing

Outcome

Rent Rules

Rent Rules

Difference

Average total tenant payment (TTP)a ($)

289

310

-20

TTPa (%)

$0

1.9

9.1

-7.2

$1–$50

5.8

11.5

-5.7

$51–$75

9.7

4.1

5.7

$76–$100

9.0

3.3

5.7

$101–$150

13.4

11.1

2.3

$151–$300

25.3

20.9

4.4

$301–$700

27.5

29.5

-2.0

$701 or more

7.4

10.5

-3.1

Sample Size (total = 6,180)

3,111

3,069

a Total Tenant Payment is the amount a family must contribute toward rent and utilities regardless of the unit selected. Under the new rent rules TTP is 28 percent of prior year income and under existing rent rules TTP is 30 percent of adjusted income.

Notes: Rounding may cause slight discrepancies in calculating sums and differences. Statistical tests were not performed.

Source: MDRC calculations, using PHA data

TTPs and Rent Subsidies at the Start of the New Rent Policy

The baseline report shows the TTPs that were calculated for families at the time of their initial recertifications (in 2015–16) when the new rent rules were first applied (Riccio, Deitch, and Verma, 2017). Some results from that analysis are summarized in exhibit 1.2. For families in the new rent rules group, the average initial TTP ($289 per month) calculated at the same time after entering the study was about $20 lower than the average TTP for the existing rent rules group ($310 per month), or about 6.5 percent lower. The lower TTP means that the average monthly housing subsidy for the new rent rules group was correspondingly higher, by about 2.6

percent ($834 compared with $813 for the existing rent rules group, not shown in exhibit 1.2).

The new rent rules also reduced the proportion of families paying very low and very high TTPs at the time of the initial recertification. For all PHAs combined, the new rent rules group had lower proportions of families paying no TTP at all and lower proportions paying more than $700 per month than was the case in the existing rent rules group. Most of the reduction in the proportion of families paying zero TTP came from Louisvil e and Washington, D.C., (not shown in table), where, in contrast to Lexington and San Antonio, the existing rent rules do not include a minimum TTP. Because of the minimum TTP provision in the new rules, families in the new rent rules group who reported having no income at all (sometimes referred 10

to as “zero-income families”) were expected to pay $50 per month in Louisvil e and $75 in Washington, D.C., unless they applied for and received hardship remedies.

The Scope of This Report

This report provides early findings from the evaluation’s impact analysis. Future reports wil provide a more comprehensive assessment, examining the new policy’s effects over a longer period of time and on a wider range of outcomes, and wil include information about the PHAs’

ongoing experiences operating the new rent rules policy, about the administrative burden and costs incurred by the PHAs, and about families’ views of the new policy.

Chapter 2 of this report briefly summarizes the baseline report’s description of the ways in which the sample for the evaluation was enrol ed in the study and some characteristics of those families. It also explains the data col ection and analysis methods that were used to estimate the early impacts presented in subsequent chapters. Chapter 3 presents the findings on employment and earnings outcomes, and Chapter 4 presents the findings on tenants’ housing costs, subsidies, and other outcomes related to their subsidy receipt. Chapter 5 concludes the report by highlighting the next steps in the evaluation.

11

Chapter 2

The Study Sample and Analysis Methods

The Rent Reform demonstration uses a randomized control ed trial, one of the most rigorous methods for determining the effectiveness of an intervention. This chapter discusses the study sample, data sources, and analysis methods that are being used in the experiment. A ful er account of the overall evaluation design and characteristics of the study sample can be found in the demonstration’s baseline report (Riccio, Verma, and Deitch, 2017). 26

Building the Research Sample

The Eligible Sample

Because it was important to test whether the new rent policy would improve tenants’

employment and earnings, families had to be existing voucher holders and meet the fol owing core criteria to be eligible for the Rent Reform Demonstration:27

• A family could not be classified as an elderly household and could not become elderly according to HUD’s definition over the course of the study.

More specifically, the head of household, spouse, and co-head had to be 56

years of age or younger at the time of study enrol ment so that a fol owup period of several years would not extend into the time when many adults begin to retire.

• A family could not be defined as a disabled household (that is, one in which the head, co-head, or spouse is disabled).

The study also excluded several other types of voucher holders. For example, some families were not eligible because they held special vouchers governed by some regulations that did not apply to the vast majority of regular voucher holders. Families who were already participating in HUD’s Family Self-Sufficiency and Homeownership programs were also excluded because the new rent rules would change some of the terms that those families had 26 The baseline report describes the study’s overall plans for collecting and analyzing administrative records related to receipt of Temporary Assistance for Needy Families (TANF), Supplemental Nutrition Assistance Program (SNAP) benefits, and services documented in the Homelessness Management Information System (HMIS), as well as qualitative data. After the baseline report was released, HUD

funded a household survey in which heads of household will be interviewed approximately 42 months after random assignment on a wide range of outcomes that are not available in administrative records.

27The study did not include new voucher holders because it was expected that a substantial number would not successfully “lease up”—that is, find appropriate housing for which they could use the voucher within the time the public housing agencies gave them to do so. Because such families would forfeit their vouchers, they could not be subject to either the new or existing rent rules and, consequently, would not contribute to the goals of the evaluation.

12

agreed to when they enrol ed in those programs. In addition, the demonstration excluded families who were currently receiving childcare deductions so that those families would not be forced to give up deductions they had come to rely on (the deductions are not a feature of the new policy). With a few additional exceptions, the remaining families who were scheduled for recertification during the study’s enrol ment period were selected for the study. 28

Enrolling the Sample

The procedures for enrol ing families into the study for the Rent Reform Demonstration were incorporated into the regular income recertification process—that is, the process that each of the four public housing agencies (PHAs) use to review whether families continue to meet the voucher program’s income and other requirements and to calculate their total tenant payments (TTPs) and housing subsidies. Once the study’s eligibility criteria were set, the PHAs and MDRC identified qualifying families who were being scheduled for upcoming recertifications.

Random assignment procedures were then used to allocate those families either to a new rent rules group, which would be subject to the new policy for the duration of the demonstration, or to an existing rent rules group, which would continue to be subject to the traditional rent rules for voucher holders (and that would serve as the study’s control group). 29 With the exception of Louisvil e, enrol ment in the demonstration was mandatory. Families had their TTPs for rent and utilities and their housing subsidy amounts calculated according to the rules of the rent policy group to which they were assigned and remained subject to all the rent rules applicable to their group for the duration of the demonstration. Although families could not opt out of the rent policy group to which they were assigned (except in Louisvil e), they could refuse to allow their individually identified data to be shared with the researchers; however, only 14 families—or 0.2

percent of those randomly assigned—across the four PHAs chose to do so. Exhibit 2.1 outlines the enrol ment process for the study PHAs. 30

28See exhibit B-4.1 in Riccio, Deitch, and Verma (2017) for a complete list of reasons for exclusion.

29The reason for not asking recertifying voucher holders to choose which rent policy would apply to them was to mimic the ways that the new policy would be likely to operate in practice if it were to be adopted as a new government policy—that is, families would not be able to choose whether the new policy would apply to them. The fact that the new rent policy includes safeguards to minimize the risk of harm while also creating opportunities for substantial benefits for those who are subject to it was among the reasons why MDRC’s Institutional Review Board deemed this random assignment design to be one that meets recognized ethical guidelines for human-subject research. HUD also deemed the random assignment design to be compliant with Moving to Work (MTW) regulations, which give MTW agencies statutory flexibility to implement new initiatives with the proper public notice and PHA board approval.

In Louisville, however, community concerns led to an agreement with the PHA that families who were assigned to the new rent rules group would be allowed to opt out of that policy and have their rent calculated using existing rules. For more on this issue, see Riccio, Deitch, and Verma (2017).

30For a detailed description of the steps in this process, see Riccio, Deitch, and Verma (2017).

13

Exhibit 2.1. Enrollment Flow Chart

Identification of Eligible Households

Eligible households are identified by the public housing agency (PHA).

Random Assignment

Households are assigned to the new rent rules (program) group or the existing rent rules (control) group.

Study Group Notification

Households are notified of their study group assignments and provided dates and times for their in-person interviews.

New Rent Rules Group

Existing Rent Rules Group

In-Person Interview

Rent Reform Policy Orientation

Households view orientation video and are given an

orientation packet.

Recertification

Income is verified and TTP is calculated according to the new or the existing rent rules.a Study Information Sheet

Household meets with a PHA staff member to review the study information sheet.

Baseline Survey

Households complete the baseline survey and are given gift cards.

Households are sent 30-day rent change notification letters. b Rent Effective Date

a

In

Loui

sville, ho

usehol

ds assigned to

t he

new re nt rul

es group r ece ive

d TTP es tim ates for

both the new and the existing rent rules at their recertification meetings and were given 30

days to decide w hether the

y want

ed

to o pt out

of the

ne w rent pol icy.

bFo r house hol ds a ssigne

d to th e new

rent rule s grou p in Loui

svi lle, the 30-day rent change

n otificatio n lette r w

a s se nt a t the end o f the 30- day

opt- out period.

Notes: The whole enrollment process—from identification of a household to the rent effective date—is expected to take between 120 and 180 days.

14

In Louisvil e, the opt-out option required the PHA staff and families to take some additional steps. Families who were assigned to the new rent policy group were asked to mail their income documentation to the PHA before the recertification meeting so that the staff could prepare estimates of the family’s TTP under the new rent rules and under the existing rent rules.

Those two estimates were then presented to families and discussed at the recertification meeting. After the recertification meeting, families were given 30 days to opt out of the new rent rules. Unless a family notified the PHA about its decision to opt out (which had to be communicated in writing, using an official agency form), the new rent rules would apply. After the 30-day opt-out period ended, the PHA finalized the TTP and subsidy and notified each family. By the end of the enrol ment period, 212 eligible families (about 22 percent of the eligible new rent rules group in Louisvil e) chose to opt out of the new policy (but, as explained later in this chapter, nearly all remain part of the study).

The enrol ment process took different amounts of time at the four PHAs, in part because the agencies had “recertification cycles” (the periods during which income recertifications had to be completed) of different lengths. The recertification cycles ranged from 90 days in Lexington to 180 days in Washington, D.C. 31

A total of 7,255 families were randomly assigned for the Rent Reform Demonstration.

As it turned out, however, about 8 percent of families across the two research groups were subsequently found to be ineligible for the study before the initial recertifications were completed (because, for example, they were disabled, moving to another PHA, or in the process of exiting the voucher program). They were excluded from the analysis sample (before any findings were produced), yielding a final sample size of 6,665 families for the four PHAs combined. 32

Characteristics of Enrolled Families

The Rent Reform Demonstration is structured around a two-group, randomized control ed trial.

The power of this research design comes from the fact that, with an adequate sample size, random assignment ensures that the intervention and control groups wil be similar in their distributions of observed and unobserved characteristics when the study begins. Thus, differences that later emerge between the two groups can be attributed with a greater degree of confidence to the intervention rather than to preexisting differences in families’ characteristics.

The effects of the new rent policy are therefore determined by comparing over time the labor-market outcomes and other outcomes of the new rent rules group with the outcomes of the existing rent rules group.

31Two PHAs, in Lexington and Washington, D.C., agreed to recertify some households early (that is, to advance their annual recertifications) to try to meet the study’s sample-size goal within the preferred enrollment timeframe. For San Antonio, the enrollment period was extended by an additional 3 months to meet the study’s sample-size goals.

32This number is slightly greater than the sample size of 6,660 reported in Riccio, Deitch, and Verma (2017) because of new information that became available after the baseline report was completed.

15

Approximately one-half of the 6,665 families who enrol ed in the study were randomly assigned to the group that was subject to the new rent policy, and one-half were assigned to the group that remained subject to the existing policy (the control group). Data on the families’

background characteristics come from PHA administrative records (based on HUD’s 50058

form) and a special background information survey administered to families by the PHAs’

housing specialists at the time of the initial recertification (see the baseline report [Riccio, Deitch, and Verma, 2017] for a ful discussion of these characteristics). As exhibit 2.2 shows, with the samples of all PHAs combined, the average household size was 3.4 family members. In addition, over one-third of families (36.9 percent) had more than one adult living in the household, and nearly one-fourth (22.8 percent) had no children under the age of 18 years. Household composition varied considerably across PHAs. In Washington, D.C., nearly 50 percent of families had more than one adult in the household, compared with 27 to 34 percent in the other PHAs. In addition, 35 percent of families in Washington, D.C., had no children under the age of 18 years, compared with 14 to 22 percent of the other PHAs’ families. This variation may partly reflect the fact that, as exhibit 2.3 shows, the heads of household in Washington, D.C., are older: More than 40 percent are 45 years of age or older, compared with 18 to 22 percent of the heads of other PHAs’ households.

Nearly all (94 percent) of household heads in the study sample are female and, on average, household heads were about 39 years of age when they entered the study (as shown in exhibit 2.3). The majority (69 percent) are Black/African-American. Almost one-fourth (23

percent) are Hispanic/Latino (of any race). In Lexington, Louisvil e, and Washington, D.C., most heads of household are Black/African-American, whereas the majority (75 percent) in San Antonio is Hispanic/Latino.

Exhibit 2.4 shows that most of the other adults in the study households appear to be the household heads’ young adult children. About 80 percent of the non-heads of household were 18 to 24 years of age, and 14 percent were 25 to 34 years of age. A very small proportion (about 7 percent) of the adults who were not household heads were spouses or co-heads of household.

This is a consistent pattern across all four PHAs. About 47 percent of the non-heads of household were female, and their race and ethnicity closely parallel that of the household heads (not shown in table).

Economically, the study sample was substantially disadvantaged at the time of random assignment (as shown in exhibit 2.2). According to PHA data, more than one-half of the study families (58 percent) had no earned income at that time (from any household members, not just household heads), ranging from 53 percent of families in San Antonio to 62 percent in Louisvil e. Even among families who did have earned income, earnings were generally low: Average annual earnings (among families with earnings) ranged from about $13,000 in San Antonio to roughly $27,000 in Washington, D.C. Almost 24 percent of families had income from Social Security, the Supplemental Security Income (SSI) program, or pensions, according to PHA data. In most cases, SSI income was recorded in the PHAs’ administrative records for children or other adults living in the household, not for the household head. (SSI income was entered for the household head for only about 3 percent of households receiving SSI.) The 16

Washington, D.C., families were the most likely by far to receive cash welfare payments: 38

percent of them did, compared with fewer than 6 percent of the families from the other PHAs. 33

33 Cash welfare includes income from TANF and state general assistance programs.

17

Exhibit 2.2. Characteristics of Families in the Impact Sample, by Public Housing Agency (PHA)

Characteristic

Lexing-

Louis- San

Washington,

All

ton

ville Antonio D.C.

PHAs

Average number of family members

3.2

3.3

3.6

3.2

3.4

Families with more than one adult (%)

26.7

33.9

32.7

49.4

36.9

Number of children in the family (%)

None

17.3

21.6

14.0

35.3

22.8

1

24.4

22.4

20.3

25.6

23.0

2

28.4

24.0

27.7

18.0

23.9

3 or more

29.9

32.0

38.1

21.1

30.3

Among families with children, age of the youngest child (%)

0–2 years

16.9

16.7

17.8

16.0

16.9

3–5 years

17.9

17.5

21.7

19.4

19.3

6–12 years

47.3

43.0

42.7

35.7

41.8

13–17 years

17.9

22.7

17.8

28.9

21.9

No earned incomea (%)

53.6

61.8

53.1

60.1

57.7

Income sourcesa (%)

Wages

46.4

38.2

46.9

39.9

42.3

Welfare

5.1

5.8

3.2

37.7

14.1

Social Security/SSI/pensions

19.4

25.8

23.0

23.9

23.5

Other income sources

49.8

44.3

53.1

17.9

40.0

Child support

35.2

28.6

38.0

13.7

28.0

Unemployment benefits

1.0

1.3

2.1

3.4

2.1

Other

17.6

17.0

15.8

1.3

12.2

Average annual income from wages,

among families with any wage incomea

($)

16,625

16,741

12,925

26,853 18,267

Sample size

979

1,908

1,869

1,909

6,665

aIncome-source categories are as defined on the HUD-50058 form. Wages include one's own business, federal wages, PHA wages, military pay, and other wages. Welfare includes general assistance, annual imputed welfare income, and Temporary Assistance for Needy Families. SSI is Supplemental Security Income. Other income sources include child support, medical reimbursement, Indian trust/per capita, unemployment benefits, and other nonwage sources.

Notes: Sample sizes for specific outcomes may vary because of missing values. Rounding may cause slight discrepancies in calculating sums and differences. Data were collected at the most recent recertification before random assignment.

Source: MDRC calculations, using PHA data

18

Exhibit 2.3. Characteristics of Heads of Households in the Impact Sample, by Public Housing Agency (PHA)

San Washington,

Characteristic

Lexington Louisville

Antonio

D.C.

All PHAs

Female (%)

96.8

95.6

93.8

90.9

94.0

Age (%)

18–24

3.5

0.9

5.7

1.4

2.8

25–34

39.0

32.3

38.9

18.6

31.2

35–44

39.8

44.7

35.6

39.4

39.9

45 or older

17.7

22.1

19.8

40.5

26.1

Average age (years)

36.9

38.7

36.7

42.2

38.9

Race (%)

White

18.6

18.2

77.0

2.0

30.2

Black/African-American

81.1

80.3

22.2

97.2

68.9

Other

0.3

1.5

0.8

0.8

0.9

Ethnicity (%)

Hispanic or Latino

1.9

1.2

74.9

3.1

22.5

Not Hispanic or Latino

98.1

98.8

25.1

96.9

77.5

Sample size

979

1,908

1,869

1,909

6,665

Notes: Sample sizes for specific outcomes may vary because of missing values. Rounding may cause slight discrepancies in calculating sums and differences. Data were collected at the most recent recertification before random assignment.

Source: MDRC calculations, using PHA data

19

Exhibit 2.4. Characteristics of Adults Who Are Not Heads of Households, by Public Housing Agency (PHA)

San Washington,

All

Characteristic

Lexington Louisville Antonio

D.C.

PHAs

Age (%)

18–24

74.3

80.4

78.8

80.4

79.5

25–34

13.9

10.8

12.1

16.7

14.0

35–44

8.4

5.8

5.4

1.4

4.0

45 or older

3.4

3.1

3.7

1.5

2.5

Relationship status (%)

Spouse or co-head of household

9.8

7.2

13.5

2.1

6.6

Sample size

296

815

784

1,502

3,397

Notes: Sample sizes represent individuals who were at least 18 years of age at the time of random assignment. Foster children and live-in aides have been excluded. Sample sizes for specific outcomes may vary because of missing values. Rounding may cause slight discrepancies in calculating sums and differences.

Data were collected at the most recent recertification before random assignment.

Source: MDRC calculations, using PHA data

The PHAs involved in the Rent Reform Demonstration were chosen to yield a sample that would broadly reflect the national voucher population. Although four sites cannot truly represent that national population, the families in the study sample (as the baseline report shows) are roughly similar to working-age, nondisabled voucher holders national y, although somewhat more disadvantaged. (For example, they were less likely to be working. 34) According to a brief survey of study families at the time of study enrol ment, many families were contending with significant barriers to employment and material hardships (Riccio, Deitch, and Verma, 2017). For example, 26 percent of household heads reported having no high school diploma or its equivalent, whereas only 12 percent had 2-year or 4-year col ege degrees. In addition, 54 percent of survey respondents reported facing potential impediments to employment, such as physical, emotional, or mental health problems they believed limited their ability to work or the kind of work they could do (31 percent of all respondents); or difficulty affording childcare (21 percent of all respondents).

Almost 70 percent of baseline survey respondents said that they had experienced financial hardship at some time in the past year, such as an inability to pay utility bil s (46

percent), telephone bil s (34 percent), or rent (20 percent). About 28 percent indicated that they sometimes did not have enough money to buy food.

34 See Riccio, Deitch, and Verma (2017) for further details on this comparison.

20

Exhibit 2.5. Data Sources for This Report

• PHA records. Al families receiving a housing voucher complete or update a 50058 form as part of their initial or recertification interview; the information collected by the PHA includes their incomes and income sources, their total tenant payment (TTP) amounts, family share, and their total housing subsidy payment. The study team is collecting this information for al study participants for 1 to 3 years before study enrollment (depending on the PHA) and during the study fol owup period. For families who are subject to the new rent policy, the study team is collecting information on grace-period TTPs, interim recertifications, hardship remedies, and retrospective income.

• Wage records. Employment and earnings data, crucial for the demonstration, wil be obtained from the National Directory of New Hires (NDNH), a national database of wage and employment information that was established by the Personal Responsibility and Work Opportunity Reconcilia-tion Act of 1996 and is maintained by the Office of Child Support Enforcement. The NDNH includes data on employment and earnings in al work covered by unemployment insurance (UI), including across state lines (for those who commute into another state for work or who moved to a different state after random assignment) and on federal employment that is not captured in state UI records. NDNH records do not cover earnings from self-employment, some agricultural work, and informal jobs. Other research suggests that administrative data may miss relatively more employment for low-income populations than for higher income groups, given the former group’s greater prevalence of work in informal jobs (Abraham et al., 2009). NDNH records also do not provide information about hours worked during a quarter or a week, or about the characteristics of jobs held, such as hourly wage rates, benefits, and schedule. For this reason, the study team wil (in a future report) supplement NDNH records data with data from a 42-month survey, which includes information on job characteristics and earnings from informal jobs.

Data Sources and Fol owup Period

The current report uses two main sources of quantitative data in estimating the effects of the new rent policy: PHA administrative records (that is, data col ected in the normal course of administering PHA programs) and unemployment insurance wage records obtained through the National Directory of New Hires (NDNH), which captures employer-reported employment and earnings on adults in the sample. Exhibit 2.5 describes these data sources in greater detail.

Although the PHA data are available for all households, the NDNH data are available for individual household members but not for the household as a whole.

The administrative records data are available for all families starting from the time of random assignment in the case of PHA records and for three quarters before the quarter of random assignment in the case of NDNH data. For the purposes of this report, the “first year of followup” is not defined as beginning at the time of random assignment, as would normally be expected. (See exhibit 2.6, which is a simplified depiction of the sample random assignment, enrol ment, and fol owup period.) This is because it usually took 4 to 5 months after a family’s random assignment date for the TTP to be recalculated (under the new or existing rent rules) and to take effect. Families did not know right away which rent policy would apply to them or what their new TTP would be, and they would not begin paying the new TTP until the designated “effective date.” When examining effects on housing-related outcomes (for example, on TTPs, subsidies, and transactions with PHA staff), it makes sense to focus on outcomes during the period that begins after the effective date; before that date, both research groups are 21

stil subject to the regulations and guidelines of the existing rent policy. If, for example, families’ subsidy receipt pat erns and interactions with the PHAs change for the new rules group, those changes wil occur only after new rules take effect for them.

Exhibit 2.6. Simplified Depiction of Random Assignment and Followup Period

For that reason, for outcomes related to families’ housing subsidies ―which are available from monthly PHA data―the first fol owup year for a family is defined as the period that begins in the month after the month in which the family’s new TTP was expected to take effect. For example, if a family’s recalculated TTP became effective on October 1, 2015, the first fol owup year for measuring the family’s total amount of housing subsidy received would begin in November 2015 and end in October 2016. 35 The same followup period definition applies when examining PHA actions for families, such as the percentage of families who receive interim recertifications in Year 1. Generally, the effective dates occurred within 4 to 6

months after random assignment.

This definition of Year 1 for PHA outcomes aligns closely but not exactly with the definition of Year 1 for the employment-related outcomes based on NDNH data. Overall, for about 82 percent of families, Year 1 as defined for NDNH outcomes and Year 1 as defined for PHA outcomes began in the same quarter. As seen in Chapter 3, the quarter in which a family was randomly assigned is referred to as Quarter 1. For most families, the new TTP effective date occurred in Quarter 3. Thus, Quarter 3 is deemed the beginning of the first followup year for the analyses based on NDNH data because it is “post-effective date” for most families, and Quarter 6 is deemed the end of the first year. 36

35Beginning Year 1 in the first month after the month of the effective date avoids counting that initial recertification as a “followup” action during the first followup year, although a very small number of late initial recertifications fall into that followup period for both research groups.

36For about 62 percent of families, Quarter 3 is the quarter in which the new TTP became effective.

For about 38 percent of families, Quarter 3 is the quarter after the quarter in which the new TTP became 22

Exhibit 2.7 shows the months when study families’ revised TTPs took effect. Across the four PHAs, the effective dates spanned a 10-month period, from June 2015 through March 2016. This means that the impact study’s first year of fol owup (the focus for many of the outcome measures for this report) ended in June 2016 for the earliest enrol ed families, and in March 2017 for the last families enrol ed.

In Washington, D.C., the effective dates fell within a single calendar quarter—between October and December 2015; consequently, the end of the first fol owup year also falls within those months a year later (October through December 2016). For families in the other PHAs, the initial effective dates stretched over a longer period, as did the end dates for the first fol owup year (through as late as March 2017 in San Antonio). Across all PHAs, because about 90 percent of families began paying their newly calculated rents by December 2015, the first fol owup year ended by December 2016 for most families.

Comparability of the Two Research Groups

For the results of an impact analysis to be unbiased, the intervention group and the control group must have a similar distribution of measured and unmeasured characteristics so that if outcomes between those groups differ, the differences can be attributed with confidence to the intervention itself rather than to “selection bias” or to differences in preexisting characteristics of the two research groups that may be related to the outcomes of interest. Random assignment is the most effective mechanism for ensuring comparability between the intervention and control groups; however, sometimes differences between the groups can emerge by chance, a risk that is larger the smaller the sample size. Thus, it is important to assess the extent to which the two groups at least have similar distributions of measurable characteristics (that is, they are in “balance”) before the followup period begins.

MDRC completed such an assessment for the Rent Reform Demonstration and presented results in the baseline report (Riccio, Deitch, and Verma, 2017). As that report shows, when the samples of all four PHAs are combined, only minor and inconsequential differences are evident between the characteristics of families who were randomly assigned to the new rent rules group and those of families assigned to the existing rent rules group. Although a few of those differences are statistically significant, they are not substantively significant (indicating that, for the pooled sample, the random assignment process worked as expected). For most characteristics, if the two groups differ at all, they differ by only a few percentage points. The same pattern is evident in each of the four studies PHAs, although somewhat larger differences between the new rent rules group and the existing rent rules group are evident in San Antonio. 37

effective. Thus, for some families in the new rules group, defining Quarter 3 as the beginning of the Year 1 followup period for the NDNH data analysis means that they will have had some exposure to the new rent rules before Year 1; however, the analyses in Chapter 3 include results for every quarter, starting with the quarter of random assignment.

37A regression analysis was conducted to determine whether background characteristics, taken together, are associated with a family’s likelihood of being assigned to the new rent rules group rather than the existing rent rules group. For the full sample from al PHAs combined, the association was not 23

As explained in the section below on regression adjustment, the impact analysis applies standard statistical techniques to adjust for any measured background differences between the two research groups in estimating the impacts of the new rent policy at each PHA and for all PHAs combined. Those adjustments reduce potential selection bias and produce more precise estimates of the new policy’s effects.

Exhibit 2.7. Random Assignment Period, New Rent Effective Dates, and Year 1

End Dates, by Public Housing Agency (PHA)

Random Assignment Perioda

New Rent Effective Last Month of Year 1

Dateb

(PHA outcomes)c

PHA

Start Date

End Date

Lexington

March

August

July 2015–

2015

2015

December 2015

July 2016-December 2016

Louisville

February

August

July 2015–

2015

2015

January 2016

July 2016-January 2017

San Antonio February

November

June 2015–

2015

2015

March 2016

June 2016-March 2017

Washington, April

June

October 2015–

October 2016-December

D.C.

2015

2015

December 2015

2016

All PHAs

February

November

June 2015–

2015

2015

March 2016

June 2016-March 2017

a Random assignment is when households were randomly assigned to the new rent rules or existing rent rules group. Households were notified that they were in the demonstration in their recertification packet from their public housing agency, and details about their research group assignment and the study were explained in their recertification meeting.

b The expected new rent effective date is the date that the new total tenant payment and housing assistance payment were expected to go into effect for the annual or triennial recertification.

c For PHA outcomes, Year 1 starts the month after the expected new rent effective date. For example, if the new rent effective date was June 2015, then Year 1 is from July 2015 through June 2016.

Another way of assessing the comparability of the two research groups is to compare their pre-random assignment employment and earnings using the NDNH data. Those data were available for all sample members for three quarters before each family’s quarter of random assignment. Exhibit 2.8 shows the findings for all PHAs combined, separately for the heads of household (top panel) and other adults (bottom panel). It shows that any differences between the new rent rules group and the existing rent rules group in the employment rate and average earnings in each of the prior quarters were minor and were generally not statistically significant. Exhibits B.1 and B.2 present comparisons for each PHA taken separately and generally show few notable differences on those measures. 38 At first statistically significant, nor was it statistically significant for the PHAs separately, with the exception of San Antonio. The statistical significance observed for San Antonio was possibly the result of certain exclusions from the sample that had to be made after random assignment. (See Appendix H in Riccio, Deitch, and Verma [2017], available at www.huduser.gov and www.mdrc.org).

38For guidance on how to read an impact table, see exhibit B-3.1 in Chapter 3.

24

glance, San Antonio may seem like an exception because some statistically significant differences are evident for household heads (as shown in exhibit B.1); however, the direction of the differences shifts between negative and positive across the pre-random assignment quarters, so the cumulative difference is very small and inconsequential. For example, the cumulative difference in the pre-random assignment quarterly earnings is only $36, or about 0.6 percent of the control group’s cumulative earnings for those three quarters. Overall, the pre-random assignment employment and earnings patterns suggest that the two research groups are well balanced and that the estimated impacts of the new rent policy will be unbiased.

25

Exhibit 2.8. Employment and Earnings in the Three Quarters Before Random Assignment, by Household Head Status

New

Existing

Rent

Rent Difference

Outcomes

Rules

Rules

(Impact)

P-Value

Heads of households

Ever employed (%)

Third quarter before random assignment

52.2

52.5

-0.3

0.718

Second quarter before random assign-

*

ment

54.0

52.6

1.4 *

0.041

Quarter before random assignment

53.4

53.6

-0.2

0.769

Total earnings ($)

Third quarter before random assignment

2,032

2,048

-17

0.663

Second quarter before random assign-

ment

2,226

2,178

48

0.148

Quarter before random assignment

2,194

2,144

49

0.169

Sample size (total = 6,665)

3,312

3,353

Non-Heads of households

Ever employed (%)

Third quarter before random assignment

42.2

41.0

1.2

0.314

Second quarter before random assign-

ment

43.5

43.5

0.0

0.994

Quarter before random assignment

45.0

45.1

-0.1

0.915

Total earnings ($)

Third quarter before random assignment

1,130

1,090

40

0.319

Second quarter before random assign-

ment

1,230

1,251

-21

0.531

Quarter before random assignment

1,284

1,275

9

0.818

Sample size (total = 3,397)

1,737

1,660

* = 10 percent; ** = 5 percent; and *** = 1 percent.

Notes: Twenty-nine heads of households from the program group and 54 heads of households from the control group are missing data for one or more pre-random assignment quarters. One-hundred fifteen non-heads of households from the program group and 131 non-heads of households from the control group are missing data for one or more pre-random assignment quarters. Estimates were regression adjusted using ordinary least squares, controlling for pre-random assignment characteristics of sample members. Rounding may cause slight discrepancies in calculating sums and differences. A two-tailed t-test was applied to differences between research groups. The p-value indicates the likelihood that the difference between the new rent rules group and the existing rent rules group arose by chance. Sample sizes for specific outcomes may vary because of missing values.

Source: MDRC calculations, using quarterly wage data from the National Directory of New Hires 26

Analysis Issues and Strategies

Units of Analysis

In examining the effects of the new rent policy on labor market outcomes based on NDNH data, Chapter 3 of this report gives primary at ention to the heads of household, who make up 66

percent of all adults in the study. The report also examines, secondarily, the effects on other adults (non-heads) as well as all adults combined (heads plus non-heads of household). The main reason for giving top priority to household heads is that most of the other adults in the research sample (nearly 80 percent across all PHAs combined) were 18 to 24 years of age at the time of random assignment and are very likely the young adult children of the household heads; very few (6.6 percent) are the spouse or co-head of household (see exhibit 2.4). This pattern generally prevails across the four PHAs. San Antonio had the largest number of non-heads who are spouses or co-heads (13.5 percent) and Washington, D.C., had the fewest (2.1 percent).

Other data (not shown) indicate that about 21 percent of other adults were no longer listed as household members in PHA records by the end of the first year of followup. It is not known whether those individuals had moved out of the household (for example, to attend col ege or to begin their own households) or remained in the household but not on the lease. This rate is sure to rise over the course of the fol owup period, meaning that fewer other adults wil be exposed to the new rent policy (or the existing rules) over time, making it progressively less likely that their behavior would be shaped by the new policy. Of course, any exposure to the new rent policy may affect their employment outcomes and possibly even their likelihood of continuing to live with their initial households or remain on the lease. For that reason, the labor market results for non-heads should not be ignored. These results, along with findings for all adults combined, are presented in appendix C.

In examining the effects of the new rent policy on housing-related outcomes (such as housing subsidies, exits from the voucher program, and transactions with the PHA), which are based on PHA data, the household is the unit of analysis (as covered in Chapter 4).

Pooled and PHA-Specific Impacts

The impact analysis examines the effects of the new rent rules using a pooled sample, which combines the samples of all four PHAs and for each PHA separately. Pooling increases the precision of impact estimates, which becomes especially relevant when estimating effects for subgroups of the study sample because of the limited size of subgroups within each PHA’s sample. PHA-specific estimates allow the analysis to test the “robustness” of the new rent model; that is, each site provides a type of independent replication test. As Chapter 3 of this report shows, some differences in impacts have begun to emerge in the first year of fol owup; however, those are very early findings and the cross-site patterns may not persist. It wil be important to determine whether they hold over the longer fol owup period before trying to explain them. Hence, this report focuses primarily on describing rather than trying to explain those early patterns.

27

Regression Adjustment

The basic estimation strategy used to assess the impacts of the new rent policy is analogous to the method that researchers have used in many social experiments over the past few decades to generate credible results. The analysis wil compare the average outcomes of the new and existing rent rules groups of specified fol owup periods, using regression adjustments to increase the precision of the statistical estimates. 39 A linear regression framework is being used to adjust impacts, with the fol owing basic impact model:

Yi = α + βPi + δXi + εi

where: Yi = the outcome measure for sample member i; Pi = 1 for program (or intervention) group members and zero for control group members; Xi = a set of background characteristics for sample member i; εi = a random error term for sample member i; β = the estimate of the impact of the program on the average value of the outcome; α = the intercept of the regression; and δ = the set of regression coefficients for the background characteristics.

Adjusting for Multiple Outcome Measures

The evaluation design includes several “confirmatory” outcome measures related to tenants’

earnings, housing subsidies, and material hardships. These confirmatory outcomes reflect the most important variables for judging the intervention’s effectiveness. Given their primacy, impact findings on those outcomes were subjected to further statistical adjustments that hold them to a higher standard of evidence. These adjustments account for the likelihood that in a study using many outcome variables, some impact estimates may emerge as statistically significant simply by chance and do not reflect true intervention effects. For example, if 10

outcomes are examined in a study of an ineffective treatment, it is likely that one of them wil be statistically significant at the 10-percent level only by chance. One can have more confidence in any confirmatory impact estimates that remain statistically significant after adjusting for the total number of confirmatory outcome measures. The current report treats first-year pooled impact estimates for household heads’ earnings and families’ housing subsidy receipt as preliminary confirmatory measures. It subjects them to adjustment using the Benjamini-Hochberg method (Benjamini and Hochberg, 1995). The final evaluation report wil present the final confirmatory impact estimates and adjustments, using longer-term data on these measures.

It wil also include a survey-based family hardship scale as a confirmatory outcome measure.

Appendix B more fully describes the evaluation’s confirmatory measures and its approach for adjusting for multiple outcomes.

39In making these adjustments, an outcome, such as “employed” or “received housing subsidy,” is regressed on an indicator for intervention group status, site (for all-sites analysis), and a range of background characteristics at random assignment, including race, ethnicity, age, number of adults in the household, age of the youngest child, family share, type of income reported for the Housing Choice Voucher program certification, number of years of subsidy receipt through the Housing Choice Voucher program, and whether gross rent exceeds the payment standard. When estimating effects for the pooled sample, site covariates are also included in the model.

28

Variation in Rent Policies Across PHAs

As discussed in Chapter 1, the four PHAs largely implemented the same new rent policy for the demonstration; however, some exceptions exist and have implications for the interpretations of the pooled and site-specific impact findings. Also important are some differences in the existing rent rules operating at each of the participating PHAs at the start of the demonstration. These rules largely mirror HUD’s traditional rent rules, operated by non-Moving to Work agencies across the country. 40 As Chapter 1 indicated, however, the PHAs that were selected for the demonstration had already implemented some policy changes before they joined, which means that the control group policy is not the same across all of them (see exhibit A.1).

The most important variation in the new rent policies across PHAs concerns minimum TTPs. As mentioned previously, Lexington had already introduced a $150 minimum TTP

before the demonstration began, with few exemptions permit ed. Because it continued that policy for both the new rules group and the control group, any impacts that were estimated for Lexington reflect only the other features of the new rent rules, not any possible effects of a minimum TTP. In the other PHAs, a differential between the two research groups on the minimum TTP element of the policy does exist, although to different degrees. Two of the four PHAs—Louisvil e and Washington, D.C.—introduced a minimum TTP for the first time ($50

and $75, respectively), whereas San Antonio increased its existing $50 minimum TTP to $100.

Although the levels vary, it is possible that in these three sites, impacts of the new rent policy may at least partially reflect the effects of the minimum TTP.

Furthermore, when PHAs were selected for the Rent Reform Demonstration, the PHA in Washington, D.C., had already adopted a biennial recertification policy according to which a working-age/nondisabled family whose anticipated income from the same income source increased up to $10,000 per year would not have its TTP recalculated until its next biennial recertification; those with larger income increases (which, for example, would include tenants going to work full time at a minimum wage) would continue to have their TTPs adjusted when the increases occurred. This policy was changed in June 2016 (during this study’s first fol owup year), however, to eliminate all income-reporting requirements before the required biennial recertification. Consequently, estimates of the first-year impacts of the new rent rules in Washington, D.C., wil not reflect the extension of the recertification period to 3 years. Because the triennial recertification is the most important financial work incentive under the new rent policy and cannot yet be tested in Washington, D.C., the impact estimates that are presented in Chapters 3 and 4 include supplementary pooled estimates combining results for the three other PHAs, for which the annual recertification policy remained in place for the ful control group.

These variations across PHAs in some aspects of the new rent policy and in the existing rules to which the control group families are subject are important to keep in mind when reviewing this study’s impact findings.

40The traditional rent rules referred to in this report are those in effect before the July 2016 passage of the Housing Opportunity Through Modernization Act of 2016.

29

Louisvil e Opt-Outs

As explained previously, families in Louisvil e who were randomly assigned to the new rent policy group were permit ed to opt out of that group and continue to be subject to the existing rent rules. About 22 percent chose to do so. Those who chose to opt out differed in important ways from those who did not make this choice. For example, they were more likely to have lower household incomes and were less likely to have any earned income, both statistically significant differences. They also had somewhat lower TTPs (and somewhat higher housing subsidies) under the existing rules than they would have had under the new rent rules. In addition, the heads of these households tended to be older than the heads of those households who did not opt out. PHA staff members reported that some families simply favored whichever policy would leave them paying the lowest initial rent. Some families may not have expected to increase their earnings and so may not have expected to benefit from the new policy. 41 Or some families may simply have felt more comfortable sticking with a set of rules they already knew and were used to following.

Few families who opted out of the new rent policy chose to opt out of the evaluation.

To avoid introducing selection bias into the impact analysis—in other words, to ensure that the same types of families are included in each research group when the outcomes of these groups are compared—the evaluation stil treats the opt-out families as members of the new rent rules group even though they are subject to the existing rent rules. This decision ensures that the evaluation’s estimated impacts wil be unbiased (in the sense that the initial characteristics of the new and existing rent rules groups remain similar), which is essential for determining whether the new rules have a causal impact on the outcomes of interest. It also means, however, that the magnitude of the estimated effects of the new rent policy may be somewhat diluted because not all members of the new rent rules group were exposed to the new policy.

Given the substantial opt-out rate in Louisvil e and recognizing that any effects on outcomes can be attributed solely to families who did not opt out, this study includes a set of supplementary estimates that adjust the impact to account for the fact that some members of the new rent rules group were not exposed to the new rent rules. These estimates are derived from what is commonly referred to as a treatment-on-treated, or TOT, analysis. For a specified outcome measure, the TOT result was computed by dividing the estimated impact by the proportion of families assigned to the new rent rules group who chose to remain with the new policy. The TOT estimates do not affect levels of statistical significance of the impact estimates.

In other words, if the original estimated impact (reflecting an intent-to-treat, or ITT, analytical approach42) is not statistically significant, the TOT estimate wil also not be statistically significant even if the magnitude of the difference in outcomes is larger than the original 41See appendix C of the baseline report for a detailed analysis comparing families in Louisville who opted out of the new rent policy with those who did not opt out (Riccio, Deitch, and Verma, 2017).

42 An ITT analysis captures the average impact on the entire group intended to receive the intervention, whether or not every member of that group actually received it.

30

estimate. Thus, a TOT adjustment cannot offer any more assurance that an estimated effect is not a result of chance.

31

Chapter 3

Early Impacts on Employment and Earnings

This chapter looks at the early effects of the new rent policy on voucher holders’ employment rates and average earnings, primarily during the first year of fol owup after the newly calculated total tenant payments (TTPs) took effect for families after they entered the study and for two extra quarters. Although the chapter presents 18 months of followup data on employment outcomes, it highlights the 1-year timeframe because that period generally aligns with the length of fol owup period available for this report for examining effects on housing-related outcomes, such as TTP levels, subsidy levels, and transactions with staff, which are presented in Chapter 4.

The first fol owup year was a time when the new rent policy was stil unfamiliar to families, limiting its potential effects. Families were just learning how the new policy would support work, and they may have come to appreciate its potential benefits only gradually.

Furthermore, some adults who were inspired by the new rules may have needed a fair amount of time to find work or increase their hours of work. Some may have had to overcome common types of impediments—for example, finding suitable job openings given their skil s, or arranging transportation or childcare—or to resolve other types of personal or family circumstances to take advantage of the new policy. For those reasons, a fair assessment of the effects of the new policy on voucher holders’ labor market outcomes requires a longer fol owup period, which future reports wil cover.

Stil , it is possible to get an early sense of potential longer-term trends with the two additional quarters of employment and earnings data that were available in time for the current report. Those extra 6 months mean that families in the new rent rules group have had a somewhat longer period of exposure to the new rules and more time to respond to them. Also, those extra months came after most families in the existing rent rules (control) group had completed their first annual recertification since their initial recertification at the beginning of the study. Control group families in Washington, D.C., are the exception because they are subject to a biennial recertification policy.

This chapter focuses most at ention on the adults who were heads of household at the time of random assignment for the four public housing agencies (PHAs) in the study—

Lexington, KY; Louisvil e, KY; San Antonio, TX; and Washington, D.C. As explained in Chapter 2, most “other adults” in the families were the young adult children of the household heads, and roughly one-fifth of them left the household or were no longer on the lease within the first year of followup. Appendix C presents the findings for those individuals, as wel as for all adults in the combined sample (household heads plus non-household heads).

As wil be seen, the new policy increased the average earnings of heads of household in the new rules group (compared with the control group) in two of the four PHAs—Lexington and San Antonio—within the first 18 months of fol owup. In Washington, D.C., the control group’s biennial recertification schedule means the impact findings for that site do not yet reflect the potential effects of the new policy’s extended recertification period. When the results for all PHAs are pooled, they show small positive effects on average quarterly employment, and 31

small positive but not statistically significant effects on average earnings, except when Washington, D.C., is excluded from the pooled estimates in a supplementary analysis. As explained in the chapter, when interpreting the pooled findings, it is important to keep in mind the variations across PHAs in some features of the model and in the policies for the existing rules group.

Early Impacts for Heads of Household

To measure the effects of the new rent policy on adults’ labor market outcomes, the evaluation uses administrative records from the National Directory of New Hires (NDNH), which col ects wage data that employers in each state report to their state unemployment insurance (UI) systems on a quarterly basis. As discussed in Chapter 2 and appendix B, the pooled impact estimate for Year 1 average earnings represents a preliminary confirmatory outcome measure for the evaluation.

Results for the Pooled Sample

Exhibit 3.1 shows results for the pooled sample, with families from all four PHAs combined. It depicts the early trends for employment and earnings outcomes for the new rent rules group and the existing rent rules group. The results cover the period from the time of random assignment (Quarter 1) through the first year of fol owup (defined here as Quarters 3 through Quarter 6

after the quarter in which random assignment occurred) and through the first two quarters of the second year of fol owup (Quarters 7 and 8). As explained in Chapter 2, the first year of fol owup begins in Quarter 3 because it is the quarter in which most families’ newly calculated TTPs took effect under the new or existing rent rules. The differences between the lines in the graphs represent the effects, or “impacts,” of the new rent policy. Any quarter in which the size of that difference is statistically significant is indicated by one, two, or three asterisks (representing statistical significance levels of 10 percent, 5 percent, and 1 percent, respectively) under the relevant quarter.

Overal , the pooled results show that heads of household in the new rules group had slightly higher employment rates and average earnings than those in the existing rules group over the course of the first 18 months of fol owup, although the differences were typically not statistically significant. Also noteworthy is that for both research groups, average earnings climbed substantially more than did employment rates. This result likely reflects an increase over time in the number of hours worked per quarter (including, for example, a shift from part-time to ful -time employment) or an increase in average hourly wages, or a combination of the two (it is not possible to distinguish between those patterns with the quarterly NDNH data). The improving economy over this period and some increases in the hourly minimum wage may have contributed to the earnings growth experienced by both research groups. 43

43Over the course of the followup period for this report, unemployment rates in the metropolitan areas in which the PHAs are located were relatively low and generally stable or falling by a small degree.

From February 2015 through May 2017, they ranged from in 4.1 to 3.7 percent in Lexington-Fayette; 4.8

to 4.2 percent in Louisville/Jefferson County; 3.9 to 3.6 percent in San Antonio-New Braunfels; and 4.7

to 3.7 percent in Washington-Arlington-Alexandria. Some changes in the minimum wage were also introduced. In Louisville, the minimum wage rose in July 2015 from $7.25 per hour to $7.75, and then in 32

Exhibit 3.1. Quarterly Impacts on Household Heads’ Employment and Earnings, Al Public Housing Agencies (PHAs)

A. Employment

80

75

70

65

)

60

New rent rules

55

ployed (%

50

Existing rent rules

Ever Em 45

40

Year 1

35

30

Q1

Q2

Q3

Q4

Q5

Q6

Q7

Q8

***

Fol owup Quarter Relative to Quarter of Random Assignment

B. Earnings

3,500

3,300

3,100

2,900

2,700

New rent rules

2,500

2,300

Total Earnings ($)

Existing rent rules

2,100

1,900

Year 1

1,700

1,500

Q1

Q2

Q3

Q4

Q5

Q6

Q7

Q8

**

**

Fol owup Quarter Relative to Quarter of Random Assignment

(continued)

July 2016, it increased to $8.25; however, in October 2016, the Kentucky Supreme Court struck down the local ordinance, and the minimum wage returned to $7.25. In Lexington, in July 2016, the minimum wage rose from $7.25 to $8.20, but the same court action returned it to $7.25. No changes were made to the minimum wage in San Antonio, where it remained at $7.25. In Washington, D.C., in July 2015, the minimum wage rose from $9.50 to $10.50, and then to $11.50 in July 2016 (U.S. Department of Labor, 2018).

33

Exhibit 3.1 (continued)

* = 10 percent. ** = 5 percent. *** = 1 percent.

Notes: Estimates were regression-adjusted using ordinary least squares, controlling for pre-random assignment characteristics of sample members. Quarter 1 (Q1) is the quarter of random assignment. Rounding may cause slight discrepancies in calculating sums and differences. A two-tailed t-test was applied to differences between research groups.

The p-value indicates the likelihood that the difference between the new rent rules group and the existing rent rules group arose by chance. Sample sizes for specific outcomes may vary because of missing values.

Source: MDRC calculations, using quarterly wage data from the National Directory of New Hires

Exhibit 3.2 shows the detailed quarter-by-quarter results, along with several summary measures (see exhibit 3.3 for an explanation of how to read the tables showing impacts in this report). The table shows, for example, that over the first year, with all four PHAs combined, more than two-thirds of household heads in either the new rent rules group (68.2 percent) or in the existing rent rules group (66.8 percent) had worked at any point during that year in a job covered by the UI system. Moreover, 56 percent of the new rules group had worked in an average quarter, which exceeded the level for the existing rules group by a small but statistically significant amount (1.6 percentage points). The new rules group was also somewhat more likely (by 2.6 percentage points) to work during all four quarters of Year 1.

Average earnings, though rising, were low. For example, average earnings for the entire new rules group (including household heads who had zero earnings) were $10,185 during Year 1. This amount translates to an average of roughly $15,000 per person who ever worked in Year 1. The new rent policy’s impacts on earnings (indicated by the differences between the two study groups) were small and generally not statistically significant.

34

Exhibit 3.2. Early Impacts on Employment and Earnings: Heads of Households New

Existing

Difference

Outcomes

Rent Rules

Rent Rules

(Impact)

P-Value

Lexington

Ever employed (%)

Year 1 (quarters 3–6)

78.5

75.9

2.6

0.226

Quarter 1 (random assignment)

62.4

59.8

2.6

0.242

Quarter 2

65.8

62.7

3.1

0.181

Quarter 3

65.4

65.7

-0.2

0.928

Quarter 4

65.5

64.5

1.0

0.688

Quarter 5

65.8

63.3

2.6

0.313

Quarter 6

66.0

62.5

3.5

0.191

Quarter 7

65.7

63.7

2.0

0.458

Quarter 8

65.2

61.5

3.8

0.154

Employed in all quarters, Year 1 (%)

51.0

48.8

2.3

0.401

Average quarterly employment, Year 1a (%)

65.7

64.0

1.7

0.395

Total earnings ($)

Year 1 (quarters 3–6)

10,387

9,921

466

0.315

Quarter 1 (random assignment)

2,096

1,881

215 **

0.014

Quarter 2

2,360

2,312

48

0.673

Quarter 3

2,544

2,560

-15

0.907

Quarter 4

2,446

2,389

56

0.661

Quarter 5

2,602

2,454

148

0.289

Quarter 6

2,809

2,582

227

0.158

Quarter 7

2,881

2,583

298 *

0.065

Quarter 8

2,869

2,517

352 **

0.028

Sample size (total = 979)

486

493

Louisville

Ever employed (%)

Year 1 (quarters 3–6)

71.8

72.1

-0.3

0.841

Quarter 1 (random assignment)

52.8

55.1

-2.3

0.133

Quarter 2

57.8

58.8

-1.0

0.548

Quarter 3

60.1

59.9

0.2

0.914

Quarter 4

59.7

58.7

1.0

0.582

Quarter 5

59.8

59.3

0.5

0.792

Quarter 6

63.2

61.3

1.9

0.327

Quarter 7

61.0

62.1

-1.2

0.540

Quarter 8

60.7

62.1

-1.4

0.474

Employed in all quarters 3–6 (%)

48.0

43.8

4.3 **

0.024

Average quarterly employment, Year 1a (%)

60.7

59.8

0.9

0.544

(continued)

35

Exhibit 3.2 (continued)

New

Existing

Difference

Outcomes

Rent Rules Rent Rules

(Impact)

P-Value

Total earnings ($)

Year 1 (quarters 3–6)

10,113

10,083

29

0.936

Quarter 1 (random assignment)

1,931

2,027

-96

0.164

Quarter 2

2,186

2,154

32

0.710

Quarter 3

2,423

2,392

31

0.763

Quarter 4

2,460

2,469

-8

0.939

Quarter 5

2,499

2,440

59

0.590

Quarter 6

2,733

2,800

-67

0.581

Quarter 7

2,775

3,027

-252 **

0.047

Quarter 8

2,822

3,003

-181

0.181

Sample size (total = 1,908)

947

961

San Antonio

Ever employed (%)

Year 1 (quarters 3–6)

71.1

67.8

3.4 *

0.052

Quarter 1 (random assignment)

56.1

56.4

-0.3

0.808

Quarter 2

59.9

56.1

3.8 **

0.023

Quarter 3

59.9

56.3

3.6 **

0.046

Quarter 4

59.2

57.2

2.1

0.263

Quarter 5

61.7

57.5

4.2 **

0.024

Quarter 6

61.3

58.2

3.1

0.107

Quarter 7

61.4

58.6

2.9

0.147

Quarter 8

60.4

56.6

3.8 *

0.056

Employed in all quarters, Year 1 (%)

48.3

45.5

2.8

0.128

Average quarterly employment, Year 1a (%)

60.5

57.3

3.2 **

0.037

Total earnings ($)

Year 1 (quarters 3–6)

10,003

9,086

916 ***

0.010

Quarter 1 (random assignment)

2,172

2,070

102

0.106

Quarter 2

2,333

2,138

195 **

0.016

Quarter 3

2,384

2,174

210 **

0.021

Quarter 4

2,455

2,320

135

0.171

Quarter 5

2,538

2,271

267 ***

0.010

Quarter 6

2,686

2,394

292 **

0.011

Quarter 7

2,704

2,379

324 ***

0.007

Quarter 8

2,786

2,395

392 ***

0.002

Sample size (total = 1,869)

935

934

Washington, D.C.

Ever employed (%)

Year 1 (quarters 3–6)

56.4

55.6

0.8

0.637

Quarter 1 (random assignment)

44.8

42.7

2.1

0.138

Quarter 2

37.1

38.9

-1.8

0.328

(continued)

36

Exhibit 3.2 (continued)

New

Existing

Difference

Outcomes

Rent Rules Rent Rules

(Impact)

P-Value

Quarter 3

48.2

48.1

0.1

0.942

Quarter 4

38.4

40.0

-1.6

0.405

Quarter 5

40.1

41.1

-0.9

0.635

Quarter 6

40.2

37.2

3.0

0.121

Quarter 7

51.1

49.1

1.9

0.311

Quarter 8

48.7

48.4

0.3

0.863

Employed in all quarters, Year 1 (%)

23.3

23.2

0.1

0.976

Average quarterly employment, Year 1a (%)

41.7

41.6

0.2

0.899

Total earnings ($)

Year 1 (quarters 3–6)

10,266

10,634

-368

0.379

Quarter 1 (random assignment)

2,731

2,546

185 **

0.037

Quarter 2

2,053

2,198

-144

0.248

Quarter 3

3,069

3,219

-150

0.213

Quarter 4

2,219

2,410

-191

0.139

Quarter 5

2,317

2,448

-132

0.361

Quarter 6

2,669

2,579

90

0.560

Quarter 7

3,387

3,315

72

0.622

Quarter 8

3,264

3,264

0

0.999

Sample size (total = 1,909)

944

965

aAverage quarterly employment is calculated as total number of quarters with employment divided by total number of quarters of followup, expressed as a percentage.

* = 10 percent. ** = 5 percent. *** = 1 percent.

Notes: Estimates were regression-adjusted using ordinary least squares, controlling for pre-random assignment characteristics of sample members. Rounding may cause slight discrepancies in calculating sums and differences. A two-tailed t-test was applied to differences between research groups. The p-value indicates the likelihood that the difference between the new rent rules group and the existing rent rules group arose by chance. Sample sizes for specific outcomes may vary because of missing values. The variation across the four PHAs in estimated impacts on total earnings in Year 1 is statistically significant at the 10 percent level based on an H-statistic test.

Source: MDRC calculations, using quarterly wage data from the National Directory of New Hires As mentioned previously, the pooled results of the new rent policy can be difficult to interpret because of differences across the PHAs in their minimum TTPs and control group conditions. Particularly important is the biennial recertification policy in effect for the control group in Washington, D.C. Because of that policy, the new rules group wil experience no meaningful advantage in terms of the triennial recertification (the new policy’s most important financial work incentive) until the third year of fol owup. For this reason, as discussed in Chapter 2, it is useful to consider labor market impacts for the pooled sample without Washington, D.C., so that all families in the pooled control group are subject to an annual recertification policy. The second panel of exhibit 3.2 presents those results. It shows somewhat 37

larger impacts of the new rent policy than the pooled estimates with Washington, D.C., including a statistically significant increase in Year 1 average earnings of $466, a gain of 4.8

percent above the control group average of $9,660 per household head. This earnings impact remained statistically significant when adjusted for multiple outcomes. 44

Results by PHA

Important differences by PHA lie behind the general patterns of early labor market impacts for the pooled sample. Exhibits 3.4 and 3.5 show that the new rent rules produced more positive results in Lexington and San Antonio than in Louisvil e and Washington, D.C. In Lexington, the effects of the new policy on earnings began to grow midway through Year 1 and were statistically significant in Quarters 7 and 8. In Quarter 8, the new rules group’s average earnings were $352 higher than those of the existing rules group, a gain of 14 percent; however, this earnings gain was achieved without a statistically significant increase in employment rates, suggesting that it resulted from an increase in hours worked and/or an increase in hourly wages (it is not possible to determine wages and hours worked from the quarterly UI records).

Lexington’s high minimum TTP policy could not have contributed to this earnings gain because that same minimum TTP applied to the control group. Thus, it most likely reflects the new policy’s effect of extending the recertification period from 1 year to 3 years, which offers a substantial financial incentive to work.

44 The impact estimate was adjusted using the Benjamini-Hochberg method described in Appendix B. The adjustment takes into account the fact that impacts were also estimated for a second preliminary confirmatory outcome measure (Year 1 housing subsidy payments to families). The adjusted p-value =

.035 for the pooled sample without Washington, D.C.

38

Exhibit 3.3. How to Read the Impact Tables in This Report

In the context of this evaluation, an “impact” is a measure of how much the intervention—the new rent rules policy implemented in the Rent Reform Demonstration—changed outcomes for program participants. The program group outcome for the intervention is compared with that of the control group. The top row of the excerpted table below, for example, shows that an average of 56.0 percent of the new rent rules, or program, group was working in an average quarter in Year 1, compared with 54.4 percent of the existing rent rules, or control, group.

Because participants were assigned randomly to either the program group or the control group, the effects of the intervention, or program, can be estimated by the difference in outcomes between the two groups. The “Difference” column in the table excerpt shows the differences between the two research groups’ outcomes—that is, the program’s estimated impacts on the outcomes. For example, the estimated impact of the program on the average number of individuals employed can be calculated by subtracting 54.4 percent from 56.0 percent, yielding a difference, or estimated impact, of 1.6 percentage points.

The p-value shows the probability that this impact arose by chance. In the table excerpt, the difference between the program and control groups in average quarterly employment in Year 1

has a 4.6 percent probability of arising as a result of chance rather than as a result of the program.

In contrast, the difference on the measure of total earnings in Year 1 has an 18.2 percent probability of having arisen by chance. For this evaluation, only differences that have a 10 percent probability or less of arising by chance are considered “statistically significant” and therefore represent true program effects. The number of asterisks indicates whether the impact is statistically significant at the 1-percent (***), 5-percent (**), or 10-percent (*) level, meaning that there is only a 1-, 5-, or 10-percent probability, respectively, that the impact arose by chance.

New

Existing

Difference

Outcomes

Rent Rules

Rent Rules

(Impact)

P-Value

All PHAs

Average quarterly

employment, Year 1 (%)

56.0

54.4

1.6 **

0.046

Total earnings ($)

Year 1 (quarters 3–6)

10,185

9,922

263

0.182

39

Exhibit 3.4. Quarterly Impacts on Household Heads’ Employment and Earnings, by Public Housing Agency (PHA)

Employment

Earnings

A. Lexington

80

3,500

75

3,300

New rent rules

) 70

3,100

65

2,900

New rent rules

60

2,700

55

Existing rent rules

2,500

50

2,300

45

2,100

Existing rent rules

Ever Employed (%

Total Earnings ($)

40

Year 1

1,900

Year 1

35

1,700

30

1,500

Q1

Q2

Q3

Q4

Q5

Q6

Q7

Q8

Q1

Q2

Q3

Q4

Q5

Q6

Q7

Q8

Fol owup quarter relative to quarter of random assignment

Fo

** l owup quarter relative to quarter of random assi*gnment**

B. Louisville

80

3,500

75

3,300

70

)

3,100

65

New rent rules

2,900

New rent rules

60

2,700

55

2,500

Existing rent rules

50

2,300

45

Existing rent rules

Ever Employed (%

2,100

Total Earnings ($)

40

Year 1

1,900

Year 1

35

1,700

30

1,500

Q1

Q2

Q3

Q4

Q5

Q6

Q7

Q8

Q1

Q2

Q3

Q4

Q5

Q6

Q7

Q8

**

Fol owup quarter relative to quarter of random assignment

Fol owup quarter relative to quarter of random assignment

(continued)

40

Exhibit 3.4 (continued)

Employment

Earnings

C. San Antonio

80

3,500

75

3,300

) 70

Year 1

New rent rules

3,100

65

2,900

New rent rules

60

2,700

55

2,500

50

Existing rent rules

2,300

45

2,100

Ever Employed (%

Total Earnings ($)

40

Year 1

Existing rent rules

1,900

35

1,700

30

1,500

Q1

Q2

Q3

Q4

Q5

Q6

Q7

Q8

Q1

Q2

Q3

Q4

Q5

Q6

Q7

Q8

**

**

**

*

**

**

***

**

***

***

Fol owup quarter relative to quarter of random assignment

Fol owup quarter relative to quarter of random assignment

D. Washington, D.C.

80

3,500

75

3,300

70

)

Year 1

3,100

65

2,900

Existing rent rules

60

2,700

55

2,500

50

New rent rules

2,300

45

2,100

Ever Employed (%

Total Earnings ($)

New rent rules

40

1,900

35

1,700

Year 1

Existing rent rules

30

1,500

Q1

Q2

Q3

Q4

Q5

Q6

Q7

Q8

Q1

Q2

Q3

Q4

Q5

Q6

Q7

Q8

Fol owup quarter relative to quarter of random assignment

F

**ol owup quarter relative to quarter of random assignment

(continued)

41

Exhibit 3.4 (continued)

* = 10 percent. ** = 5 percent. *** = 1 percent.

Notes: Average quarterly employment rate is calculated as total number of quarters with employment divided by total number of quarters of followup, expressed as a percentage. Quarter 1 (Q1) is the quarter of random assignment. Estimates were regression adjusted using ordinary least squares, controlling for pre-random assignment characteristics of sample members. Rounding may cause slight discrepancies in calculating sums and differences. A two-tailed t-test was applied to differences between research groups. The p-value indicates the likelihood that the difference between the new rent rules group and the existing rent rules group arose by chance.

Sample sizes for specific outcomes may vary because of missing values.

Source: MDRC calculations, using quarterly wage data from the National Directory of New Hires In San Antonio, the policy had statistically significant and positive effects on employment as well as earnings. For example, the average quarterly employment rate during Year 1 increased by 3.2

percentage points (see exhibit 3.5). The impact on average earnings for Year 1 was $916 (an increase of 10 percent), with the largest quarterly impact evident in Quarter 8 ($392, or 16 percent above the control group mean; the variation across the four PHAs in effects on total earnings in Year 1 is statistically significant at the 10-percent level based on an H-statistic test).

It is also noteworthy that San Antonio’s positive impacts on earnings began to emerge in Quarter 2—just before the start of the first fol owup year. It is possible that some tenants began to respond to the new policy soon after their new TTPs took effect, which, for most families, occurred early in Quarter 2, 45 or soon after first learning about the new rules at their initial recertification meetings with housing specialists before the effective dates. For example, it was at those meetings that they learned, not only from a video they watched but also from their discussions with staff and from writ en materials, that their TTPs would not rise if they increased their earnings, and that the minimum TTP would be increased.

The results for Louisvil e and Washington, D.C., were less positive. In fact, in Louisvil e, the effects on earnings are negative early in Year 2. This is not because the earnings of those in the new rules group fell over time; indeed, the earnings trends over time are positive for both groups, but those trends are less positive for the new rules group than for the control group. As exhibit 3.5 shows, this trend resulted in a statistical y significant negative impact on earnings in Quarter 7; however, that negative effect attenuated and was no longer statistically significant by Quarter 8. Employment rates for both groups throughout the 18 months of fol owup were similar. Longer-term followup data, when they become available, wil show whether the negative effect on earnings in Quarter 7 is part of a longer-term pattern or is an aberration.

In Louisvil e, because 22 percent of families opted out of the new rent policy, the estimated impacts shown in exhibit 3.5, which are averaged over al heads of households, including those from the opt-out families, may be understated. Therefore, as explained in Chapter 2, treatment-on-treated (TOT) adjustments were made, which attribute all effects to only those individuals who were exposed 45 About 90 percent of families in San Antonio had effective dates within 4 months of random assignment.

42

to the policy; however, the TOT adjustments do not alter statistical significance levels. 46 Exhibit C.1

presents the results of the TOT analysis. As it shows, TOT impact on the average quarterly employment rate in Year 1 is slightly larger than the original intent-to-treat (ITT) impact estimate, 47

but it remains less than 2 percentage points and statistically insignificant. The TOT impact on average earnings in each quarter also remains statistically insignificant except in Quarter 7, where it grows to a negative $325.

46 The treatment-on-treated analysis adjusts the impact estimate to account for the fact that some members of the new rent rules group were not exposed to the new rent rules. A TOT analysis does not change the statistical significance of impact estimates from what they are in an intent-to-treat (ITT) analysis, so no new statistical tests were run.

47 An ITT analysis captures the average impact on the entire group intended to receive the intervention, whether or not every member of that group actually received it.

43

Exhibit 3.5. Early Impacts on Employment and Earnings, by Public Housing Agency (PHA): Heads of Households

New

Existing Difference

Outcomes

Rent Rules Rent Rules

(Impact)

P-Value

Lexington

Ever employed (%)

Year 1 (quarters 3–6)

78.5

75.9

2.6

0.226

Quarter 1 (random assignment)

62.4

59.8

2.6

0.242

Quarter 2

65.8

62.7

3.1

0.181

Quarter 3

65.4

65.7

-0.2

0.928

Quarter 4

65.5

64.5

1.0

0.688

Quarter 5

65.8

63.3

2.6

0.313

Quarter 6

66.0

62.5

3.5

0.191

Quarter 7

65.7

63.7

2.0

0.458

Quarter 8

65.2

61.5

3.8

0.154

Employed in all quarters, Year 1 (%)

51.0

48.8

2.3

0.401

Average quarterly employment, Year 1a (%)

65.7

64.0

1.7

0.395

Total earnings ($)

Year 1 (quarters 3–6)

10,387

9,921

466

0.315

Quarter 1 (random assignment)

2,096

1,881

215 **

0.014

Quarter 2

2,360

2,312

48

0.673

Quarter 3

2,544

2,560

-15

0.907

Quarter 4

2,446

2,389

56

0.661

Quarter 5

2,602

2,454

148

0.289

Quarter 6

2,809

2,582

227

0.158

Quarter 7

2,881

2,583

298 *

0.065

Quarter 8

2,869

2,517

352 **

0.028

Sample size (total = 979)

486

493

Louisville

Ever employed (%)

Year 1 (quarters 3–6)

71.8

72.1

-0.3

0.841

Quarter 1 (random assignment)

52.8

55.1

-2.3

0.133

Quarter 2

57.8

58.8

-1.0

0.548

Quarter 3

60.1

59.9

0.2

0.914

Quarter 4

59.7

58.7

1.0

0.582

Quarter 5

59.8

59.3

0.5

0.792

Quarter 6

63.2

61.3

1.9

0.327

Quarter 7

61.0

62.1

-1.2

0.540

Quarter 8

60.7

62.1

-1.4

0.474

Employed in all quarters 3–6 (%)

48.0

43.8

4.3 **

0.024

Average quarterly employment, Year 1a (%)

60.7

59.8

0.9

0.544

(continued)

44

Exhibit 3.5 (continued)

New

Existing

Difference

Outcomes

Rent Rules

Rent Rules

(Impact)

P-Value

Total earnings ($)

Year 1 (quarters 3–6)

10,113

10,083

29

0.936

Quarter 1 (random assignment)

1,931

2,027

-96

0.164

Quarter 2

2,186

2,154

32

0.710

Quarter 3

2,423

2,392

31

0.763

Quarter 4

2,460

2,469

-8

0.939

Quarter 5

2,499

2,440

59

0.590

Quarter 6

2,733

2,800

-67

0.581

Quarter 7

2,775

3,027

-252 **

0.047

Quarter 8

2,822

3,003

-181

0.181

Sample size (total = 1,908)

947

961

San Antonio

Ever employed (%)

Year 1 (quarters 3–6)

71.1

67.8

3.4 *

0.052

Quarter 1 (random assignment)

56.1

56.4

-0.3

0.808

Quarter 2

59.9

56.1

3.8 **

0.023

Quarter 3

59.9

56.3

3.6 **

0.046

Quarter 4

59.2

57.2

2.1

0.263

Quarter 5

61.7

57.5

4.2 **

0.024

Quarter 6

61.3

58.2

3.1

0.107

Quarter 7

61.4

58.6

2.9

0.147

Quarter 8

60.4

56.6

3.8 *

0.056

Employed in all quarters, Year 1 (%)

48.3

45.5

2.8

0.128

Average quarterly employment, Year 1a (%)

60.5

57.3

3.2 **

0.037

Total earnings ($)

Year 1 (quarters 3–6)

10,003

9,086

916 ***

0.010

Quarter 1 (random assignment)

2,172

2,070

102

0.106

Quarter 2

2,333

2,138

195 **

0.016

Quarter 3

2,384

2,174

210 **

0.021

Quarter 4

2,455

2,320

135

0.171

Quarter 5

2,538

2,271

267 ***

0.010

Quarter 6

2,686

2,394

292 **

0.011

Quarter 7

2,704

2,379

324 ***

0.007

Quarter 8

2,786

2,395

392 ***

0.002

Sample size (total = 1,869)

935

934

Washington, D.C.

Ever employed (%)

Year 1 (quarters 3–6)

56.4

55.6

0.8

0.637

Quarter 1 (random assignment)

44.8

42.7

2.1

0.138

Quarter 2

37.1

38.9

-1.8

0.328

(continued)

45

Exhibit 3.5 (continued)

New

Existing Difference

Outcomes

Rent Rules Rent Rules

(Impact)

P-Value

Quarter 3

48.2

48.1

0.1

0.942

Quarter 4

38.4

40.0

-1.6

0.405

Quarter 5

40.1

41.1

-0.9

0.635

Quarter 6

40.2

37.2

3.0

0.121

Quarter 7

51.1

49.1

1.9

0.311

Quarter 8

48.7

48.4

0.3

0.863

Employed in all quarters, Year 1 (%)

23.3

23.2

0.1

0.976

Average quarterly employment, Year 1a

(%)

41.7

41.6

0.2

0.899

Total earnings ($)

Year 1 (quarters 3–6)

10,266

10,634

-368

0.379

Quarter 1 (random assignment)

2,731

2,546

185 **

0.037

Quarter 2

2,053

2,198

-144

0.248

Quarter 3

3,069

3,219

-150

0.213

Quarter 4

2,219

2,410

-191

0.139

Quarter 5

2,317

2,448

-132

0.361

Quarter 6

2,669

2,579

90

0.560

Quarter 7

3,387

3,315

72

0.622

Quarter 8

3,264

3,264

0

0.999

Sample size (total = 1,909)

944

965

aAverage quarterly employment is calculated as total number of quarters with employment divided by total number of quarters of followup, expressed as a percentage.

* = 10 percent. ** = 5 percent. *** = 1 percent.

Notes: Estimates were regression-adjusted using ordinary least squares, controlling for pre-random assignment characteristics of sample members. Rounding may cause slight discrepancies in calculating sums and differences. A two-tailed t-test was applied to differences between research groups. The p-value indicates the likelihood that the difference between the new rent rules group and the existing rent rules group arose by chance.

Sample sizes for specific outcomes may vary because of missing values. The variation across the four PHAs in estimated impacts on total earnings in Year 1 is statistically significant at the 10 percent level based on an H-statistic test.

Source: MDRC calculations, using quarterly wage data from the National Directory of New Hires In Washington, D.C., few differences in employment and earnings outcomes are evident during the fol owup period. The small negative earnings effect estimated for Year 1 is not statistically significant. In interpreting these results, it is important to understand that these short-term findings do not speak to the effects of extending the recertification period under the new rent policy. Because the control group is subject to biennial recertifications, it has not yet faced the work disincentive believed to be associated with the 30-percent-of-income rent rule, and, therefore, the new rent rules group has not yet experienced a more favorable work incentive from the triennial recertification feature of the new rent policy. It might be hypothesized that just knowing that the recertification period is a year longer might have increased work effort for the new rules group. Stil , that is a somewhat distant 46

advantage and is probably less compelling than an actual differential incentive that begins sooner.

Moreover, because the eventual difference in recertification periods wil be only 1 year longer for the new rules group than for the existing rules group, it is reasonable to expect that, if delaying recertifications has positive effects on tenants’ employment and earnings, those effects may be smaller in Washington, D.C., than might be the case if the control group were subject to annual recertifications.

More relevant for the Year 1 analysis in Washington, D.C., is the PHA’s new minimum TTP.

Unlike the control group, the new rules group is subject to a $75 minimum TTP. This feature may serve as an inducement to increase work effort because it represents a new obligation for families regardless of their income level (although a time-limited hardship exemption is available). The absence of positive short-term impacts of the new rent policy on employment and earnings in this PHA suggests that the $75 minimum TTP did not produce better employment or earnings outcomes in Year 1.

The patterns of employment and earnings trends for heads of household in Washington, D.C., for both research groups, also deserve comment. The quarter-to-quarter variability, especially for earnings, is much more pronounced than in any of the other PHAs. As exhibit 3.4 il ustrates, the trend lines in Washington, D.C., show striking peaks and troughs. The peaks occur in the third and seventh quarters relative to families’ random assignment dates (see also exhibit 3.5). As it turns out, this pattern reflects a seasonality phenomenon, resulting from the fact that the entire sample in this PHA was randomly assigned within a single calendar quarter (April to June 2015). 48 That variability does not affect the accuracy of the impact estimates.

Subgroup Results

It is possible that different types of voucher holders will respond differently to the new rent policy’s financial incentives that reward work. For example, even if inspired by the new policy to work or earn more, some adults may have greater difficulty doing so because of certain disadvantages, such as low education and skil levels, personal and family problems, childcare problems, transportation problems, health issues, or other work impediments. Others may seek and achieve employment outcomes even without the added inducement of a more favorable rent policy. For such tenants, the new rent policy may have lit le effect. In contrast, other tenants—who have been discouraged from trying to work or to increase their earnings because they are concerned that much of their earnings gains wil be offset 48Random assignment occurred in what is referred to in this analysis as relative Quarter 1, which is the second calendar quarter of 2015. Relative Quarter 3 for the full Washington, D.C, sample thus falls in the fourth calendar quarter (October through December) of 2015, and Quarter 7 falls in the fourth calendar quarter of 2016. Nationally, the fourth calendar quarter is a time when employment and hours of work increase as the holiday season unfolds, after which employment and hours tend to decline. This appears to be the case for the Washington, D.C., sample. The trends in the other three PHAs are smoother because random assignment occurred over several calendar quarters. Indeed, in those PHAs, when impacts on earnings are estimated just for household heads who were randomly assigned at the same time as the Washington, D.C. sample (that is, April through June 2015), a similar pattern of peaks and troughs is evident, as can be seen from the control group patterns presented in exhibit C.1.

47

by reductions in their housing subsidies—may respond well to a policy that addresses that disincentive.

This report examines differential responses to the new rent policy primarily for subgroups of voucher holders as defined by their employment status in the quarter before random assignment (using NDNH data) and by the age of the youngest child in the household at the time of random assignment (using PHA data). 49 Other studies of workforce interventions for voucher holders, including the Family Self-Sufficiency program, have shown that the degree of prior employment is often a good predictor of the likelihood of future employment and earnings. Moreover, a number of studies have found that impacts on future employment and earnings are larger for individuals with less prior employment as it is often easier for programs to help individuals who are not employed to get jobs than it is to help those who are already working to increase their earnings or advance to higher wage jobs (Hendra, et al., 2011; Michalopoulos, 2005; and Nuñez, Verma, and Yang, 2015). In the Rent Reform Demonstration sample, about 46 percent of the heads of household (for all PHAs combined) were not working in the quarter prior to random assignment.

It is also commonly expected that low-income parents with young children have greater difficulty working at al , or working full time, because of the difficulty they have in finding affordable childcare. Moreover, concerns about leaving older children and teenagers unsupervised after school may discourage parents with older children from working or working full time. In the Rent Reform Demonstration sample, about 77 percent of household heads had children who were under the age of 18, and 28 percent had a child 5 years of age or younger at the time of random assignment.

Exhibit 3.6 shows the early impacts of the new rent policy for household heads according to their employment status in the quarter prior to random assignment. In reviewing these results, it is useful first to consider the variation in the control group’s outcomes according to their pre-random assignment employment status. The table shows stark differences in labor market experiences among those who were working and those who were not employed at that time. For example, the average quarterly employment rate during Year 1 was only 24.5 percent for household heads in the existing rules group who were not already working; in contrast, it was 80.7 percent for those who were already working. Their average earnings for Year 1 were $3,003 and $15,994, respectively (those averages include zeroes for individuals who had no earnings.) Thus, household heads in the control group who were not already working remained substantially disconnected from the labor force during the early followup period. 50

For al PHAs combined, as exhibit 3.6 shows, the outcomes for the new rules group are not substantially different from those of the control group when examined separately, according to 49Results for other subgroups were also explored, including subgroups defined in terms of the number of children and the combination of single parenthood and employment status at baseline. No distinctive patterns are evident in Year 1. The evaluation will continue to examine effects for all subgroups as longer term data become available.

50 A similar pattern has been observed in other studies of voucher holders. For example, in MDRC’s evaluation of New York City’s Family Self-Sufficiency program, adults in control group families receiving housing choice vouchers who were not working at the time of random assignment had an average quarterly employment rate of about 25 percent over the 6-year followup period, compared with 62 percent among those who were already employed at that time (Verma et al., 2017).

48

tenants’ pre-random assignment employment status, and few differences are statistically significant.

Thus, the effects of the new rent policy on labor market outcomes—so far, for the pooled sample including al four PHAs—are not clearly and consistently stronger or weaker for household heads who were not already working compared with those who were already working.

The results for each PHA are more nuanced, as exhibit 3.7 shows. For example, in San Antonio, the impacts on average quarterly employment are substantial and statistically significant for the nonemployed subgroup (6.2 percentage points) in Year 1, but not for the employed subgroup (0.5

percentage points). The same is not the case for earnings. Positive earnings effects appear to be more concentrated in the already-employed subgroup; however, the differences in employment and earnings impacts across the two subgroup categories are not statistically significant, suggesting uncertainty as to whether the policy’s effects truly vary by subgroup. In Lexington, similarly, the impacts on employment appear larger for the nonemployed subgroup, whereas the estimated effects on earnings are more positive for the already-employed subgroup and are statistically significant in Quarters 7 and 8; however, the differences in impact estimates are not themselves statistically significant across the subgroup categories. In Louisvil e and Washington, D.C., few impacts are evident for either subgroup category. Overall, although the emerging patterns in both Lexington and San Antonio are suggestive, no clear and consistent evidence exists at this time indicating that the new rent rules have differential early effects on tenants’ labor market outcomes according to their initial employment status. Future reports wil show whether sharper and more consistently statistically significant patterns emerge when longer-term fol owup data are examined.

49

Exhibit 3.6. Early Impacts on Employment and Earnings, by Employment Status in the Quarter Before Random Assignment: Heads of Households

New

Existing Difference

Outcomes

Rent Rules Rent Rules

(Impact)

P-Value

Lexington—Not Employed

Ever employed, Year 1 (%)

53.6

47.8

5.9

0.192

Average quarterly employment, Year

1a (%)

36.8

33.9

3.0

0.417

Total earnings ($)

Year 1 (quarters 3–6)

3,881

3,739

142

0.822

Quarter 7

1,291

1,225

66

0.773

Quarter 8

1,306

1,134

172

0.434

Sample size (total = 416)

209

207

Lexington—Employed

Ever employed, Year 1 (%)

96.4

97.2

-0.7

0.639

Average quarterly employment, Year

1a (%)

86.7

86.5

0.2

0.942

Total earnings ($)

Year 1 (quarters 3–6)

15,204

14,500

704

0.286

Quarter 7

4,045

3,601

444 **

0.048

Quarter 8

3,998

3,566

433 *

0.055

Sample size (total = 563)

277

286

Louisville—Not Employed

Ever employed, Year 1 (%)

43.7

46.1

-2.4

0.454

Average quarterly employment, Year

1a (%)

30.3

31.9

-1.7

0.518

Total earnings ($)

Year 1 (quarters 3–6)

3,524

3,707

-184

0.677

Quarter 7

1,024

1,318

-294 *

0.055

Quarter 8

1,162

1,344

-182

0.269

Sample size (total = 854)

404

450

Louisville—Employed

Ever employed, Year 1 (%)

94.6

93.2

1.4

0.338

Average quarterly employment, Year

1a (%)

85.5

82.2

3.3 *

0.063

Total earnings ($)

Year 1 (quarters 3–6)

15,534

15,205

329

0.554

Quarter 7

4,210

4,414

-204

0.293

Quarter 8

4,190

4,336

-146

0.478

Sample size (total = 1,050)

541

509

(continued)

50

Exhibit 3.6 (continued)

New

Existing Difference

Outcomes

Rent Rules Rent Rules

(Impact)

P-Value

San Antonio—Not Employed

Ever employed, Year 1 (%)

42.2

35.9

6.2 *

0.061

Average quarterly employment, Year 1a

(%)

29.8

23.9

5.9 **

0.023

Total earnings ($)

Year 1 (quarters 3–6)

3,171

2,524

647

0.104

Quarter 7

1,066

815

251 *

0.067

Quarter 8

1,111

906

206

0.157

Sample size (total = 815)

418

397

San Antonio—Employed

Ever employed, Year 1 (%)

93.2

92.7

0.5

0.775

Average quarterly employment, Year 1a

(%)

84.1

83.3

0.8

0.675

Total earnings ($)

Year 1 (quarters 3–6)

15,308

14,185

1,123 **

0.042

Quarter 7

3,971

3,600

371 **

0.047

Quarter 8

4,074

3,566

508 ***

0.009

Sample size (total = 1,053)

517

536

Washington, D.C.—Not Employed

Ever employed, Year 1 (%)

25.9

24.6

1.3

0.653

Average quarterly employment, Year 1a

(%)

14.7

14.8

-0.1

0.963

Total earnings ($)

Year 1 (quarters 3–6)

2,155

2,461

-306

0.479

Quarter 7

936

1,028

-91

0.577

Quarter 8

891

948

-57

0.728

Sample size (total = 976)

488

488

Washington, D.C.—Employed

Ever employed, Year 1 (%)

92.4

90.6

1.7

0.354

Average quarterly employment, Year 1a

(%)

73.4

72.2

1.3

0.554

Total earnings ($)

Year 1 (quarters 3–6)

19,735

20,136

-402

0.587

Quarter 7

6,233

5,953

280

0.267

Quarter 8

6,036

5,891

144

0.575

Sample size (total = 861)

431

430

(continued)

51

Exhibit 3.6 (continued)

aAverage quarterly employment is calculated as total number of quarters with employment divided by total number of quarters of followup, expressed as a percentage.

* = 10 percent. ** = 5 percent. *** = 1 percent.

Notes: Estimates were regression-adjusted using ordinary least squares, controlling for pre-random assignment characteristics of sample members. Rounding may cause slight discrepancies in calculating sums and differences. A two-tailed t-test was applied to differences between research groups. The p-value indicates the likelihood that the difference between the new rent rules group and the existing rent rules group arose by chance.

Sample sizes for specific outcomes may vary because of missing values.

Source: MDRC calculations, using quarterly wage data from the National Directory of New Hires

52

Exhibit 3.7. Early Impacts on Employment and Earnings, by Employment Status at Random Assignment and Public Housing Agency (PHA): Heads of

Households

New

Existing Difference

Outcomes

Rent Rules Rent Rules

(Impact)

P-Value

Lexington—Not Employed

Ever employed, Year 1 (%)

53.6

47.8

5.9

0.192

Average quarterly employment, Year 1a (%)

36.8

33.9

3.0

0.417

Total earnings ($)

Year 1 (quarters 3–6)

3,881

3,739

142

0.822

Quarter 7

1,291

1,225

66

0.773

Quarter 8

1,306

1,134

172

0.434

Sample size (total = 416)

209

207

Lexington—Employed

Ever employed, Year 1 (%)

96.4

97.2

-0.7

0.639

Average quarterly employment, Year 1a (%)

86.7

86.5

0.2

0.942

Total earnings ($)

Year 1 (quarters 3–6)

15,204

14,500

704

0.286

Quarter 7

4,045

3,601

444 **

0.048

Quarter 8

3,998

3,566

433 *

0.055

Sample size (total = 563)

277

286

Louisville—Not Employed

Ever employed, Year 1 (%)

43.7

46.1

-2.4

0.454

Average quarterly employment, Year 1a (%)

30.3

31.9

-1.7

0.518

Total earnings ($)

Year 1 (quarters 3–6)

3,524

3,707

-184

0.677

Quarter 7

1,024

1,318

-294 *

0.055

Quarter 8

1,162

1,344

-182

0.269

Sample size (total = 854)

404

450

Louisville—Employed

Ever employed, Year 1 (%)

94.6

93.2

1.4

0.338

Average quarterly employment, Year 1a (%)

85.5

82.2

3.3 *

0.063

Total earnings ($)

Year 1 (quarters 3–6)

15,534

15,205

329

0.554

Quarter 7

4,210

4,414

-204

0.293

Quarter 8

4,190

4,336

-146

0.478

Sample size (total = 1,050)

541

509

(continued)

53

Exhibit 3.7 (continued)

New

Existing Difference

Outcomes

Rent Rules Rent Rules

(Impact)

P-Value

San Antonio—Not Employed

Ever employed, Year 1 (%)

42.2

35.9

6.2 *

0.061

Average quarterly employment, Year 1a (%)

29.8

23.9

5.9 **

0.023

Total earnings ($)

Year 1 (quarters 3–6)

3,171

2,524

647

0.104

Quarter 7

1,066

815

251 *

0.067

Quarter 8

1,111

906

206

0.157

Sample size (total = 815)

418

397

San Antonio—Employed

Ever employed, Year 1 (%)

93.2

92.7

0.5

0.775

Average quarterly employment, Year 1a (%)

84.1

83.3

0.8

0.675

Total earnings ($)

Year 1 (quarters 3–6)

15,308

14,185

1,123 **

0.042

Quarter 7

3,971

3,600

371 **

0.047

Quarter 8

4,074

3,566

508 ***

0.009

Sample size (total = 1,053)

517

536

Washington, D.C.—Not Employed

Ever employed, Year 1 (%)

25.9

24.6

1.3

0.653

Average quarterly employment, Year 1a (%)

14.7

14.8

-0.1

0.963

Total earnings ($)

Year 1 (quarters 3–6)

2,155

2,461

-306

0.479

Quarter 7

936

1,028

-91

0.577

Quarter 8

891

948

-57

0.728

Sample size (total = 976)

488

488

Washington, D.C.—Employed

Ever employed, Year 1 (%)

92.4

90.6

1.7

0.354

Average quarterly employment, Year 1a (%)

73.4

72.2

1.3

0.554

Total earnings ($)

Year 1 (quarters 3–6)

19,735

20,136

-402

0.587

Quarter 7

6,233

5,953

280

0.267

Quarter 8

6,036

5,891

144

0.575

Sample size (total = 861)

431

430

(continued)

54

Exhibit 3.7 (continued)

aAverage quarterly employment is calculated as total number of quarters with employment divided by total number of quarters of followup, expressed as a percentage.

* = 10 percent. ** = 5 percent. *** = 1 percent.

Notes: Estimates were regression-adjusted using ordinary least squares, controlling for pre-random assignment characteristics of sample members. Rounding may cause slight discrepancies in calculating sums and differences. A two-tailed t-test was applied to differences between research groups. The p-value indicates the likelihood that the difference between the new rent rules group and the existing rent rules group arose by chance. Sample sizes for specific outcomes may vary because of missing values.

Source: MDRC calculations, using quarterly wage data from the National Directory of New Hires

To explore whether the early effects of the new rent policy differed for heads of household depending on the presence of children in the household at the time of random assignment, families were divided into four groups: families had no children under the age of 18 years, youngest child was 5 years of age or younger, youngest child was 6 to 12 years of age, or youngest child was 13 to 17

years of age. One hypothesis is that families with very young children may have more difficulty responding to the stronger financial work incentives embedded in the new rent policy, in part because of childcare issues.

Interestingly, using the pooled sample with all four PHAs and looking first at outcomes for household heads in the control group, it appears that attachment to the labor force does not vary greatly according to the age of the youngest child. For example, as exhibit 3.8 shows, control group parents whose youngest child was 5 years of age or younger had an average quarterly employment rate of 53.1 percent in Year 1, which is only modestly lower than the 56.4-percent rate for those with teenage children and somewhat higher than the rate for those with no children (51.4 percent); however, their average Year 1 earnings were lower.

When the impact findings are compared across these four subgroup categories, statistically significant earnings gains are evident for parents whose youngest child was a teenager (13 to 17 years of age) but not for those with younger children or no children. For example, among the parents of teenagers, earnings for those in the new rent rules group increased by a statistically significant $929

(or 9 percent) in Year 1 relative to similar parents in the control group. At the same time, no impacts on employment are evident for those parents of teenagers. It may be that among parents of teenagers, those who responded to the new rent policy would have worked anyway, but, because of the policy change, they increased the number of hours they worked in an average quarter.

55

Exhibit 3.8. Early Impacts on Employment and Earnings, by Age of Youngest Child in the Household at the Time of Random Assignment: Heads

of Households

New

Existing Difference

Outcomes

Rent Rules Rent Rules

(Impact)

P-Value

No children under age 18 years

Ever employed, Year 1 (%)

64.2

62.4

1.8

0.340

Average quarterly employment, Year 1a

(%)

53.3

51.4

1.9

0.253

Total earnings ($)

Year 1 (quarters 3–6)

9,824

10,024

-200

0.633

Quarter 7

2,720

2,908

-188

0.190 †

Quarter 8

2,677

2,842

-165

0.271

Sample size (total =1,517)

741

776

Children ages 0-5 years

Ever employed, Year 1 (%)

67.8

67.0

0.8

0.658

Average quarterly employment, Year 1a

(%)

53.2

53.1

0.1

0.971

Total earnings ($)

Year 1 (quarters 3–6)

8,771

8,782

-11

0.975

Quarter 7

2,594

2,553

40

0.735 †

Quarter 8

2,607

2,499

108

0.377

Sample size (total = 1,866)

910

956

Children ages 6-12 years

Ever employed, Year 1 (%)

71.9

68.2

3.7 **

0.020

Average quarterly employment, Year 1a

(%)

60.0

56.7

3.3 **

0.017

Total earnings ($)

Year 1 (quarters 3–6)

10,809

10,311

499

0.166

Quarter 7

3,144

3,005

139

0.271 †

Quarter 8

3,138

3,005

133

0.311

Sample size (total = 2,154)

1,076

1,078

(continued)

56

Exhibit 3.8 (continued)

New

Existing Difference

Outcomes

Rent Rules Rent Rules

(Impact)

P-Value

Children ages 13-17 years

Ever employed, Year 1 (%)

67.2

69.7

-2.5

0.244

Average quarterly employment, Year 1a (%)

56.4

56.4

0.0

0.997

Total earnings ($)

Year 1 (quarters 3–6)

11,810

10,881

929 *

0.065

Quarter 7

3,425

3,056

369 **

0.030 †

Quarter 8

3,467

3,080

387 **

0.027

Sample size (total = 1,128)

585

543

aAverage quarterly employment is calculated as total number of quarters with employment divided by total number of quarters of followup, expressed as a percentage.

* = 10 percent. ** = 5 percent. *** = 1 percent.

Notes: Estimates were regression-adjusted using ordinary least squares, controlling for pre-random assignment characteristics of sample members. Rounding may cause slight discrepancies in calculating sums and differences. A two-tailed t-test was applied to differences between research groups. The p-value indicates the likelihood that the difference between the new rent rules group and the existing rent rules group arose by chance.

Sample sizes for specific outcomes may vary because of missing values. The H-statistic test was used to test for statistically significant differences in impact estimates across different subgroups. Statistical significance levels are indicated as: † = 10 percent; †† = 5 percent; and ††† = 1 percent.

Source: MDRC calculations, using quarterly wage data from the National Directory of New Hires The dif erences in earnings impacts across subgroup categories are not statistically significant, however, except in Quarter 7. Consequently, although suggestive, it is not certain whether the earnings impact is truly larger for parents of teenagers than for other household heads (a subgroup analysis that focused on the age of the youngest child was not conducted for each PHA separately owing to the small sample sizes of these subgroup categories within each site).

Impacts for Other Adults and All Adults in the Household

Approximately 37 percent of the study’s households included adults who were not heads of household. As mentioned previously, those 3,397 “other adults” were primarily the young adult children of the household heads at the time of random assignment. Few were spouses or partners of the household heads, and about 21 percent were no longer on the household’s lease a year after the initial recertification. Appendix C presents the findings on these adults, as well as on all adults combined.

Exhibit C.2 shows the overal pooled results for the non-heads of household within the first 18 months of followup. Within the control group, employment rates for this group were roughly comparable with those of the heads of household, with more than two-thirds having worked some time in a UI-covered job in the first year, and more than one-half working in an average quarter, although their average earnings were somewhat lower. That table also shows that the new rent rules produced no impacts on employment for the non-heads of households. Average earnings are 57

somewhat lower for those in the new rules group than in the existing rules group, but the differences are not statistically significant.

This pattern of negative but not statistically significant earnings effects is evident in three of the four PHAs (see exhibit C.3). Lexington is the exception. It produced a sizable positive and statistically significant effect on earnings for the non-heads of household, just as it did for household heads.

Additional analyses combine the heads of household with other adults in the households at the time of random assignment. The results generally tell a story that is consistent with the pattern of results found for the heads of household (see exhibits C.4 and C.5).

Conclusion

This chapter has shown that the new rent policy, based on the experience of two PHAs, one in Lexington and one in San Antonio, can lead voucher holders to increase their earnings modestly in the short term. Whether those encouraging early effects persist and even grow in the longer term—and emerge over time in the two other PHAs—remains to be seen, and although the research design does not permit a rigorous determination of the relative effects of the different components of the new rent policy, the pattern of findings suggests at least two early insights. First, the early positive effects in Lexington, where both research groups were subject to a $150 minimum TTP, can be ful y attributed to other features of the policy—most likely the extension of the income recertification period from 1

year to 3 years. Any effects that Lexington’s high minimum TTP may have independently had on tenants’ employment and earnings outcomes cannot be determined by this study (because both research groups were subject to that same feature). In contrast, in Washington, D.C., the study is not yet able to assess the effects of extending the recertification period, given the control group’s biennial recertification schedule. This means that the introduction of a $75 minimum TTP under the new rent policy is perhaps the most important distinction between the two research groups during the early followup period. The finding of no early earning impacts in Washington, D.C., indicates that this component of the policy was not enough to change tenants’ labor market outcomes at that PHA; however, it did have some consequences for families’ housing-related outcomes, as discussed in the next chapter.

58

Chapter 4

Early Impacts on Housing-Related Outcomes

The new rent rules substantially change the way in which families’ contributions to their rent and utilities (called their “total tenant payments,” or TTPs) are determined and the way their TTPs are determined. The new rules also modify the requirements for reporting income changes and adjusting TTPs over time. The baseline report on the Rent Reform Demonstration describes the consequences of those changes for families’ TTPs and housing subsidies at the time of the initial recertification when the study began (Riccio, Deitch, and Verma, 2017). 51 This chapter looks at these and other outcomes 1 year later. It examines the effects of the new rules on families’ TTPs, subsidies, and the likelihood of exiting the voucher program within the first year and on families’ transactions with the public housing agencies (PHAs)—that is, the actions or tasks that PHA staff had to execute for families.

Overall, relative to the outcomes of the existing rent rules (control) group, the new rent policy modestly increased family members’ likelihood of remaining in the Housing Choice Voucher (HCV) program within the first year of fol owup, reduced their TTPs, and increased the amount of subsidy they received. These findings reflect the expected short-term consequences of the policy’s efforts to reward work by allowing tenants who increase their earnings to keep more of those earnings until their TTPs and subsidies are reset at the triennial recertification. The early findings also reveal substantial reductions in PHA staff actions, particularly those that are likely to be more time consuming, such as regular and interim recertifications. Some important differences in impacts on staff actions exist across the PHAs, in part reflecting differences in preexisting PHA policies that affected the control group.

Calculating Families’ Contributions to Their Housing Costs

Under traditional rent rules, the recertification process entails reassessing a family’s continued eligibility for the voucher program, recalculating its expected contribution to its rent and utilities, and redetermining its housing subsidy. This process typically begins several months before the 1-year anniversary of the family’s soon-to-be-expiring TTP. PHA housing specialists col ect and verify the information that families submit on their current income and the income they anticipate having in the upcoming year and on changes in household composition or other pertinent circumstances. The housing specialists enter the data into the rent-calculation software system, have the system estimate the TTPs, and notify families 30 days before their new rent

“effective dates”—that is, the dates when their new TTPs go into effect. 52 These recertification-related activities take different amounts of time at different PHAs, from about 90 days in Lexington to 180 days in Washington, D.C.

51 For a summary, see Chapter 1 of the current report.

52For the initial recertification under the study, the PHA in Louisville included an additional 30-day period to allow families the option of opting out of the new rent policy.

59

Under the new rent policy, that process was modified. As explained in Chapter 1 (and more ful y in the study’s baseline report (Riccio, Deitch, and Verma, 2017), families who were assigned to the new rent rules group were required to document the income they had received from jobs or other sources during a defined 12-month period leading up to their initial recertification meetings. That information was used to calculate the families’ retrospective incomes, which was necessary to determine their TTPs. The retrospective, or 12-month look-back period, ended the month before the family’s recertification. For example, if a family was scheduled for a recertification meeting on February 21, 2015, the 12-month period used to determine retrospective income was February 1, 2014, through January 31, 2015. 53

The Rent Reform Demonstration did not change the rules about the types of income that were or were not counted in calculating TTPs and subsidies. 54 Families were required to make a good-faith effort to provide proof of countable income for the requested period. When families were unable to provide appropriate income documentation, or when the PHAs were unable to verify past income using their standard methods, 55 the PHAs fol owed agreed-upon procedures to impute gaps in reported household income. The MDRC study team and the PHAs anticipated scenarios in which families would struggle to obtain the required income documents—for example, pay stubs from early in the retrospective period—and developed rules and guidance for staff members to use in such situations.

After computing a family’s expected TTP, the PHA pays the difference between the family’s gross rent (that is, the contract rent charged by the landlord plus utilities that are not included in the contract rent) and the family’s TTP, so long as the gross rent is no higher than the PHA’s payment standard set for the local area. This subsidy is referred to as the housing assistance payment (HAP). If the landlord charges a rent that exceeds the payment standard, the family is responsible for that extra amount in addition to its TTP. 56 The TTP plus that extra amount make up the family’s total housing cost, which HUD calls the “family share” of rent and utilities. Exhibit 4.1 offers a simple il ustration of these concepts in the case of Paige, a fictional voucher holder.

53 For a fuller discussion of estimating retrospective incomes, see Riccio, Deitch, and Verma (2017).

54 However, as mentioned in Chapter 1, nonwage income that was set to expire by the end of the look-back period, however, such as Temporary Assistance for Needy Families or unemployment insurance benefits, was not counted when calculating base income because a family would not be able to count on such income going forward.

55 Retrospective income was verified using the HUD Verification Hierarchy and the guidance provided in HUD Notice PIH 2010-19 (HA) and the PHA Administrative Plan.

56 Voucher holders are allowed to rent units for which the contract rent exceeds the payment standard as long as those units do not require them to pay more than 40 percent of their incomes toward rent and utilities when they sign the lease. Under HUD’s traditional rent rules, that 40 percent means 40 percent of their current/anticipated adjusted incomes. Under the new rent rules, it is 40 percent of their current/anticipated gross incomes.

60

Exhibit 4.1 Total Tenant Payment and Family Share

Total tenant payment (TTP) is the amount a family must contribute toward its rent and utilities. TTP is based on 28 percent of gross income for families in the new rent rules group of the Rent Reform Demonstration.

Housing assistance payment (HAP) is the housing subsidy (for rent and utilities) paid by the housing agency.

Family share includes the TTP and any extra housing costs above the payment standard, paid by the family.

Payment standard is the maximum combined rent and utilities subsidy that a PHA will pay for families of given sizes, specific to each area and its fair-market rent. If a landlord charges a rent that exceeds the payment standard, the family is responsible for that extra amount in addition to its TTP.

Example: Paige is renting a housing unit that has a $1,150 contract rent. The payment standard for her housing subsidy is $1,100. She is responsible for paying a total of $200 (the family share), which includes her TTP of $150 (based on 28 percent of her income of $536 per month) and an additional $50, the amount by which the contract rent exceeds the payment standard. Thus, her rent is subsidized by $950 ($1,150 contract rent minus $200 family share).

1,300

1,200

Amount above

payment standard

1,100

Payment standard

50

Family share

TTP

150

1,000

900

800

)

700

rent ($

600

ntractCo

500

Housing subsidy

400

300

200

100

0

61

Early Impacts on Families’ Housing Expenditures and Subsidies

As explained in Chapter 2, this report defines “Year 1” for the analysis of PHA data as the 12-month period beginning in the first month after the month in which a family’s newly recalculated TTP was expected to take effect (the “effective date”) after entering the study.

Depending on a family’s initial expected TTP effective date (which occurred sometime between June 2015 and March 2016), Year 1 ended between June 2016 and March 2017. Although this first year represents only one-third of the time until the next required recertification takes place for the new rent rules group, the analysis shows that the new rent policy has begun to influence their housing expenditures and subsidies, as well as their transactions with PHA staff. As discussed in Chapter 2 and appendix B, the pooled impact estimate for average housing subsidy payments in Year 1 represents a preliminary confirmatory outcome measure for the evaluation.

Impacts for Al PHAs Combined

One early finding on families’ housing-related outcomes is that the new rent rules increased the likelihood of remaining in the voucher program during the first year of fol owup. The top panel of exhibit 4.2 shows that, for al PHAs combined, 88.2 percent of the existing rules group (the control group) were stil in the voucher program and leased up at the end of Year 1. 57 In contrast, 92.4 percent of the new rent rules group remained in the voucher program and leased up—an increase of 4.3 percentage points above the control group rate. When the results for the three PHAs other than Washington, D.C., are combined, the average pooled impact is slightly higher, at 5.6 percentage points (as shown in exhibit 4.3.)

While they were in the voucher program, families in the new rent rules group were living in housing units where the gross rent (that is, the total contract rent plus utilities) averaged $1,231, which was only $2 lower than the amount for the existing rules group. In both groups, nearly all families were renting units costing less than $1,500 per month except in Washington, D.C., as discussed below.

57 Families still formally enrolled in the voucher program but who received zero HAP, zero Family Share, and zero TTP (as well as having zero Gross Rent) in Month 13, were considered “active, not leased up” in the last month of Year 1 for the purposes of this analysis.

62

Exhibit 4.2. Impacts on Families’ Housing Costs and Subsidies in Year 1, Al Public Housing Agencies (PHAs)

New

Existing Difference

Outcome

Rent Rules Rent Rules

(Impact)

P-Value

Enrollment status at the end of Year 1 (%)

Currently enrolled in HCV program and

leased up

92.4

88.2

4.3 ***

0.000

Currently enrolled in HCV, not leased up

1.6

3.0

-1.5

n/a

Exited HCV program

5.5

8.0

-2.5 ***

0.000

Ported out to another housing agency

0.6

0.9

-0.3

n/a

Gross rent

Gross rent in last month of Year 1 if received

HCVa (%)

- -

- -

Less than $1,000

34.3

34.8

- -

$1,000 - $1,499

46.5

45.2

- -

$1,500 or more

19.2

20.0

- -

Average gross rent in last month of Year 1 if

- -

- -

received HCV in that month ($)

1,231

1,233

TTP

TTP in last month of Year 1b (%)

***

0.000

Exited HCV or not leased up during Year 1

7.6

11.9

-4.3

$0

2.5

8.0

-5.5

$1 - $50

5.2

10.1

-5.0

$51 - $75

8.3

3.1

5.3

$76 - $100

7.5

2.9

4.7

$101 - $150

11.4

8.9

2.5

$151 - $300

23.6

19.2

4.3

$301 - $700

27.1

26.4

0.8

$701 or above

6.9

9.6

-2.7

Average monthly TTP in months received HCVb

($)

289

303

- -

- -

Has a utility allowance in last month of Year 1

(%)

84.1

80.6

3.5 ***

0.000

Family share

Family share in last month of Year 1c (%)

***

0.000

Exited HCV or not leased up during Year 1

7.6

11.9

-4.3

$0

1.7

6.1

-4.4

$1 - $100

16.7

13.8

2.9

$101 - $300

33.8

27.2

6.6

$301 - $700

31.9

29.2

2.6

$701 or above

8.4

11.9

-3.5

(continued)

63

Exhibit 4.2 (continued)

New

Existing Difference

Outcome

Rent Rules Rent Rules

(Impact)

P-Value

Average monthly family share in months

received HCVc ($)

335

351

- -

- -

Family share as percentage of gross rent in last

month of Year 1, if received HCV (%)

28.1

30.7

- -

- -

Housing subsidy

Average number of months received housing

subsidyd

11.4

11.2

0.2 ***

0.000

Average monthly housing subsidy in months

received HCVd ($)

863

848

- -

- -

Average annual housing subsidy in Year 1 ($)

9,977

9,719

258 ***

0.008

Average annual housing subsidy (%)

***

0.000

Exited HCV or not leased up during Year 1

1.4

2.1

-0.7

$0

0.3

0.7

-0.4

$1 - $4,999

12.9

16.5

-3.6

$5,000 - $9,999

45.4

43.2

2.2

$10,000 - $14,999

23.7

20.8

2.9

$15,000 or more

16.4

16.8

-0.4

Average housing subsidy in last month of Year 1,

if received HCV ($)

901

880

- -

- -

Sample size (total = 6,665)

3,312

3,353

HCV = Housing Choice Voucher. TTP = total tenant payment.

aGross rent is the contract rent plus the utility allowance of the unit.

bTotal tenant payment is the amount a family must contribute toward rent and utilities regardless of the unit selected. Under the new rent rules, TTP is 28 percent of prior-year gross income, and under existing rent rules TTP is 30 percent of adjusted income.

cFamily share is the family's contribution toward its gross rent. It may be higher than the TTP if the family rents a unit with a gross rent that exceeds the payment standard.

dHousing subsidy is the full subsidy amount paid by the housing agency and includes any utility allowance payments made to the tenant in addition to rent paid to the owner by the housing agency.

Notes: Estimates were regression-adjusted using ordinary least squares, controlling for pre-random assignment characteristics of sample members. Differences between the new rent rules group and the existing rent rules group were assessed using a two-tailed t-test for continuous variables and selected outcomes expressed as proportions. For categorical variables a chi-square test was used to determine whether there is a difference in the distribution of related outcomes for the new rent rules group compared with the existing rent rules group. The p-value indicates the likelihood that the difference between the new rent rules group and the existing rent rules group arose by chance. Statistical significance levels are indicated as: *** = 1 percent; ** = 5 percent; * = 10 percent. When categorical variables are part of a distribution, the statistical significance levels, which apply to the distribution, are shown above the distribution. A statistical test was not performed on differences in the percentage of households currently enrolled in HCV

but not leased up, or differences in the percentage of households that ported out to another housing agency due to small sample sizes within those categories. Rounding may cause slight discrepancies in calculating sums and differences. Square brackets indicate that the chi-square test may not be valid due to small sample sizes within the cross-tabulation distribution. Italic type indicates comparisons that are nonexperimental. Statistical tests were not performed; therefore, there are no impacts or p-values to report.

Confirmatory outcomes were tested for multiple hypothesis testing using the Benjamini-Hochberg procedure. The adjusted p-value = .016 for the impact on the total Year 1 housing subsidy for all four PHAs combined.

Source: MDRC calculations using PHA data.

64

Exhibit 4.3. Selected Impacts on Tenants’ Housing Costs and Subsidies in Year 1, Al Public Housing Agencies (PHAs) Except Washington, D.C.

New

Existing Difference

Outcome

Rent Rules Rent Rules

(Impact)

P-Value

Currently enrolled in HCV program and

leased up at the end of Year 1 (%)

90.4

84.8

5.6 ***

0.000

Average monthly TTP in months received

HCVa ($)

255

273

- -

- -

Average annual housing subsidy in Year 1

($)

7,505

7,188

316 ***

0.000

Any action that requires staff responseb

56.1

85.0

-28.9 ***

0.000

Sample size (total = 4,756)

2,368

2,388

HCV = Housing Choice Voucher. TTP = total tenant payment.

aTotal tenant payment is the amount a family must contribute toward rent and utilities regardless of the unit selected. Under the new rent rules, TTP is 28 percent of prior year income and under existing rent rules TTP is 30 percent of adjusted income.

bCertification actions that require staff interaction include annual reexaminations, interim reexaminations (except for end of grace period and end of hardship records), and change-of-unit actions.

*** = 1 percent. ** = 5 percent. * = 10 percent.

Notes: Estimates were regression-adjusted using ordinary least squares, controlling for pre-random assignment characteristics of sample members. Differences between the new rent rules group and the existing rent rules group were assessed using a two-tailed t-test. The p-value indicates the likelihood that the difference between the new rent rules group and the existing rent rules group arose by chance. Rounding may cause slight discrepancies in calculating sums and differences. Confirmatory outcomes were tested for multiple hypothesis testing using the Benjamini-Hochberg procedure. The adjusted p-value = .000 for the impact on the total Year 1 housing subsidy for all PHAs combined excluding Washington, D.C.

Source: MDRC calculations, using PHA data

Over the course of the first year, the new rent rules group paid an average monthly TTP

of $289 while in the voucher program, or $14 less than the control group households paid while stil receiving vouchers. 58 As was true after the initial recertifications at the beginning of the study, a year later, the new rent rules group was less likely than the existing rules group to be paying a very low TTP ($0 to $50), owing to the minimum TTP policy. They were also somewhat less likely to be paying a very high TTP (more than $700). 59

58The table does not present impact estimates on these measures, because differences between the two research groups in the average length of time receiving vouchers in the first year mean that the full samples of each group could not be included in the 12-month averages. Excluding families who exited the voucher program could bias the impact estimates for these measures.

59 At the time of initial recertification, when the base income for calculating TTPs was known for both groups, the new rent rules led to a reduction in the proportion of families in the highest base monthly 65

The average monthly family share (which includes payments by tenants above their obligated TTP contribution) was also lower by $16 for the new rules group than for the existing rules group ($335 versus $351,) while the families were stil in the voucher program. Overall, in the last month of Year 1, family share averaged 28.1 percent of the average gross rental cost for families in the new rules group and about 30.7 percent of gross rent for families in the existing rules group.

The new rules group was more likely by 3.5 percentage points to receive a utility allowance during the first year. That allowance may reflect the requirement under the new rent policy that all families pay at least the minimum TTP to their landlords. Those families whose TTPs were less than the minimum (because of a hardship exemption) were stil required to pay the minimum amount to their landlords, but they could receive a reimbursement for the exempted amount through the utility allowance payment.

The lower average TTP for the new rules group combined with a longer duration in the voucher program means, of course, that tenants in that group received a somewhat larger total housing subsidy than they would have received in the absence of the new policy (represented by the control group’s subsidy amount). On average, the new rules group received $9,977 in housing subsidies during the first fol owup year, which is $258 (or 2.7 percent) more than the control group average ($9,719). The difference was somewhat higher—$316 (or 4.4 percent)—when the findings from all PHAs except Washington, D.C., are pooled, as shown in exhibit 4.3. Those impacts for both pooled samples are statistically significant, and they remain statistically significant after being adjusted for multiple outcomes. 60

This general pattern of results—somewhat longer tenure on the voucher program, somewhat lower TTPs, and small increases in the total subsidy amount that the new rent rules group received relative to the control group—is to be expected during the 3-year period until the next recertification under the new rent rules takes place. Although several factors may be at play, changes in how much money families earned over time and how the new and existing rent policies treat income changes help explain this pattern.

The control groups’ earnings trends—which indicate what the new rules group would have earned in the absence of rent reform—are key. The earnings of household heads in the control group were on an upward trajectory in Year 1 (see, for example, Chapter 3, exhibits 3.1

and 3.4, in this report). 61 Except in Washington, D.C., (where the control group faced a biennial income bracket relative to the control group, thus reducing the proportion with very high TTPs, see Riccio, Deitch, and Verma (2017). Although base income will not be recalculated for the new rules group until the triennial recertification, the TTP estimates at the end of Year 1 suggest a continuation of this pattern.

60 The pooled impact estimates were adjusted using the Benjamini-Hochberg method, described in Appendix B. The adjustment takes into account the fact that impacts were also estimated for average Year 1 earnings (a second preliminary confirmatory outcome measure). The adjusted p-value = .016 for the four-PHA pooled sample, and the adjusted p-value = .000 for the pooled sample without Washington, D.C.

61Ideally, to understand the alignment between earnings and housing subsidies, earnings would be 66

recertification policy), families in the control group were expected to report those earnings increases to the PHAs by the end of the first fol owup year. Unless they had earned their way off the voucher program (or left for other reasons), their TTPs would be increased and their subsidies would be lowered after their annual recertification at the end of Year 1.

In contrast, families in the new rent rules group (and in the control group in Washington, D.C.) whose earnings grew over Year 1 did not report those increases to the PHAs; consequently, they did not have their TTPs raised and subsidies reduced. Also, they could not earn their way off the voucher program during that period, no matter how much their incomes grew. That was intended by the policy design so that families would experience the benefits of their increased work effort during the 3 years between recertifications. For the PHAs, this meant a small increase in housing subsidies during that period; they had to forgo the normal opportunity to save on subsidy expenditures (until the triennial recertification) for families in the new rules group who increased their incomes. An important open question is whether the PHAs wil recoup those forgone subsidy reductions after the triennial recertifications are completed. They may recoup the reductions if, by that time, more tenants in the new rules group than in the control group are steadily working and begin paying a higher TTP for the subsequent 3 years than they paid in the prior 3-year period. The evaluation’s longer-term analysis wil determine whether this turns out to be the case.

Impacts by PHA

In considering the variation in effects across the four PHAs—in Lexington, KY; Louisvil e, KY; San Antonio, TX; and Washington, D.C.—it is useful to keep in mind the big differences between the housing market in Washington, D.C., and in the other sites. This is reflected in the differences in gross rents charged to families and in the payment standards and subsidy levels paid by the PHAs. In the tight Washington, D.C. housing market, gross rents in the last month of Year 1 (for families who were stil in the voucher program at that time) averaged $1,796 for the new rules group (and only slightly lower for the existing rules group; see exhibit 4.4).

Indeed, almost 62 percent of those voucher holders in the new rules group were renting units that cost $1,500 or more per month. In contrast, families in Lexington, Louisvil e, and San Antonio spent roughly $900 to $1,000 per month, on average—and hardly any were renting units costing $1,500 or more.

In the latter three PHAs—Lexington, Louisville, and San Antonio—the new rent policy increased families’ likelihood (relative to the existing rules group) of still being in the voucher program and leased up at the end of Year 1. As exhibit 4.4 shows, this effect varied from an increase of 3.6 percentage points over the control group rate in Lexington to an 8-percentage-point increase in San Antonio. Formal exits from the voucher program for the new rules group were lower than for the control group by 3.1 to 3.5 percentage points measured at the household level, counting the earnings of all adults in any given household. As discussed in Chapter 3, however, it is not possible for this evaluation to estimate household-level earnings using National Directory of New Hires (NDNH) data (because it was not possible in that database to link nonheads of household with the appropriate household heads). Thus, household heads’ NDNH earnings should be viewed only as an approximation of household-level earnings. The evaluation’s survey will eventually provide some information on earnings and other income at the household level.

67

across these three PHAs. In contrast, the new rent policy had little effect on voucher receipt in Year 1 in Washington, D.C. This finding may largely be the consequence of the biennial recertification for the control group in Washington, D.C., which meant that, just as was true for the new rules group, income growth would not result in any families earning their way off the voucher program within this early period, nor would early income growth reduce their housing subsidies (any such effects would begin to show up only in Year 3).

68

Exhibit 4.4. Impacts on Families’ Housing Costs and Subsidies in Year 1, by Public Housing Agency (PHA)

New Rent

Existing

Difference

Outcome

Rules

Rent Rules

(Impact)

P-Value

Lexington

Enrollment status at the end of Year 1 (%)

Currently enrolled in HCV program and

leased up

90.7

87.1

3.6 *

0.078

Currently enrolled in HCV program, not

leased up

0.0

0.0

0.0

NA

Exited HCV

8.0

11.4

-3.5 *

0.067

Ported out to another housing agency

1.4

1.5

-0.1

NA

Gross rent

Gross rent in last month of Year 1 if received

HCVa (%)

- -

- -

Less than $1,000

54.4

56.9

- -

$1,000– $1,499

45.6

43.1

- -

$1,500 or more

0.0

0.0

- -

Average gross rent in last month of Year 1 if re-

ceived HCV in that month ($)

934

921

- -

- -

TTP

TTP in last month of Year 1b (%)

*** 0.000

Exited HCV or not leased up during Year 1

9.4

12.9

-3.6

$0

0.0

0.0

0.0

$1–$50

-0.1

0.7

-0.7

$51–$75

0.0

0.0

0.0

$76–$100

0.0

0.0

0.0

$101–$150

32.7

29.4

3.3

$151–$300

27.7

19.7

8.0

$301–$700

27.1

30.9

-3.8

$701 or above

3.2

6.4

-3.3

Average monthly TTP in months received HCVb ($)

276

308

- -

- -

Has a utility allowance in last month of Year 1 (%)

83.6

79.5

4.1 *

0.098

Family share

Average monthly family share in months received

HCVc ($)

327

355

- -

- -

Family share as percentage of gross rent in last

month of Year 1, if received HCV (%)

35.1

40.9

- -

- -

Housing subsidy

Average number of months received housing

subsidyd

11.2

11.1

0.1

0.588

(continued)

69

Exhibit 4.4 (continued)

New Existing

Difference

Outcome

Rent Rules Rent Rules

(Impact)

P-Value

Average monthly housing subsidy in months

received HCV ($)

598

562

- -

- -

Average annual housing subsidy in Year 1 ($)

6,777

6,418

359 **

0.029

Average annual housing subsidy (%)

[ ]

0.100

Exited HCV or not leased up during Year 1

1.2

1.2

0.0

$0

0.3

1.0

-0.7

$1–$4,999

22.1

26.3

-4.3

$5,000–$9,999

60.5

58.0

2.5

$10,000–$14,999

15.9

13.5

2.4

$15,000 or more

0.0

0.0

0.0

Average housing subsidy in last month of Year 1,

if received HCV ($)

607

550

- -

- -

Sample size (total = 979)

486

493

Louisville

Enrollment status at the end of Year 1 (%)

Currently enrolled in HCV program and

leased up

93.7

89.7

4.1 ***

0.001

Currently enrolled in HCV program, not

leased up

0.3

0.3

0.0

NA

Exited HCV

5.7

9.1

-3.4 ***

0.005

Ported out to another housing agency

0.2

0.9

-0.7

NA

Gross rent

Gross rent in last month of Year 1 if received

HCVa (%)

- -

- -

Less than $1,000

43.9

43.9

- -

$1,000– $1,499

55.8

55.7

- -

$1,500 or more

0.3

0.5

- -

Average gross rent in last month of Year 1 if re-

ceived HCV in that month ($)

997

998

- -

- -

TTP

TTP in last month of Year 1b (%)

***

0.000

Exited HCV or not leased up during Year 1

6.3

10.4

-4.1

$0

5.0

11.2

-6.2

$1–$50

14.3

15.0

-0.7

$51–$75

5.2

3.4

1.8

$76–$100

4.5

3.7

0.8

$101–$150

7.0

5.2

1.9

$151–$300

26.7

19.5

7.1

$301–$700

28.7

26.3

2.4

(continued)

70

Exhibit 4.4 (continued)

Existing Difference

Outcome

New

Rent Rules

Rent Rules

(Impact)

P-Value

$701 or above

2.3

5.4

-3.1

Average monthly TTP in months received HCVb

($)

228

239

- -

- -

Has a utility allowance in last month of Year 1

(%)

90

86

4.0 ***

0.006

Family share

Average monthly family share in months re-

ceived HCVc ($)

317

328

- -

- -

Family share as percentage of gross rent in last

month of Year 1, if received HCV (%)

28.5

30.6

- -

- -

Housing subsidy

Average number of months received housing

subsidyd

11.6

11.4

0.2 ***

0.009

Average monthly housing subsidy in months

received HCV ($)

678

664

- -

- -

Average annual housing subsidy in Year 1 ($)

7,898

7,659

239 *

0.066

Average annual housing subsidy (%)

[** ]

0.045

Exited HCV or not leased up during Year 1

1.0

1.8

-0.8

$0

0.1

0.1

0.0

$1–$4,999

14.0

19.4

-5.3

$5,000–$9,999

58.4

52.3

6.1

$10,000–$14,999

26.1

26.1

-0.1

$15,000 or more

0.4

0.3

0.1

Average housing subsidy in last month of Year 1,

if received HCV ($)

708

688

- -

- -

Sample size (total = 1,908)

947

961

San Antonio

Enrollment status at the end of Year 1 (%)

Currently enrolled in HCV program and

leased up

86.8

78.8

8.0 ***

0.000

Currently enrolled in HCV program, not

leased up

3.6

8.2

-4.6

NA

Exited HCV

8.6

11.6

-3.1 **

0.029

Ported out to another housing agency

1.0

1.4

-0.4

NA

Gross rent

Gross rent in last month of Year 1 if received

HCVa (%)

- -

- -

Less than $1,000

51.2

51.6

- -

$1,000– $1,499

47.9

46.7

- -

$1,500 or more

0.8

1.8

- -

Average gross rent in last month of Year 1

if received HCV in that month ($)

990

1,007

- -

-

-

(continued)

71

Exhibit 4.4 (continued)

New

Existing

Difference

Outcome

Rent Rules Rent Rules

(Impact)

P-Value

TTP

TTP in last month of Year 1b (%)

***

0.000

Exited HCV or not leased up during Year 1

13.2

21.2

-8.0

$0

0.0

0.0

0.0

$1–$50

3.4

7.3

-4.0

$51–$75

0.2

4.7

-4.5

$76–$100

17.3

3.1

14.2

$101–$150

9.1

7.2

1.9

$151–$300

26.8

22.4

4.4

$301–$700

27.4

28.0

-0.6

$701 or above

2.8

6.1

-3.4

Average monthly TTP in months received HCVb

($)

271

291

- -

- -

Has a utility allowance in last month of Year 1

(%)

76.1

70.6

5.5 ***

0.007

Family share

Average monthly family share in months re-

ceived HCVc ($)

312

340

- -

- -

Family share as percentage of gross rent in last

month of Year 1, if received HCV (%)

31.2

37.1

- -

- -

Housing subsidy

Average number of months received housing

subsidyd

11.1

10.8

0.3 **

0.013

Average monthly housing subsidy in months re-

ceived HCV ($)

669

645

- -

- -

Average annual housing subsidy in Year 1 ($)

7,507

7,088

419 ***

0.002

Average annual housing subsidy (%)

**

0.023

Exited HCV or not leased up during Year 1

1.9

3.3

-1.4

$0

0.5

1.1

-0.6

$1–$4,999

15.9

20.3

-4.5

$5,000–$9,999

59.1

58.0

1.1

$10,000–$14,999

22.5

17.2

5.3

$15,000 or more

0.2

0.1

0.1

Average housing subsidy in last month of Year 1,

if received HCV ($)

678

627

- -

- -

Sample size (total = 1,869)

935

934

(continued)

72

Exhibit 4.4 (continued)

New Rent

Existing Difference

Outcome

Rules

Rent Rules

(Impact)

P-Value

Washington, D.C.

Enrollment status at the end of Year 1 (%)

Currently enrolled in HCV program and

leased up

97.5

96.5

1.0

0.206

Currently enrolled in HCV program, not

leased up

1.6

2.1

-0.5

NA

Exited HCV

0.9

1.5

-0.5

0.291

Ported out to another housing agency

0.0

0.0

0.0

NA

Gross rent

Gross rent in last month of Year 1 if received

HCVa (%)

- -

- -

Less than $1,000

1.7

1.7

- -

$1,000– $1,499

36.6

35.5

- -

$1,500 or more

61.7

62.8

- -

Average gross rent in last month of Year 1, if re-

ceived HCV in that month ($)

1,796

1,787

- -

- -

TTP

TTP in last month of Year 1b (%)

***

0.000

Exited HCV or not leased up during Year 1

2.5

3.5

-1.0

$0

3.6

16.8

-13.2

$1–$50

0.6

12.7

-12.0

$51–$75

23.7

2.8

20.9

$76–$100

5.1

2.9

2.2

$101–$150

6.7

4.3

2.5

$151–$300

15.6

15.3

0.4

$301–$700

25.1

22.7

2.3

$701 or above

17.1

19.1

-2.0

Average monthly TTP in months received HCVb

($)

371

378

Has a utility allowance in last month of Year 1

(%)

85.7

85.1

0.6

0.709

Family share

Average monthly family share in months re-

ceived HCVc ($)

378

385

- -

- -

Family share as percentage of gross rent in last

month of Year 1, if received HCV (%)

21.5

21.1

- -

- -

Housing subsidy

Average number of months received housing

subsidyd

11.7

11.5

0.1

0.147

Average monthly housing subsidy in months re-

ceived HCV ($)

1,379

1,369

- -

- -

(continued)

73

Exhibit 4.4 (continued)

Existing Difference

Outcome

New Rent

Rules Rent Rules

(Impact)

P-Value

Average annual housing subsidy in Year 1 ($)

16,211

15,953

258

0.319

Average annual housing subsidy (%)

0.460

Exited HCV or not leased up during Year 1

1.3

1.7

-0.4

$0

0.3

0.6

-0.3

$1–$4,999

3.9

5.2

-1.3

$5,000–$9,999

11.3

11.9

-0.6

$10,000–$14,999

25.9

23.0

2.9

$15,000 or more

57.3

57.6

-0.3

Average housing subsidy in last month of Year 1,

if received HCV ($)

1,415

1,420

- -

- -

Sample size (total = 1,909)

944

965

HCV = Housing Choice Voucher. TTP = total tenant payment.

aGross rent is the contract rent plus the utility allowance of the unit.

bTotal tenant payment is the amount a family must contribute toward rent and utilities regardless of the unit selected. Under the new rent rules, TTP is 28 percent of prior-year gross income, and under existing rent rules TTP is 30 percent of adjusted income.

cFamily share is the family's contribution toward its gross rent. It may be higher than the TTP if the family rents a unit with a gross rent that exceeds the payment standard.

dHousing subsidy is the full subsidy amount paid by the housing agency and includes any utility allowance payments made to the tenant in addition to rent paid to the owner by the housing agency.

*** = 1 percent. ** = 5 percent. * = 10 percent.

Notes: Estimates were regression adjusted using ordinary least squares, controlling for pre-random assignment characteristics of sample members. Differences between the new rent rules group and the existing rent rules group were assessed using a two-tailed t-test for continuous variables and selected outcomes expressed as proportions. For categorical variables a chi-square test was used to determine whether there is a difference in the distribution of related outcomes for the new rent rules group compared with the existing rent rules group. The p-value indicates the likelihood that the difference between the new rent rules group and the existing rent rules group arose by chance. When categorical variables are part of a distribution, the statistical significance levels, which apply to the distribution, are shown above the distribution. A statistical test was not performed on differences in the percentage of households currently enrolled in HCV but not leased up, or differences in the percentage of households that ported out to another housing agency due to small sample sizes within those categories. Rounding may cause slight discrepancies in calculating sums and differences. Square brackets indicate that the chi-square test may not be valid due to small sample sizes within the cross-tabulation distribution. Italic type indicates comparisons that are nonexperimental. Statistical tests were not performed; therefore, there are no impacts or p-values to report. The variation across the four PHAs in estimated impacts on average annual housing subsidy in Year 1 is not statistically significant based on an H-statistic test.

Source: MDRC calculations, using PHA data

74

The policy’s first-year effects on average annual housing subsidies were statistically significant in three sites, ranging from an increase in housing subsidy over the existing rules group of $419 in San Antonio to $239 in Louisvil e. Put differently, the average Year 1 subsidy for the new rules group exceeded the control group mean by 6.0 percent in San Antonio and 3.1

percent in Louisvil e. The changes in subsidy payments were not statistically significant in Washington, D.C., where the biennial recertification policy would minimize any first-year effects on subsidy payments.

In Louisvil e, because 22 percent of families opted out of the new rent policy, the estimated intent-to-treat (ITT) impacts shown in exhibit 4.4 may be understated because they are averaged over all families who enrol ed in the new rules group, whether or not those families were actually subject to the new rules; therefore, as explained in Chapter 2, treatment-on-treated (TOT) adjustments were made, which attribute all effects to only those families who were exposed to the policy (similar adjustments were produced for selected employment outcomes, as discussed in Chapter 3). Exhibit C.1 presents the results of the TOT analysis. It shows, for example, that the TOT impact on the average annual subsidies in Year 1 is $308

(compared with the $239 ITT estimate).

In each of the four PHAs, families in the new rent rules group paid a somewhat lower average monthly TTP while they were enrolled in the voucher program than did their counterparts in the control group; however, the actual patterns varied across different levels of TTPs. At the lower end, as exhibit 4.4 shows, in Louisvil e and Washington, D.C., fewer families in the new rules group paid zero toward their rent and utilities in the last month of Year 1. This finding reflects the institution of a minimum TTP for the new rules group in PHAs that previously had no minimum TTPs. In Louisvil e, the proportion of zero-TTP families was 4.1

percentage points lower than the control group rate of about 11 percent. In Washington, D.C., it was about 13 percentage points lower than the control group rate of almost 17 percent. Some families in the new rent rules group could stil pay zero TTP as part of a hardship remedy. In addition, in Louisvil e, some families in the new rent rules group paying a zero TTP may have been families who opted out of the new rules and, therefore, would not be subject to a minimum TTP.

In Lexington and San Antonio, because of preexisting minimum TTP policies, no families in the new rent rules group or the existing rent rules group had a zero TTP at the end of Year 1. However, in San Antonio, where the preexisting $50 minimum TTP was raised to $100

for the new rent rules group, the proportion of families in the new rules group paying TTPs of $50 or less fell by 4 percentage points relative to the control group rate, whereas the proportion paying from $76 to $100 increased by 14.2 percentage points. 62 In Lexington, because both research groups were required to pay a $150 minimum rent, with few exceptions, the new rent policy could have lit le effect on the proportion of families falling in the lower range of the TTP

distribution.

62Some families could pay less than the minimum TTP, in either research group, if they received a hardship exemption.

75

Exhibit 4.4 also shows that families’ total contributions toward their housing costs, as indicated by the estimate of family share (that is, TTP plus additional family payments for rent and utilities above the payment standard), varied across the four PHAs. For families in the existing rules group, it ranged from a low of $328 per month while in the voucher program in Louisvil e to a high of $385 in Washington, D.C. In each of the PHAs, families in the new rules group paid a lower family share than did families in the existing rules group. When family share is expressed as a percentage of gross rent, the proportions varied from 35.1 percent for the new rules group in Lexington to a low of 21.5 percent in Washington, D.C.

Paying the Minimum TTP

Exhibit 4.5 shows how the TTPs paid by the new rent rules group compared with the PHAs’

minimum TTP levels. For all PHAs combined, only 5.7 percent of families in the new rent rules group ever paid less than the minimum TTP during Year 1. Those who did pay less include some families who received a time-limited hardship remedy (although not all families with a hardship remedy paid below the minimum TTP). Most families (77.6 percent) paid more than the minimum TTP sometime during Year 1, whereas 29.3 percent paid exactly the minimum sometime during Year 1. Across the PHAs, Lexington stands out, with almost one-half (48.1

percent) of its families having paid exactly the minimum TTP. That rate is considerably higher than in the other PHAs (where it ranges from 21.3 percent to 31.6 percent) and reflects Lexington’s relatively high $150 per month minimum TTP and its limited exemptions policy.

No Lexington families paid less than the minimum during Year 1.

In the other three PHAs, families paying below the minimum TTP did so for roughly 4

to 6 months within Year 1. This average duration may grow as more fol owup data become available. Some families were stil in the midst of a hardship remedy spell at the end of Year 1, in which they were paying less than the minimum (and wil continue doing so at least partly into Year 2), and other families wil begin new spells in Year 2 or later.

76

Exhibit 4.5. Total Tenant Payment (TTP) Relative to the Local Minimum TTP

and Use of Safeguards in Year 1, by Public Housing Agency (PHA),

New Rent Rules Group Only

Outcome

San Washington,

All

Lexington Louisville Antonio

D.C. PHAs

Minimum TTP ($)

150

50

100

75

Family TTP relative to the local

minimum TTP (%)

Ever paid below the minimum TTPa

0.0

5.9

7.1

6.9

5.7

Ever paid the minimum TTP

48.1

21.3

23.7

31.6

29.3

Ever paid above the minimum TTP

66.2

86.1

78.8

75.5

77.6

Number of months paidb

Below the minimum TTP

0.0

5.2

4.1

6.5

5.2

The minimum TTP

9.3

8.3

9.9

9.3

9.3

Above the minimum TTP

10.2

11.0

10.8

11.0

10.8

Ever had grace-period TTPc (%)

21.6

21.4

19.2

27.1

22.5

Ever received a restricted interim recerti-

fication (%)

5.2

2.0

1.5

2.1

2.4

Ever received a hardship remedy (%)

5.8

11.3

4.6

11.8

8.4

Received hardship remedy in last month

of Year 1, if received HCV in that month

2.2

8.6

4.5

6.9

5.9

(%)

Average number of months of a hardship

4.1

5.4

4.8

6.2

5.0

(for those who received hardship)

Sample size

486

735

935

944 3,100

HCV = Housing Choice Voucher.

aSome families paid less than the minimum TTP during Year 1 without receiving a hardship exemption because they were not immediately subject to the new rent rules and have some months during the followup period where they were paying a TTP under the existing rent rules. Housing outcome measures are relative to the expected initial effective date; for some families a triennial did not become effective when expected resulting in new rent rules families being on the existing rules for some part of Year 1. This occurred due to families having their initial triennial become effective late, porting out or terminating from HCV without completing a triennial recertification, or having an old effective certification under the existing rules without a termination or port out which is probably a data error.

bThe "number of months paid" measures limit the sample to those who ever paid that family TTP

relative to the local minimum TTP. For example, the number of months paid below the minimum TTP is shown only for those who ever paid the minimum TTP.

cAt the initial recertification, families receiving grace-period TTPs have their TTPs calculated based on current/anticipated income for 6 months, rather than retrospective income. The grace-period TTP is used if a family’s current/anticipated income is more than 10 percent lower than its retrospective income.

Notes: Estimates were regression-adjusted using ordinary least squares, controlling for pre-random assignment characteristics of sample members. Rounding may cause slight discrepancies in calculating 77

sums and differences. Louisville families who opted out of the study are excluded because their rent calculation is subject to existing rules. Total tenant payment is the amount a family must contribute toward rent and utilities regardless of the unit selected. Under the new rent rules TTP is 28 percent of prior year income and under existing rent rules TTP is 30 percent of adjusted income. The minimum TTP varies by site and research group. The measures are created using the relevant minimum TTP.

Source: MDRC calculations, using PHA data

Use of Safeguards by the New Rent Rules Group

Grace Period Rents

Some families in the new rent rules group found that their retrospective gross incomes were substantially higher than their current/anticipated gross incomes at the time of their initial recertifications. The new rent policy includes several critical safeguards (described in Chapter 1 and summarized in exhibit 4.6) to help protect such families from excessive rent burdens. The grace period is one such safeguard. If at the initial recertification a family’s current/anticipated gross income is more than 10 percent lower than its retrospective income, the family automatically qualifies for (and receives) a 6-month grace-period TTP based on 28 percent of its current/anticipated gross income. The family would need to pay the minimum TTP if that 28

percent were less than the minimum TTP threshold set by its PHA, unless the family applies for and receives a hardship exemption. Only available at the beginning of the 3-year period (and at any subsequent triennial recertifications), the 6-month grace period temporarily protects the household from a high rent burden while it tries to restore its income to its prior level. At the end of the 6-month grace period, the TTPs for those families automatically reverts to the TTPs that were based on the retrospective income originally calculated for them; however, if the family cannot restore its current income to that original retrospective gross income level, it may request an interim recertification (limited to one per year) or a hardship remedy.

78

Exhibit 4.6. Safeguards Built into the New Rent Rules Policy

Safeguard

Timing

Eligibility Criteria

Modified TTP

Grace-period TTP At triennial certification

Current or anticipated

Based on 28 percent of

monthly income is

the current or

more than 10 percent

anticipated monthly

less than retrospective

income. The modified

monthly income.

TTP lasts for 6 months

and then automatically

switches back to being

based on retrospective

income.

Interim

Upon family’s request, up

The family’s

Set at 28 percent of

recertificationa

to once per year.

retrospective income at retrospective income

the time of the request

based on the 12 months

for the interim

before the request.

recertification is more

than 10 percent below

its previously

established income.

Hardship

At any time

TTP is more than 40

Set at 28 percent of a

remedies

percent of current or

family’s current or

anticipated monthly

anticipated income

income or

(which may be less than

the family is at risk of

the minimum TTP,

eviction.

except in Lexington) for

up to 180 days (can be

renewed), or

set at the minimum TTP

for up to 180 days (can

be renewed), or

based on an additional

interim recertification

beyond the normal one-

per-year option, or

supplemented with a

“transfer voucher” to

help a move to a more

affordable unit.

Notes: Income refers to gross income. The new rent policy uses gross income (e.g., without adjustments) regardless of whether using current, anticipated, or retrospective income.

a Interim recertification refers to restricted interims to reduce TTP.

More than one-fifth (22.5 percent) of families had received a grace-period TTP at the initial recertification when the study period began, reaching as high as 27.1 percent in Washington, D.C., as shown in exhibit 4.5. The substantial degree of reliance on this safeguard testifies to its importance in protecting families from difficult-to-afford TTPs, at least temporarily, in a rent system that bases TTPs primarily on retrospective income.

79

Interim Recertifications

As another safeguard, the new rent policy allows families one interim recertification per year.

For the new rent rules group, these are referred to as “restricted interim recertifications” because of the numerical restriction placed on them. A family qualifies for this mechanism to lower its TTP only if its income drops by more than 10 percent of its retrospective income over the 12

months immediately prior to the time it requests an interim adjustment. Exhibit 4.5 shows that, during Year 1, 2.4 percent of the new rent rules group received a restricted interim recertification for this purpose. The rate ranged from 1.5 percent in San Antonio to 5.2 percent in Lexington. In general, the new rules group was less likely to receive an interim recertification for a TTP reduction than the existing rules group, as discussed later in this chapter.

Hardship Remedies

As exhibit 4.6 shows, in addition to grace-period TTPs and interim recertifications, the new rent policy offers potential further relief to families whose TTPs exceed 40 percent of their current/anticipated gross income. Such families are considered to have excessive rent burdens and are generally eligible to request a hardship remedy. In Lexington, however, families are eligible for a hardship remedy only if they are paying TTPs that exceed the PHA’s $150

minimum and stil meet the 40-percent threshold. No families can pay less than the $150

minimum except in cases in which households become classified as disabled.

Earlier in the demonstration, when very low rates of hardship requests became apparent after the initial recertifications were completed, the PHAs, HUD, and MDRC discussed the possibility that some families might be eligible for but not be sufficiently aware of the hardship provisions of the new rent policy. To address that concern, the PHAs sent flyers to all families in the new rent rules group reminding them not only of the benefit of not needing to report earnings increases until the triennial recertification but also that if they were experiencing difficulty meeting their rent obligations, they might qualify for hardship remedies or interim recertifications to reduce their TTPs, and that they should contact their housing specialists to find out whether they qualified (see exhibit 4.7). In addition, the agencies mailed a special letter to families that MDRC identified (using PHA data) as having initial TTPs that might qualify them for a hardship remedy.

80

Exhibit 4.7. San Antonio Housing Authority’s Rent Reform Flyer

The letters encouraged those families to contact the PHA to see whether they did, in fact, qualify (Riccio, Deitch, and Verma, 2017). Of course, it is possible that not al families who qualified for a hardship remedy wanted to apply for one. For example, doing so may have required some families to interact with the PHA more than they would have liked. In addition, some potentially eligible families may have had an increase in income after the initial recertification, which they realized would disqualify them from receiving a hardship remedy.

Other considerations may have been factors as well.

81

A hardship remedy can be issued at any time during the 3-year period. Across all PHAs, about 8.4 percent of families requested and received a hardship remedy in Year 1, ranging from 4.6 percent in San Antonio to 11.8 percent in Washington, D.C. (see exhibit 4.5). This rate is higher than the rate at the time of initial recertification, when, as shown in the baseline report, only 0.5 percent of families across the four PHAs received a hardship remedy. 63 The small increase over time may reflect the possibility that a growing proportion of families experienced a substantial post-recertification loss of income during Year 1. It may also reflect a growing awareness among already-qualifying families of the availability of this safeguard and increased wil ingness to request it. 64

Exhibit 4.5 also shows that a somewhat smaller proportion of families had received a hardship remedy in the last month of Year 1 compared with the proportion who ever received a hardship remedy in Year 1. For example, in Washington, D.C., the rate fell to 6.9 percent, indicating that, for at least some families, those time-limited remedies expired and were not renewed.

Impacts on PHA Actions for Families

One goal of the new rent policy is to simplify the rent determination process. Doing so, it was hoped, would reduce the administrative burden and costs for the PHAs, as well as lighten the burden on families. Toward that goal, as discussed previously, the new policy eliminates deductions from income in calculating TTPs (focusing on gross rather than adjusted income); ignores any income from (and documentation requirements for) assets that were valued at less than $25,000 (rather than the traditional $5,000 limit); simplifies the approach to estimating utilities costs; switches from an annual to triennial recertification process; and limits to one per year the number of interim recertifications permit ed as a result of income reductions (and only when a decrease in income exceeds 10 percent of retrospective income).

Potentially offsetting these burden-reducing features is the new policy’s reliance on income received during a 12-month retrospective period in determining the average monthly base income that is used to calculate a family’s TTP (except for grace-period and hardship TTPs), rather than relying on income that a family anticipates having in an average month during the coming 3 years. 65 In addition, the institution of minimum TTPs where none existed before (in Louisvil e and Washington, D.C.) and the new policy’s hardship remedies, when requested, may offset to some degree the reduction in administrative burden. MDRC’s implementation research is continuing to explore the PHAs’ experiences in operationalizing these features (fol owing up on early experiences that are discussed in the demonstration’s baseline report: Riccio, Deitch, and Verma, 2017), and the findings wil be presented in a future 63See Riccio, Deitch, and Verma (2017). At the time of initial recertification, the rates were zero in Lexington, 1.6 percent in Louisville, 0.2 percent in San Antonio, and 0.2 percent in Washington, D.C.

64It is not possible to determine the proportion of all families in the new rules group who might be eligible for a hardship remedy because, under the policy’s triennial recertification feature, they are not required to report their incomes for 3 years; however, future reports will include findings from qualitative and survey data on families’ understanding and views of the hardship protections.

65For details on how retrospective income is determined, see Riccio, Deitch, and Verma (2017).

82

report. In the meantime, it is possible to assess whether the new policy has begun to affect the incidence of formal actions that staff must complete to address changes in families’

circumstances.

Under HUD’s traditional rent policy, PHA housing specialists conduct annual income recertifications for families and take certain actions when families report changes in particular circumstances—for example, if the family moves to a different housing unit, if its income falls, if its income increases before its annual recertification takes place (although this reporting requirement can be waived under current HUD rules), 66 if its household composition changes (for instance, if individuals move out of or into their housing unit, if a child is born, or if a family member passes away), if the rent charged by the landlord for the family’s unit (the contract rent) changes, or for a number of other reasons. Under the new rent policy, al mandatory annual recertifications and interim reporting of income increases are eliminated, and no more than one recertification per year is permit ed for income reductions. Families must continue to report all other types of changes in circumstances required by traditional HUD rules.

Exhibit 4.8 compares the likelihood and frequency of these types of actions across the two research groups during Year 1 for all four PHAs combined. Almost three-fourths (73.6

percent) of the existing rules group had an action that required a response from PHA staff, compared with about one-half (49.6 percent) of the new rules group—a reduction of 24

percentage points. The main factor contributing to this change was the reduction in annual recertifications, which were 40 percentage points lower for the new rules group. Also contributing to the overall reductions in PHA actions were reductions in the likelihood of interim recertifications for decreases in income (by 10 percentage points) and reductions in interim recertifications for increased income (by 15.5 percentage points). 67 Although the new rent rules eliminated interim recertifications for increased income, a small proportion of families (1.4 percent) had received such an action, some of whom were likely families in Louisvil e who opted out of the new rent rules. Other cases may be the result of errors on the part of housing specialists or a result of inconsistencies in the recording of income.

66The Housing Opportunities Through Modernization Act of 2016, among other changes, eliminates the requirement for families to report increases in earned income between annual recertifications (codifying an option that had previously been left to local PHA discretion) and eliminates interim recertifications for families’ whose incomes drop by less than 10 percent. As of early 2019, however, HUD had not issued implementation guidance, and these provisions had not yet gone into effect.

67The reasons for interim recertification in exhibit 4.5 are not mutually exclusive. Thus, the same family could have had two reasons for an interim recertification, sometimes occurring at the same time (for example, a change in household composition and an income increase). These actions are counted separately except in “any action” measures.

83

Exhibit 4.8. Public Housing Agency (PHA) Actions in Year 1, Al PHAs

Outcome (%)

New

Existing Difference

Rent Rules

Rent Rules

(Impact)

P-Value

Ever had type of action (%)

Any action that requires staff responsea

49.6

73.6

-23.9 ***

0.000

Annualb

10.0

50.3

-40.4 ***

0.000

Move/change of unitc

8.2

9.5

-1.3 *

0.059

Interimsd

Decreased income

10.8

20.7

-10.0 ***

0.000

Restricted interim

2.2

NA

- -

- -

Hardship exemptione

6.6

0.1

6.5 ***

0.000

Household composition changef

1.5

5.0

-3.5 ***

0.000

Increased incomeg

1.4

16.9

-15.5 ***

0.000

Household composition change

5.7

8.7

-3.0 ***

0.000

Contract rent changeh

18.0

7.7

10.3 ***

0.000

Other actioni

9.5

3.7

5.9 ***

0.000

Number of actions

Average number of actions

0.6

1.1

-0.5 ***

0.000

Any action that requires staff responsea

(%)

***

0.000

None

50.4

26.4

23.9

1

40.2

44.7

-4.5

2

8.0

21.1

-13.1

3–4

1.4

7.5

-6.0

5 or more

0.0

0.4

-0.4

Move/change of unitc

[** ]

0.031

None

91.9

90.5

1.3

1

8.2

9.3

-1.2

2 or more

0.0

0.1

-0.1

Decreased income (%)

***

0.000

None

89.2

79.3

10.0

1

10.0

18.5

-8.5

2 or more

0.7

2.2

-1.5

Increased income (%)

***

0.000

None

98.6

83.1

15.5

1

1.3

15.4

-14.1

2 or more

0.1

1.5

-1.4

(continued)

84

Exhibit 4.8 (continued)

New Rent

Existing Difference

Outcome (%)

Rules Rent Rules

(Impact)

P-Value

Household composition changef (%)

**

*

0.000

None

94.3

91.4

3.0

1

5.2

8.2

-3.0

2 or more

0.5

0.5

0.0

Contract rent changeh (%)

**

*

0.000

None

82.0

92.3

-10.3

1

17.7

7.5

10.2

2 or more

0.3

0.2

0.1

Other actioni (%)

**

*

0.000

None

90.5

96.4

-5.9

1

9.0

3.4

5.6

2 or more

0.5

0.2

0.3

Sample size (total = 6,665)

3,312 3,353

aCertification actions that require staff interaction include annual reexaminations, interim reexaminations (except for end-of-grace-period and end-of-hardship records), and change-of-unit actions.

bAnnual reexaminations reflect actions recorded as "Action code 2: annual reexamination" on the 50058 form.

cMove/change of unit actions reflect actions recorded as "Action code 7: other change of unit" on the 50058 form.

If a move was recorded through an annual or interim action, it is not reflected in this outcome.

dInterims reflect all actions recorded as "Action code 3: interim reexamination" on the 50058 form except interim reexaminations to end a grace period or hardship rent. Types of interim actions are not mutually exclusive. Any action counts as each action once. At the same interim certification event, a household may have reported changes in its situation that fell into more than one of the categories displayed in this table.

eHouseholds in the existing rent rules groups in Louisville and Washington, D.C. were not subject to a minimum rent.

Thus, there was no hardship exemption available. This only includes hardships received through an interim recertification.

fThis outcome indicates a decrease in income that occurred at the same time that household composition changed.

When household members are removed, so is their income.

gInterims with increased income include interims in which income increased but TTP did not increase. This occurs when a family’s earnings did not increase enough for TTP to be over the minimum rent.

hThe control group often has contract rent changes included in their annual reexamination, and in that case the contract rent increase is not included in this category.

iOther actions include interims (or some other reason but not end of grace or hardship), which are difficult to clas-sify from the available data.

*** = 1 percent. ** = 5 percent. * = 10 percent.

Notes: Estimates were regression adjusted using ordinary least squares, controlling for pre-random assignment characteristics of sample members. Differences between the new rent rules group and the existing rent rules group were assessed using a two-tailed t-test for continuous variables and selected outcomes expressed as proportions. For categorical variables a chi-square test was used to determine whether there is a difference in the distribution of related outcomes for the new rent rules group compared with the existing rent rules group. The p-value indicates the likelihood that the difference between the new rent rules group and the existing rent rules group arose by chance. When categorical variables are part of a distribution, the statistical significance levels, which apply to the distribution, are shown above the distribution. Rounding may cause slight discrepancies in calculating sums and differences.

Source: MDRC calculations, using PHA data

85

At the same time, some types of actions were more frequent for the new rules group than for the existing rules group, thus offsetting somewhat the larger reduction in actions resulting primarily from the reduction in annual recertifications. Among those were staff actions to process changes in rent contracts with landlords, typically when landlords raised the rent. For control group families, those changes were usually addressed as part of the regular annual recertification process. For the new rules group, which was subject to triennial recertifications, changes in contract rents during the 3-year period required a separate action. Those actions, however, primarily required that staff determines that the increase was reasonable, update the contract rent amount, and notify the landlord and tenant about the changes to the rent calculations. This type of action does not require a review of the family’s income, which is one of the more time-consuming aspects of processing recertifications.

Across both research groups, few families had more than one or two actions requiring a staff response during Year 1 (exhibit 4.8). Overall, the new rent policy decreased the average number of actions requiring a staff response by 0.5 during Year 1, from 1.1 for the existing rules group to 0.6 for the new rules group; however, three actions that were among the most likely to be reduced—annual recertifications, interim recertifications for reductions in income, and interim recertifications for increases in income—were generally the most time-consuming actions for staff because they required reviewing household income so that the PHA’s software system could recalculate TTPs and subsidies.

These patterns vary substantially across the four PHAs, as shown in exhibit 4.9. In reviewing these results, it is important to keep in mind some important differences across PHAs in the policies concerning PHA actions that applied to the control group. In Lexington and Louisvil e, control group families remained subject to the traditional HUD requirements, such as annual recertifications and unlimited interim reporting of income decreases or increases between annual recertifications. (Also recall that in Louisvil e, 22 percent of families who were assigned to the new rent rules group opted out, which means they are subject to the same traditional rules as the control group.) In contrast, San Antonio’s control group members are not required to report earnings increases before their annual recertifications, and in Washington, D.C., the control group members must only report income increases every 2 years.

Some of the cross-PHA differences are driven primarily by these control group conditions in Washington, D.C., and the Louisvil e opt-outs. For example, at the high end, the reduction in annual recertifications was 75.8 percentage points in Lexington and 59.5

percentage points in San Antonio, compared with 40.3 percentage points in Louisvil e (where the opt-out families remained subject to the existing rules) and a very low 3.7 percentage points in Washington, D.C. (In Louisvil e, the TOT adjustment, which averages impacts only over the non-opt-out families, shows a reduction in annual recertifications of 51.9 percentage points; see exhibit C.1).

86

Exhibit 4.9. Public Housing Agency (PHA) Actions in Year 1, by PHA New

Existing Difference

Outcome (%)

Rent Rules Rent Rules

(Impact)

P-Value

Lexington

Ever had type of action (%)

Any action that requires staff responsea

46.6

91.2

-44.7 ***

0.000

Annualb

3.3

79.1

-75.8 ***

0.000

Move/change of unitc

12.1

13.4

-1.3

0.549

Interimsd

Decreased income

10.7

26.7

-16.0 ***

0.000

Restricted interim

5.2

NA

- -

- -

Hardship exemptione

5.0

0.4

4.6 ***

0.000

Household composition changef

0.8

4.1

-3.3 ***

0.001

Increased incomeg

2.0

22.8

-20.8 ***

0.000

Household composition change

6.9

8.4

-1.6

0.356

Contract rent changeh

11.5

9.0

2.5

0.193

Other actioni

15.1

6.6

8.5 ***

0.000

Number of actions

Average number of actions

0.6

1.7

-1.0 ***

0.000

Any action that requires staff responsea

(%)

***

0.000

None

53.4

8.8

44.7

1

33.5

44.1

-10.6

2

11.1

28.0

-16.9

3–4

2.0

17.1

-15.1

5 or more

-0.1

2.1

-2.1

Sample size (total = 979)

486

493

Louisville

Ever had type of action (%)

Any action that requires staff responsea

89.0

89.7

-0.6

0.652

Annualb

24.8

65.1

-40.3 ***

0.000

Move/change of unitc

7.3

12.5

-5.2 ***

0.000

(continued)

87

Exhibit 4.9 (continued)

New

Existing Difference

Outcome (%)

Rent Rules Rent Rules

(Impact)

P-Value

Household composition changef (%)

***

0.000

None

94.3

91.4

3.0

1

5.2

8.2

-3.0

2 or more

0.5

0.5

0.0

Contract rent changeh (%)

***

0.000

None

82.0

92.3

-10.3

1

17.7

7.5

10.2

2 or more

0.3

0.2

0.1

Other actioni (%)

***

0.000

None

90.5

96.4

-5.9

1

9.0

3.4

5.6

2 or more

0.5

0.2

0.3

Sample size (total = 6,665)

3,312 3,353

aCertification actions that require staff interaction include annual reexaminations, interim reexaminations (except for end-of-grace-period and end-of-hardship records), and change-of-unit actions.

bAnnual reexaminations reflect actions recorded as "Action code 2: annual reexamination" on the 50058 form.

cMove/change of unit actions reflect actions recorded as "Action code 7: other change of unit" on the 50058 form.

If a move was recorded through an annual or interim action, it is not reflected in this outcome.

dInterims reflect all actions recorded as "Action code 3: interim reexamination" on the 50058 form except interim reexaminations to end a grace period or hardship rent. Types of interim actions are not mutually exclusive. Any action counts as each action once. At the same interim certification event, a household may have reported changes in its situation that fell into more than one of the categories displayed in this table.

eHouseholds in the existing rent rules groups in Louisvil e and Washington, D.C. were not subject to a minimum rent. Thus, there was no hardship exemption available to them. This only includes hardships received through an interim recertification.

fThis outcome indicates a decrease in income that occurred at the same time that household composition changed.

When household members are removed, so is their income.

gInterims with increased income include interims in which income increased but TTP did not increase. This occurs when a family’s earnings did not increase enough for TTP to be over the minimum rent.

hThe control group often has contract rent changes included in their annual reexamination, and in that case the contract rent increase is not included in this category.

iOther actions include interims (or some other reason but not end of grace or hardship), which are difficult to clas-sify from the available data.

*** = 1 percent. ** = 5 percent. * = 10 percent.

Notes: Estimates were regression adjusted using ordinary least squares, controlling for pre-random assignment characteristics of sample members. Differences between the new rent rules group and the existing rent rules group were assessed using a two-tailed t-test for continuous variables and selected outcomes expressed as proportions. For categorical variables a chi-square test was used to determine whether there is a difference in the distribution of related outcomes for the new rent rules group compared with the existing rent rules group. The p-value indicates the likelihood that the difference between the new rent rules group and the existing rent rules group arose by chance. When categorical variables are part of a distribution, the statistical significance levels, which apply to the distribution, are shown above the distribution. Rounding may cause slight discrepancies in calculating sums and differences.

Source: MDRC calculations, using PHA data

88

Exhibit 4.9 also shows that not al control group families in Lexington, Louisvil e, and San Antonio completed an annual recertification by the end of Year 1, even though they are required under traditional HUD rules. In most of those cases, the recertification occurred later than the 1-year anniversary date because of normal scheduling delays or processing delays. In other cases, the family moved to another unit, and the ful income review conducted by the PHA when it processed that move (a “move action”) substituted for the annual recertification.

In a small number of cases, the families had exited the voucher program or moved to another PHA. 68

The reduction in the number of interim recertifications caused by the new rent policy for increased income was largest in Louisvil e (30.8 percentage points) despite the opt-out option for the new rules group. This is because a relatively high proportion of families in the control group experienced earnings increases during Year 1, which they were required to report to the PHA. In Louisvil e, 34.1 percent of control group families received an interim recertification for increased income, compared with 22.8 percent in Lexington, 1.8 percent in San Antonio, and 11.4 percent in Washington, D.C. Those rates were distinctively lower in San Antonio, which did not require control group families to report interim income increases during Year 1, and in Washington, D.C., which instituted a biennial recertification policy for the control group.

Conclusion

As this chapter shows, the new rent policy had already begun to change families’ experiences with the Housing Choice Voucher program during the first year that the policy was in effect.

Relative to the existing rent rules, the new policy reduced the likelihood of paying zero TTP or very low TTPs, as a result of the introduction of or increase in minimum TTPs in three of the four PHAs (excluding Lexington, which instituted the same high TTP for both research groups).

On average, however, the new policy reduced families’ monthly payments for rent and utilities relative to the control group’s payments and increased the likelihood of being on the voucher program by the end of the first year, and those effects led to an increase in the average amount of subsidy received, which was intended by the policy design to reward work. The new policy also reduced the likelihood of certain transactions with staff, especially the time-consuming ones that involved income reviews for adjustments to TTPs and subsidies (for example, annual recertifications and interim recertifications as a result of increases or decreases in family income). Those patterns vary somewhat across the four PHAs, in part because of differences in 68A separate analysis (not captured in any tables in this chapter) showed that approximately 91

percent of the control group members in the three sites that were subject to an annual recertification schedule either completed an annual recertification, were searching for a new unit, had a move action completed, or had exited the voucher program or transferred (“ported out”) to another PHA during the first year following enrollment. This separate analysis also revealed that about 2 percent of the control group in these sites appeared to have had an interim recertification in place of an annual recertification in the first year. PHA data on recertifications are analyzed using the effective date rather than the date that staff enter a recertification action into the software system. Thus, late recertifications would be missing from tables in this report if the effective dates occurred after the expected 12-month period for annual recertifications or if PHA staff entered information on effective dates into the system after the data were sent to MDRC.

89

the existing policies applied to the control group. Overall, however, they are early signs of trends that are likely to grow over Years 2 and 3 of the followup period, while the new policy’s TTP cap and restrictions on the number of interim recertifications remain in effect, until families reach their triennial recertifications and, if they are stil receiving vouchers, have their TTPs recalculated and capped for the subsequent 3 years.

90

Chapter 5

Summary and Next Steps in the Rent Reform

Demonstration

The early impact findings discussed in this report offer initial but limited findings on the new rent policy designed as part of HUD’s Rent Reform Demonstration. Although it is too soon to draw firm conclusions about the merits or disadvantages of this new policy, some important patterns have begun to emerge concerning its effects on tenants’ labor market and housing assistance outcomes. Those patterns point to the potential of the policy to make a difference for important labor market and housing outcomes, but the early effects are not always consistent across the four public housing agencies (PHAs) included in this study—in Lexington, KY; Louisvil e, KY; San Antonio, TX; and Washington, D.C.

Exhibit 5.1 summarizes key impacts—that is, the dif erences in outcomes between the new and existing rent rules groups—within each of the four PHAs, for all of them combined, and for the combined set excluding Washington, D.C. As explained in earlier chapters, because the existing policy in Washington, D.C., included biennial recertifications, the early results in that PHA do not reflect the effects of extending the recertifications beyond the traditional annual recertification schedule (because neither research group was subject to an annual schedule).

As il ustrated in exhibit 5.1, the strongest indications that the new rent policy has the potential to improve tenants’ earnings (by a small or moderate degree) come from Lexington and San Antonio. In both PHAs, the new rent rules led to positive effects on heads of household’s average earnings over the first 18 months of fol owup. It is likely that these effects primarily reflect the extension of the recertification policy from an annual to a triennial schedule. This policy reduces the implicit marginal “tax” on any increased earnings from 30

percent to zero during the 3-year period.

91

Exhibit 5.1. Summary of Early Impacts on Employment, Earnings, and Housing Subsidy-Related Outcomes

All PHAs Except

San Washington,

Outcomes

All PHAs Washington, D.C. Lexington Louisville Antonio

D.C.

Household heads

Average quarterly

employment, Year 1a (%)

1.6 **

1.9 **

1.7

0.9

3.2 **

0.2

Total earnings ($)

Year 1 (quarters 3-6)

263

466 **

466

29

916 ***

-368

Quarter 7

79

83

298 *

-252 **

324 ***

72

Quarter 8

116

162 **

352 **

-181

392 ***

0

Sample size

6,665

4,756

979

1,908

1,869

1,909

Households

Average annual housing

subsidy in Year 1b ($)

258 ***

316 *** 359 **

239 *

419 ***

258

Any action that requires

staff responsec (%)

-23.9 ***

-28.9 *** -44.7 *** -0.6

-49.5 ***

-12.0 ***

Sample size

6,665

4,756

979

1,908

1,869

1,909

aAverage quarterly employment is calculated as total number of quarters with employment divided by total number of quarters of followup, expressed as a percentage.

bHousing subs idy is the full subs idy amou nt paid by the

ho

usi ng agenc y and inc

lude

s any ut ility allow-

ance paym

ents made to the tena nt in addi

tion

to rent

paid to the

owne r by t he hous

ing age

nc

y.

cCertification actions that require staff interaction include annual reexaminations, interim reexaminations (except for end

-of-grace

-pe riod and

end- of-hardship records ), and c hange-of-unit ac tions.

* = 10 percent. ** = 5 percent. *** = 1 percent.

Notes: Estimates were regression adjusted using ordinary least squares, controlling for pre-random as-

signment characteristics of sample members. Rounding may cause slight discrepancies in calculating sums and difference s. A tw

o-tailed t-test was ap

plied to differen

ces between research gro

up s. The p-valu e

indicates the like

lihood that th e differenc e between the new re nt rul es group

and the existing re nt rules

group arose by chance.

S ample sizes for specific ou tcom es m ay vary becau

se of mis sing valu es. The va ri-

ation acros s the four PH As in estim

ate d impacts on to tal earnings in Year 1 is sta tistic ally signif icant at the 10- percen

t level based

on an H-s tatistic te st. The v ariation across th e four PH As in estim ated imp acts on average annual housing subsidy in Year 1 is not statistically significant based on an H-statistic test.

Confirm

atory out comes wer e tes ted for m ultiple hyp othesis te

sting

using the Benj

amini-H ochberg pr oce-

dure. The

adj

ust ed p-value = .182 for the

impac t o n tot

al Year 1 earni ngs for all four

PHAs com

bine d.

The adjus ted p-value = .035 for the

impac t on tot

al Year 1 earnings for al l PHAs combine d excludi ng

Washingt

on,

D. C. T he a djusted

p- value = .016

for the im pac t on

the total Y ear 1 housi ng subs

idy for all

four PHA

s com bine d. T he adjus

ted

p-va lue = . 000 for the im pa ct on the

total Ye ar 1 housing

subsidy for

all PHAs combined excluding Washington, D.C.

Source: MD RC calcu

lat ion s, usin g quarterly wage data

from the Nati onal Directory of New Hires

92

At the same time, it is unlikely that the positive effects in those two locations reflect the new policy’s minimum total tenant payment (TTP) requirement. Lexington had already instituted a minimum TTP (at $150 per month) before the demonstration began, which applied to both research groups. Consequently, that feature cannot account for any of the difference in earnings that subsequently emerged between the two research groups. The findings from Washington, D.C., are also instructive here. Although that PHA does not yet offer a good test of the effects of extending the recertification period under the new rent rules (because of its biennial recertification schedule for the control group), it does offer an opportunity to assess the early effects of a $75 minimum TTP for the new rules group because the control group faced no minimum TTP. That feature creates a greater incentive for families in the new rules group to increase their earnings, even in the short term. Thus, the absence of any short-term earnings effects in Washington, D.C., suggests that a minimum TTP, by itself, may not necessarily generate increased work effort or earnings. This evidence is considered limited, however, because it comes from only one site and for a minimum TTP of a particular dollar value.

The absence of early positive effects in Louisvil e and evidence of a statistically significant negative earnings effect in one quarter add an important cautionary note. These results are complicated to interpret because, as explained in previous chapters, 22 percent of Louisvil e’s new rent rules group opted out of the new rent policy. Whether this circumstance, something about the local context or voucher population, or some other factors weakened the potential impacts of the new policy in the short term is not known. This issue will be explored further if the pat ern persists through the longer followup period, but the absence of early positive impacts does not mean that Louisvil e voucher holders were distinctly less attached to the labor market. Indeed, as shown in Chapter 3, both research groups increased their earnings steadily over the first eight quarters of fol owup.

The early impact findings also point to some tentative patterns that are worth watching as longer-term data become available. For example, although not consistently statistically significant, some patterns in the data suggest that impacts on earnings may be concentrated on household heads who were already employed in the quarter prior to random assignment.

Earnings effects may have also been concentrated among families whose youngest child was a teenager at the time of random assignment. Firm conclusions about the robustness of these patterns must await longer-term data.

The new rent policy produced some early impacts on housing-related outcomes that affected families’ housing subsidies, as well as the transactions between tenants and PHA staff.

For example, relative to the control group’s outcomes, the new policy caused a small reduction in families’ probability of exiting the voucher program in the first year and a small increase in the average amount of housing subsidies that families received. As exhibit 5.1 shows, average annual subsidy payments increased by $258 per family. This finding reflects an increase of 2.7

percent above the control group mean subsidy payment of $9,719 in Year 1. That the new rules group would receive more in subsidy payments than the control group early on was an expected 93

result, reflecting the intent of the new policy to help “make work pay” by allowing families to refrain from reporting income increases to the PHA for a 3-year period.

The new rent rules also began to reduce the most time-consuming transactions required by HUD’s traditional rent policy once the initial recertification was completed at the beginning of the study. Those transactions include annual recertifications and interim adjustments in TTPs and subsidies that normally had to be made as families’ incomes changed. As exhibit 5.1 shows, the new policy reduced the likelihood of any PHA actions with or on behalf of families during Year 1 by almost 24 percentage points for all PHAs combined and by almost 50 percent in San Antonio. At the same time, early qualitative feedback from PHA staff points to the extra efforts needed to col ect the information that new rent rules families must provide to calculate their retrospective income, which is the basis for setting TTPs under the new rent policy.

Al these patterns wil be important to reexamine as the longer-term data become available—and especially once families in the new rent rules group have completed their triennial recertifications. At that point, families whose earnings in the 12 months leading up to that recertification are higher than what they were at the initial recertification wil begin paying higher TTPs and receiving smaller housing subsidies, thus allowing the PHAs to begin recouping forgone savings in housing subsidy expenditures. Families’ new TTPs wil then be capped for another 3 years, allowing them to keep any further increases in earnings they achieve during that period.

The next report in the series on the Rent Reform Demonstration, “Interim Findings,” wil cover these same labor market and housing-related outcomes over a longer fol owup period, through at least the time of the triennial recertification and through the first two subsequent quarters for an early cohort of families. The “Interim Findings” report wil also cover the policy’s effects on a wider array of outcomes that are important to a ful assessment of the effects on families. Those effects include impacts on families’ receipt of other government transfer benefits (including Temporary Assistance for Needy Families, the Supplemental Nutrition Assistance Program, and homelessness assistance), based on administrative records data col ected from state and local agencies.

The fourth report in the series, expected in 2021, wil provide additional outcome data from a household survey through which the heads of household are being interviewed at approximately 3 and a half years after families’ initial TTPs (those set at the beginning of the study) took effect. 69 The survey covers a wide range of outcomes, including job characteristics, reasons for not working, family composition, total family income, family poverty, housing stability, relationships with landlords, savings, debt, financial practices, material hardships, and additional quality-of-life indicators.

The final report in the series, expected in 2023, wil present findings on families’

perspectives on the new policy through indepth qualitative interviews with a small sample of 69 The survey firm Decision Information Resources (DIR) is administering this 42-month followup survey.

94

household heads in the new rent rules group and use quantitative data from the full-sample survey. For example, information wil be col ected on families’ awareness and understanding of the incentive features of the new rent rules as well as its safeguards to protect them from hardship, how families have responded to those features, and their views on the fairness of the policy.

Finally, the evaluation is continuing to col ect data on the PHAs’ experiences in implementing the new policy from its inception. Those data include information obtained from indepth qualitative interviews with staff at each PHA about advantages and challenges in administering the new policy from their perspective, particularly in comparison with HUD’s traditional rent rules. Those data, along with further analysis of the policy’s effects on PHA staff actions, wil provide insights into how well the goal of administrative simplification and cost reduction have been achieved.

95

Appendix A: Supplementary Materials for Chapter 1

Exhibit A.1. Existing Rent Policies of Public Housing Agencies (PHAs) Participating in the Rent Reform Demonstration Rent-Policy Component

Lexington

Louisville

San Antonio

Washington, D.C.

Percentage of adjusted income for

total tenant payment (TTP)

30

30

30

30

$5,000; if assets total more than this amount,

Threshold of asset value below

income from the assets is “imputed” and the

None; self-certification

which asset income is ignored

greater of actual asset income and imputed

None. None; self-certification of assets sold for

less than fair market value.

of individual assets

asset income is counted in annual income.

less than $15,000.

Working age or nondisabled: biennial

Working age or nondisabled: annual.

Working age or

Working age or

nondisabled: annual.

for some, annual for Rent Reform nondisabled: bienniala.

Recertification

Elderly or disabled (on fixed income):

Demonstration control group.

triennial [proposed].

Elderly or disabled:

Elderly or disabled:

biennial.

Elderly or disabled (on fixed income):

biennial [triennial proposed].

biennialb.

Minimum TTP

$150

$0

$50

$0

Uses the appropriate utility allowance for the

size of dwelling unit actually leased by the

Simplified by bedroom

Utility policy

family (rather than the family-unit size as

Current HUD

and voucher size

determined under the housing authority

policy.

Current HUD policy.

[planned].

subsidy standards).

Suspension of minimum rent if a household

If the TTP calculated at recertification is

experiences an increase in rent as a direct

lower than the minimum TTP, a

Hardship policy for minimum rent

result of the Moving to Work Rent Reform

hardship exists, and the family share is

Demonstration; reduction in rent if a

[No minimum rent]. calculated at the highest of 30 percent of

[No minimum rent].

household experiences a loss of income due

gross income, 10 percent of adjusted

to circumstances beyond the family’s control.

income, or the welfare rent.

a Starting in June 2016, income increases did not have to be reported between biennial recertifications. Before June 2016, a family had to report an increase in income even if it occurred before the family’s next scheduled biennial recertification. If the increase was $10,000 or more, then the housing agency calculated a new TTP. If the increase was less than $10,000, then this income was excluded until the next biennial recertification.

b Starting in September 2016, disabled and fixed-income families were on a triennial recertification.

Notes: Current HUD utility policy is based on typical utilities costs in housing of similar size and type, on community consumption patterns, and on current utility rates.

Source: Housing agency Moving to Work annual plans and other agency documents 96

Appendix B: Supplemental Materials for Chapter 2

Strategy for Addressing Multiple Outcome Measures in the Rent

Reform Demonstration

When an impact evaluation includes many different outcome measures—each of which implies a test of a different hypothesis about an intervention’s effect—the possibility exists that some impact estimates may emerge as statistically significant simply by chance and do not reflect true intervention effects. For example, if 10 outcomes are examined in a study of ineffective treatment, it is likely that the impact on one of them wil be statistically significant at the 10

percent level only by chance. As the number of outcome measures expands, the number of

“false positive” results may also increase. This section of Appendix B describes the approach adopted for the Rent Reform Demonstration to address this problem.

Confirmatory Outcome Measures

Addressing the issue of multiple outcome measures begins with specifying a small set of primary outcome measures—that is, the most important measures for assessing the effectiveness of the intervention, commonly referred to in the evaluation literature as

“confirmatory” outcomes. Given their primacy, impact findings on confirmatory outcomes are subjected to further statistical adjustments (beyond basic tests of statistical significance) that hold them to a higher standard of evidence. One can have more confidence in any confirmatory impact estimates that remain statistically significant after adjusting for the total number of confirmatory outcome measures (the exact adjustment method is discussed later, under

“Adjustment Measures”).

For the Rent Reform Demonstration, three outcomes have been designated as confirmatory: (1) household heads’ average cumulative earnings, (2) families’ cumulative housing subsidies (that is, housing assistance payments, or HAP), and (3) a measure of material hardship for the household head and family. Increasing tenants’ earnings is one of the most important goals that HUD set for the new rent policy, but it is important to achieve that goal without making the voucher program substantially costlier than it would be with the traditional rent policy. The new rent policy would be a far less compelling policy option if it achieved positive effects on tenants’ earnings only at the expense of large increases in housing subsidies (the main driver of voucher program costs). A policy that both increased tenants’ earnings and kept subsidy costs the same or reduced them would hold much more appeal.

Effects on average earnings and average HAP subsidies also hold primary importance because they can drive other important outcomes. For example, it is unlikely that the new rent policy would have substantial effects on benefits receipt, financial security, family moves, and other dimensions of family well-being in the absence of any impacts (positive or negative) on either average earnings or HAP. Though stil very important, these other measures might be considered deriving largely from the effects on earnings and HAP.

97

Also supporting their designation as confirmatory variables, average earnings, and average HAP have the advantage of being more inclusive variables within their respective domains in the sense that they capture other important effects within those domains. For example, effects on average earnings capture effects on employment rates, steadiness of employment, wage rates, and hours. Effects on average HAP capture effects on how much tenants contribute to their rent and utilities and how long they remain in the voucher program.

Impacts on average earnings and HAP are too narrow, by themselves, for understanding the effects of the new rent policy on overall family well-being. For example, an increase in earnings and a reduction in subsidies may suggest progress toward self-sufficiency, but even with those results, some families might experience an increase in material hardships, particularly in the face of subsidy reductions, which could severely diminish the policy appeal of the new rent rules. Because another major goal of the new policy is to promote increased work without causing an increase in hardships that families experience, a composite measure of

“material hardship” has been designated as the study’s third confirmatory outcome measure.

The evaluation’s survey of tenants (to be administered in 2019) includes several items on family hardships. Hardship is a complicated construct to measure because it has many dimensions, and survey measures involve subjective assessments and respondents’ self-reports. 70 Moreover, the degree of intensity of any given hardship and the difficulty it poses for a family, relative to other types of hardships, is hard to gauge. Several types of hardships, however—including inadequate ability to purchase food, difficulty paying rent and utilities costs, and difficulty sustaining access to telephone service—are broadly recognized as fundamental hardships for anyone and indicators of precarious economic circumstances. They are commonly included in studies of anti-poverty programs that seek to measure effects on material hardship. Some studies point to a tendency among families to make tradeoffs between food purchases and paying their rent or utilities to make ends meet. The current study wil construct a composite measure of hardship composed of several items included in the survey covering different types of hardship and the duration of those hardships (see below for further information on the construction of this measure). 71

Cumulative Versus Annual Measures of Earnings and HAP

In interpreting the policy implications of the new rent policy’s effects on earnings and HAP, it is important to consider the different stages of the study’s followup period. As followup years accumulate, cumulative measures, in addition to the annual measure, take on more importance.

For example, some past studies of labor market interventions have found that early impacts on earnings faded over time as the control group caught up. Other studies have found that impacts emerged only later in the fol owup period, as participants began to experience a payoff from 70See Ouellette et al. (2004) for an extensive review of measures of material hardship.

71The original Rent Reform Data Collection and Analysis Plan (RRDCAP) underscores the prime importance of average earnings and HAP in the evaluation (MDRC, 2016); however, the original evaluation workplan did not include a survey, which HUD subsequently funded. Consequently, effects on survey-based hardship measures were not included in the original RRDCAP plan.

98

acquiring new skil s through the intervention or as the intervention itself matured and strengthened.

In the Rent Reform Demonstration, the extension of the recertification schedule from an annual to a triennial schedule—the new policy’s main vehicle for increasing the financial incentive to work—elevates the importance of cumulative impact estimates. This feature reduces the voucher program’s normal 30-percent implicit tax on earnings to zero by capping families’ total tenant payments (TTPs)—but for a specific block of time: the 3-year period until the next triennial recertification. After the triennial, a new cap is set for the next 3-year block of time, and so on. Thus, in judging the overal success of the new policy, its effects on cumulative earnings during the period in which the 3-year reduction in the marginal tax rate was in effect are more important than its effects on earnings in any single year. For this reason, the confirmatory earnings measure is defined in terms of cumulative rather than annual earnings.

Similarly, a cumulative perspective is important in considering the policy’s impacts on HAP, which wil be a key determinant of whether the policy is cost neutral relative to the existing rent rules. As a statistical modeling exercise completed as part of the design of the new rent policy showed, the impacts on HAP were predicted to be positive initially (that is, yield higher subsidies for the new rent rules group) prior to the triennial, but then flip direction after the triennial, resulting either in no higher or only modestly higher HAP expenditures cumulatively relative to the control group or possibly some HAP reductions (depending on the magnitude of the earnings effects). 72 Thus, the cumulative HAP expenditures are much more directly relevant to the policy goal of “cost neutrality” or “cost savings” than are annual HAP

expenditures. For that reason, cumulative rather than annual HAP is deemed a confirmatory variable.

Because the Rent Reform Demonstration is a multiyear initiative with sequential reports on findings as they become available, the latest available data on cumulative measures should supersede in importance any earlier evidence of effects on those same measures. In that sense, interim findings should be considered preliminary indicators of the effects on important confirmatory variables. They should not be the basis for making policy decisions. They are more akin to exploratory outcomes and wil not carry the same weight in assessing the policy’s effects as those presented in the final report, which will present the main findings on the study’s confirmatory hypotheses. Nonetheless, although they may be considered “preliminary” or

“exploratory” hypotheses, there is value in subjecting them to multiple hypothesis testing, given their central importance to the evaluation of the new rent policy. These adjustments can help establish the degree of certainty to attach to the interim estimates (of course, in Year 1 of the fol owup period, cumulative and annual earnings measures are the same).

72MDRC (2016).

99

Adjustment Method

Weighing the strengths and weaknesses of different statistical tests to adjust for multiple hypothesis testing and considering some of the limitations of using National Directory of New Hires (NDNH) data, the evaluation team decided to adopt the Benjamini-Hochberg method, applied to p-values. 73 The Westfall-Young method would have been preferable because it accounts for correlations among outcomes; however, that method would require that all outcomes are available on the same dataset. NDNH data come in a de-identified form and can be linked with other data only through a pass-through file submit ed to the Office of Child Support Enforcement (OCSE). Moreover, variables on the pass-through file are not in the same form as they are on MDRC’s network because OCSE requires that variables be made categorical and not have small cell sizes. For example, in the pass-through file, monthly HAP in Year 1 is a categorical variable with seven categories rather than the continuous measure used in the impact analysis. It is important to base the multiple hypothesis adjustment on variables that exactly match the specification of those variables in the impact analysis.

The Number of Hypotheses

The evaluation design further specifies confirmatory outcomes as those pertaining to the pooled sample of the study’s four public housing agencies (PHAs). Although the same measures are included in PHA-specific analyses, the latter analyses are considered exploratory. The analysis plan did not include hypotheses predicting a particular pattern of variation in impacts by PHA, although it did call for exploring possible factors contributing to any substantial variation in effects observed across those sites (see below for a further discussion of the PHA-specific analysis strategy). Because the PHA-specific estimates are considered exploratory, and because the pooled estimates already incorporate (are built up from) the PHA-specific impacts, the multiple hypothesis adjustment for the pooled sample wil not adjust for the site impact estimates.

For the current report, the multiple hypothesis adjustment for the pooled sample wil be applied to the estimated impacts on average pooled earnings impact and average pooled HAP in Year 1. For the second impact report, the adjustments wil be applied to cumulative pooled earnings impacts (through the latest quarter available for each report) and average cumulative HAP impacts (through the latest month available). In the second and final impact reports, adjustments for the latest available cumulative impact estimates wil not adjust for the prior cumulative impact estimates covering shorter fol owup periods. The latter wil constitute preliminary results, not distinct hypotheses.

73See Benjamini and Hochberg (1995). P-values of confirmatory outcomes will be ordered from smallest to largest and assigned a rank (a rank of 1 will be assigned to the smallest p-value, and a rank equal to the number of tests will be assigned to the largest p-value). Starting with the largest p-value, an adjusted p-value is calculated as the (Number of Tests / Rank) * unadjusted p-value. If the adjusted p-value is equal to or less than .10, then that outcome measure and all outcome measures with lower p-values are statistically significant after adjusting for the false discovery rate.

100

The final report wil include estimates of impacts on longer-term earnings and HAP

outcomes and on the composite hardship measure. Each of these estimates wil thus be adjusted for a total of three confirmatory outcomes.

Pooled Estimates Excluding Washington, D.C.

The evaluation includes a set of pooled estimates without Washington, D.C. This study is considered an exploratory analysis because it represents a change in analysis plans after the evaluation was launched. The rationale for excluding this PHA from the pooled estimates is based on its decision—after the launch—to expand access to its existing biennial recertification policy for the voucher population to all families in the control group. (Before it took that action, the biennial policy was to apply only to families with quite low earnings, and it was expected that most control group members would stil be subject to the traditional annual recertifications.) This decision meant that the effects of the new rent policy’s most important feature for increasing work incentives —extending the recertification period—could not be tested in Washington, D.C., for the interim report because both groups experienced an extension during the early followup period. Thus, a better-pooled estimate of the new policy’s early effects on earnings is arguably one based on a pooled sample that excludes Washington, D.C. 74

Although the four-PHA pooled sample is used to estimate confirmatory impacts, and although exploratory impacts are not normally subject to adjustments for multiple hypotheses, the special importance of the three-PHA pooled sample in assessing the overall effectiveness of the new rent rules makes it prudent to hold the results from that sample to a higher standard of evidence. Thus, the results for the three-PHA pooled sample wil also be adjusted for multiple hypothesis testing. Because of the obviously high overlap between these two samples, the adjustments of the impact estimates for the three-PHA sample wil not take into account the impact estimates of the four-PHA sample and vice versa.

PHA-Specific Analyses

The PHA-specific analyses are being approached differently. Here, the focus is on whether any substantial variation observed across PHAs in the new rent policy’s impacts on the primary outcomes is likely to be due to chance. As planned in the RRDCAP, and as is common practice in multisite evaluations, the evaluation calculates the statistical significance of the dif erences in impacts across sites. Here, the H-statistic is applied to each of the confirmatory outcome measures (this approach is the same as the one that is commonly used for subgroup analyses). If cross-PHA differences in impacts are statistically significant, one would have more confidence concluding that the policy was more effective in some PHAs than in others. This approach is a 74This strategy was not a post hoc idea. MDRC had first proposed pooling without this PHA in an early draft of the research design paper it submitted to HUD on May 29, 2014, well before any outcome data were available. The decision to produce a pooled estimate without Washington, D.C., as an exploratory outcome was finalized when preparing the draft interim report, when it was recognized that the PHA had dropped its $10,000 threshold policy and extended the biennial recertification policy to all members of the control group.

101

way to address what might be considered a “first-order” question for PHA-specific analysis—

whether to consider the site-by-site variation in effects meaningful.

Subgroups

The approach used for the PHA-specific analyses is also being fol owed for the confirmatory subgroup analyses. This approach involves computing the H-statistic to assess the statistical significance of the dif erence in impacts across subgroup categories on a primary outcome measure. For example, this method is used to assess whether a difference in the new policy’s impacts on, say, average earnings for sample members “working at baseline,” and those “not working at baseline” is itself statistically significant.

Constructing the Hardship-Related Primary Outcome Measure

As mentioned previously, the analysis wil use a composite measure of material hardship as a third confirmatory outcome (the final evaluation report wil present the results). This measure wil incorporate survey questions related to food, shelter, recurring monthly utility and phone bil s, and medical care. 75 The specific questions in the survey include the fol owing:

• In the last 12 months, was there ever a time when [you/you or your household] did not pay the full amount of the rent?

• In the last 12 months, has there been a time when [you/you or your household] had service turned off by the gas or electric company, or the oil company would not deliver oil because payments were not made?

• In the last 12 months, have [you/you or your household] had cellular or land service disconnected because payments were not made?

• In the last 12 months, was there ever a time when, because of cost, [you/you or your household] were not able to buy food?

For each of these questions, a respondent who gives a positive response is asked a fol owup question to gauge the general frequency of the problem:

• In the last 12 months, about how many months has this happened? (Response categories include 1 month, 2-3 months, 4-6 months, or 7 months or more.) The survey also asks about unmet medical care in the prior year:

• In the last 12 months, was there ever a time when, because of cost, [you/you or your household] were not able to buy prescription medicine?

75These items are commonly used in national surveys—such as the Survey of Income and Program Participation, the American Housing Survey, and the National Health Interview Survey—to measure material hardships.

102

• In the last 12 months, was there ever a time when, because of cost, [you/you or members of your household] were not able to see a doctor or get medical assistance?

Using the responses to the preceding questions, a cumulative hardship scale wil be constructed to reflect the average number of months of reported hardships within the last 12

months. For the unmet medical care and prescription items, for which the fol owup duration question is not asked, a positive response (“yes, there was a time”) wil be counted as “1

month.” This is considered appropriate because prescriptions and doctor visits are not necessarily monthly needs.

Overall, a respondent experiencing none of the measured hardships would earn a score of zero. Higher scores wil reflect a higher number of enduring hardships. A person experiencing only one of these hardships, and in only one month, would be assigned a score of 1; a person experiencing one hardship in “2–3 months” would be assigned the midpoint of that range, or 2.5 months; someone experiencing all four hardships, and each one for “2–3 months,”

would be assigned a score of 10 (that is, 4 × 2.5); and so on. The analysis wil compute an average composite score for each of the two research groups and estimate the impact of the new rent rules on that score as the confirmatory test of the new rent policy’s overall effect on hardship.

The exploratory analysis wil examine the policy’s effects on each of the items in the scale separately. It wil also examine effects on several other outcomes related to financial well-being and food security.

103

Exhibit B.1. Employment and Earnings in the Three Quarters Before

Random Assignment, by Public Housing Agency (PHA): Heads of

Households

New

Existing Difference

Outcomes

Rent Rules Rent Rules

(Impact)

P-Value

Lexington

Ever employed (%)

Third quarter before random assignment

55.7

57.1

-1.4

0.544

Second quarter before random assignment

59.9

56.6

3.3

0.110

Quarter before random assignment

57.6

57.4

0.2

0.926

Total earnings ($)

Third quarter before random assignment

1,757

1,898

-141

0.163

Second quarter before random assignment

1,938

1,802

135

0.106

Quarter before random assignment

1,886

1,811

75

0.425

Sample size (total = 979)

486

493

Louisville

Ever employed (%)

Third quarter before random assignment

55.9

55.8

0.1

0.954

Second quarter before random assignment

56.4

54.2

2.3

0.126

Quarter before random assignment

55.2

55.1

0.1

0.932

Total earnings ($)

Third quarter before random assignment

1,793

1,813

-20

0.772

Second quarter before random assignment

1,894

1,878

16

0.794

Quarter before random assignment

1,897

1,918

-22

0.750

Sample size (total = 1,908)

947

961

San Antonio

Ever employed (%)

Third quarter before random assignment

54.3

54.3

-0.1

0.966

Second quarter before random assignment

56.2

54.6

1.6

0.194

Quarter before random assignment

55.2

57.5

-2.3

0.104

Total earnings ($)

Third quarter before random assignment

1,923

2,046

-123 **

0.048

Second quarter before random assignment

2,092

1,984

109 **

0.026

Quarter before random assignment

2,127

2,077

50

0.426

Sample size (total = 1,869)

935

934

(continued)

104

Exhibit B.1 (continued)

New

Existing Difference

Outcomes

Rent Rules Rent Rules

(Impact)

P-Value

Washington, D.C.

Ever employed (%)

Third quarter before random assignment

44.7

44.9

-0.2

0.876

Second quarter before random assignment

46.8

47.0

-0.3

0.817

Quarter before random assignment

47.7

46.3

1.5

0.222

Total earnings ($)

Third quarter before random assignment

2,486

2,396

90

0.278

Second quarter before random assignment

2,834

2,861

-28

0.711

Quarter before random assignment

2,711

2,610

101

0.154

Sample size (total = 1,909)

944

965

* = 10

pe rcent. ** = 5 pe rc ent. *** = 1 pe

r ce nt.

Notes: Among heads of households in the new rent rules group, 28 participants from Washington, D.C., and one from Loui

sville ar e missing dat a f or a t lea st one pre- random as signment qua rter. A mong heads of

household in the exis ting r ent r ules group, 51 p artic ipants fr om W

ashi

ngt

on, D.C ., two from L ouisville,

and one from San Anto nio are m

issing

da ta for at lea st one pre-rand om assign ment qu ar ter. Estimates were re gression adjus ted us ing or

dina ry least squa

res, cont rolling

for pre-rand om assignm ent ch aract

er

is-

tics of sampl

e members . Rounding m ay cause slight disc repanc

ies in calculating sum

s and

differenc

es. A

two-tailed t-test was ap plied

to differen ces between research groups . The p-va lue indicat es the likeli hood that th e differ ence b etween the new

rent rules grou

p an d the existing rent rul

es group arose by cha

nce.

Sample si zes for specific outcome s may var y becaus e of missing values.

Source: MDR

C calcul atio ns, us ing quarter ly wag e dat a f rom the Natio nal D irector y of N ew Hires

105

Exhibit B.2. Employment and Earnings in the Three Quarters Before Random Assignment, by Public Housing Agency (PHA): Adults Who Were

Not Heads of Households

New

Existing Difference

Outcomes

Rent Rules Rent Rules

(Impact)

P-Value

Lexington

Ever employed (%)

Third quarter before random assignment

59.7

53.5

6.2

0.172

Second quarter before random assignment

59.1

57.0

2.1

0.585

Quarter before random assignment

55.5

59.6

-4.1

0.337

Total earnings ($)

Third quarter before random assignment

1,168

1,158

10

0.941

Second quarter before random assignment

1,236

1,299

-62

0.609

Quarter before random assignment

1,401

1,345

56

0.684

Sample size (total = 296)

131

165

Louisville

Ever employed (%)

Third quarter before random assignment

46.5

45.2

1.2

0.652

Second quarter before random assignment

48.6

49.9

-1.4

0.584

Quarter before random assignment

50.1

51.1

-1.0

0.713

Total earnings ($)

Third quarter before random assignment

945

898

48

0.617

Second quarter before random assignment

1,082

1,117

-35

0.647

Quarter before random assignment

1,128

1,175

-47

0.608

Sample size (total = 815)

429

386

San Antonio

Ever employed (%)

Third quarter before random assignment

46.6

45.6

1.0

0.695

Second quarter before random assignment

48.8

47.7

1.1

0.627

Quarter before random assignment

50.5

53.3

-2.8

0.283

Total earnings ($)

Third quarter before random assignment

1,332

1,213

119

0.155

Second quarter before random assignment

1,362

1,407

-45

0.510

Quarter before random assignment

1,526

1,585

-59

0.505

Sample size (total = 784)

412

372

(continued)

106

Exhibit B.2 (continued)

New

Existing Difference

Outcomes

Rent Rules Rent Rules

(Impact)

P-Value

Washington, D.C.

Ever employed (%)

Third quarter before random assignment

34.9

33.3

1.5

0.343

Second quarter before random assignment

34.5

35.4

-0.9

0.498

Quarter before random assignment

37.0

35.2

1.7

0.227

Total earnings ($)

Third quarter before random assignment

1,104

1,127

-23

0.685

Second quarter before random assignment

1,230

1,244

-14

0.751

Quarter before random assignment

1,229

1,148

81

0.111

Sample size (total = 1,502)

765

737

* = 10

pe rcent. ** = 5 pe

r cent. *** = 1 pe

r ce nt.

Notes: Among nonheads of household in the new rent rules group, 112 participants from Washington, D.C., and one participan t from each of the other th ree

sit es are miss ing dat a for a t lea st one pre-ran dom as signment quarter. Among nonhe ads of

ho usehold in t he exi sting rent rul es group,

127 part icipa nts from Was hingt

on,

D.C., thr

ee f rom Louisvi l e, and one fr om San Antonio a re missing da ta for at leas t one pr e-random

ass ignment

quarter. Estima tes w

ere regression

adj usted using ord inary least squa

res , controlling for pr

e-ra ndom

assignment char

acteristi cs of sam ple mem bers . Rounding m ay cause slig ht discrep ancies in ca lcula ting sums

and differences. A two- tailed t-test was applied

to differ ences between

research groups

. The p-val ue indicates

the likelihood that th e differe nce betwee n the new re nt rul

es group

and the existing rent r ules group ar ose by

chance.

Sample sizes

f or specific outcom es m ay vary be cause of mis sing value s.

Source: MDRC calculations, using quarterly wage data from the National Directory of New Hires 107

Appendix C: Supplementary Materials for Chapter 3

Exhibit C.1. Treatment-on-Treated (TOT) Early Impacts for Selected Outcomes, Louisvil e

New

Existing Difference Impact per

Outcome

Rent Rules Rent Rules

(Impact) Participant

Employment and earnings for the head of household

Ever employed (%)

Year 1 (quarters 3–6)

71.8

72.1

-0.3

-0.4

Quarter 1 (random assignment)

52.8

55.1

-2.3

-3.0

Quarter 2

57.8

58.8

-1.0

-1.3

Quarter 3

60.1

59.9

0.2

0.3

Quarter 4

59.7

58.7

1.0

1.3

Quarter 5

59.8

59.3

0.5

0.6

Quarter 6

63.2

61.3

1.9

2.4

Quarter 7

61.0

62.1

-1.2

-1.5

Quarter 8

60.7

62.1

-1.4

-1.8

Total earnings ($)

Year 1 (quarters 3–6)

10,113

10,083

29

38

Quarter 1 (random assignment)

1,931

2,027

-96

-124

Quarter 2

2,186

2,154

32

41

Quarter 3

2,423

2,392

31

40

Quarter 4

2,460

2,469

-8

-11

Quarter 5

2,499

2,440

59

76

Quarter 6

2,733

2,800

-67

-86

Quarter 7

2,775

3,027

-252

-325 **

Quarter 8

2,822

3,003

-181

-233

Employed in al quarters, Year 1 (%)

48.0

43.8

4.3

5.5 **

Average quarterly employment, Year 1a (%)

60.7

59.8

0.9

1.2

Housing subsidy

Average number of months received housing subsidy in Year 1b

11.6

11.4

0.2

0.3 ***

Average annual housing subsidy in Year 1 ($)

7,898

7,659

239

308 *

Public housing agency (PHA) actions

Ever had any action that requires staff response in Year 1c (%)

89.0

89.7

-0.6

-0.8

Annuald

24.8

65.1

-40.3

-51.9 ***

Average number of actions in Year 1

1.1

1.5

-0.4

-0.5 ***

Sample size (total = 1,908)

947

961

(continued)

108

Exhibit C.1 (continued)

aAverage quarterly employment is calculated as total number of quarters with employment divided by total number of quarters of followup, expressed as a percentage.

bHousing subsidy is the full subsidy amount paid by the housing agency and includes any utility allowance payments made to the tenant in addition to rent paid to the owner by the housing agency.

cCertification actions that require staff interaction include annual reexaminations, interim reexaminations (except for end-of-grace period and end of hardship records), and change-of-unit actions.

dAnnual reexaminations include all action type 2 certifications.

*** = 1 percent. ** = 5 percent. * = 10 percent.

Notes: "Impact per participant" refers to the difference between the new rent rules group and existing rent rules group means divided by the participation rate (0.776). Estimates were regression adjusted using ordinary least squares, controlling for pre-random assignment characteristics of sample members. Differences between the new rent rules group and the existing rent rules group were assessed using a two-tailed t-test. The p-value indicates the likelihood that the difference between the new rent rules group and the existing rent rules group arose by chance. Rounding may cause slight discrepancies in calculating sums and differences.

Source: MDRC calculations, using PHA data and quarterly wage data from the National Directory of New Hires

Exhibit C.2. Early Impacts on Employment and Earnings: Adults Who

Were Not Heads of Households

New

Existing Difference

Outcomes

Rent Rules Rent Rules

(Impact)

P-Value

Ever employed (%)

Year 1 (quarters 3–6)

69.5

69.4

0.1

0.933

Quarter 7

57.2

58.4

-1.2

0.442

Quarter 8

57.1

55.3

1.9

0.232

Average quarterly employment, Year 1a

(%)

51.6

51.9

-0.3

0.819

Total earnings ($)

Year 1 (quarters 3–6)

7,257

7,536

-279

0.269

Quarter 7

2,310

2,399

-89

0.328

Quarter 8

2,294

2,375

-82

0.373

Sample size (total = 3,397)

1,737

1,660

aAverage quarterly employment is calculated as total number of quarters with employment divided by total number of quarters of followup, expressed as a percentage.

* = 10 percent. ** = 5 percent. *** = 1 percent.

Notes: Estimates were regression adjusted using ordinary least squares, controlling for pre-random assignment characteristics of sample members. Rounding may cause slight discrepancies in calculating sums and differences. A two-tailed t-test was applied to differences between research groups. The p-value indicates the likelihood that the difference between the new rent rules group and the existing rent rules group arose by chance. Sample sizes for specific outcomes may vary because of missing values.

Source: MDRC calculations, using quarterly wage data from the National Directory of New Hires 109

Exhibit C.3. Early Impacts on Employment and Earnings, by Public Housing Agency (PHA): Adults Who Were Not Heads of Households

New

Existing Difference

Outcomes

Rent Rules Rent Rules

(Impact)

P-Value

Lexington

Ever employed (%)

Year 1 (quarters 3–6)

79.5

76.9

2.6

0.531

Quarter 7

69.1

56.6

12.5 **

0.013

Quarter 8

66.5

62.9

3.6

0.486

Average quarterly employment, Year 1a (%)

65.0

59.8

5.3

0.165

Total earnings ($)

Year 1 (quarters 3–6)

8,982

7,362

1,620 **

0.049

Quarter 7

2,704

1,961

743 **

0.013

Quarter 8

2,310

2,183

127

0.666

Sample size (total = 296)

131

165

Louisville

Ever employed (%)

Year 1 (quarters 3–6)

76.4

77.7

-1.3

0.608

Quarter 7

58.3

62.9

-4.6

0.140

Quarter 8

61.0

59.3

1.7

0.599

Average quarterly employment, Year 1a (%)

58.5

58.7

-0.2

0.927

Total earnings ($)

Year 1 (quarters 3–6)

7,049

7,474

-425

0.370

Quarter 7

2,024

2,262

-238

0.183

Quarter 8

2,104

2,230

-126

0.476

Sample size (total = 815)

429

386

San Antonio

Ever employed (%)

Year 1 (quarters 3–6)

72.7

69.9

2.8

0.343

Quarter 7

61.5

60.3

1.2

0.707

Quarter 8

60.5

56.4

4.1

0.210

Average quarterly employment, Year 1a (%)

58.3

58.2

0.1

0.966

Total earnings ($)

Year 1 (quarters 3–6)

8,574

9,136

-562

0.295

Quarter 7

2,406

2,561

-155

0.392

Quarter 8

2,492

2,531

-39

0.834

Sample size (total = 784)

412

372

(continued)

110

Exhibit C.3 (continued)

New

Existing Difference

Outcomes

Rent Rules Rent Rules

(Impact)

P-Value

Washington, D.C.

Ever employed (%)

Year 1 (quarters 3–6)

62.5

62.8

-0.4

0.882

Quarter 7

52.3

55.3

-2.9

0.218

Quarter 8

51.8

50.6

1.3

0.609

Average quarterly employment, Year 1a (%)

42.1

43.1

-1.0

0.597

Total earnings ($)

Year 1 (quarters 3–6)

6,357

6,813

-456

0.264

Quarter 7

2,338

2,500

-162

0.276

Quarter 8

2,266

2,441

-175

0.247

Sample size (total = 1,502)

765

737

aAverage quar terly em ployment is cal culated as tota l num ber of qua

rters wit h em ployment di vided by total

numbe

r of quarters of followu p, expressed as a percen

tag

e.

* = 10 percent. ** = 5 percent. *** = 1 percent.

Notes: Estimates were regression adjusted using ordinary least squares, controlling for pre-random assignmen

t charact eristics

of sample member s. Rou

nding m ay cau

se sligh t discrepan cies in calculati ng

sums and diffe rences. A two- taile

d t-tes

t was app lied to diff erences

bet ween research group s. The p-valu e

indicate s the likelihood tha t the differ ence betw een the ne w rent rules group and the exi sting rent

rul es

group ar ose by cha

nc e. Sample sizes for specific out

comes may

var y bec ause of mis sing val ues.

Source: MDR C calcu

lat ions, us ing quar terly wag e data f rom the Nation al Di rectory o f Ne w Hir es

Exhibit C.4. Early Impacts on Employment and Earnings: Household Heads and Other Adults

New

Existing Difference

Outcomes

Rent Rules

Rent Rules

(Impact) P-Value

Ever employed (%)

Year 1 (quarters 3–6)

68.6

67.6

1.0

0.179

Quarter 7

58.3

57.9

0.4

0.599

Quarter 8

57.7

56.0

1.7 *

0.054

Average quarterly employment, Year 1a (%)

54.5

53.5

1.0

0.124

Total earnings ($)

Year 1 (quarters 3–6)

9,188

9,121

67

0.667

Quarter 7

2,730

2,707

23

0.669

Quarter 8

2,726

2,679

46

0.412

Sample size (total = 10,062)

5,049

5,013

(continued)

111

Exhibit C.4 (continued)

aAverage quarterly employment is calculated as total number of quarters with employment divided by total number quarters of followup, expressed as a percentage.

* = 10 percent. ** = 5 percent. *** = 1 percent.

Notes: Estimates were regression adjusted using ordinary least squares, controlling for pre-random assignment characteristics of sample members. Rounding may cause slight discrepancies in calculating sums and differences. A two-tailed t-test was applied to differences between research groups. The p-value indicates the likelihood that the difference between the new rent rules group and the existing rent rules group arose by chance. Sample sizes for specific outcomes may vary because of missing values.

Source: MDRC calculations, using quarterly wage data from the National Directory of New Hires

112

Exhibit C.5. Early Impacts on Employment and Earnings, by Public Housing Agency (PHA): Household Heads and Other Adults

New

Existing Difference

Outcomes

Rent Rules Rent Rules

(Impact)

P-Value

Lexington

Ever employed (%)

Year 1 (quarters 3–6)

78.6

76.3

2.2

0.237

Quarter 7

66.4

61.9

4.6 **

0.050

Quarter 8

65.6

61.7

3.9

0.100

Average quarterly employment, Year 1a (%)

65.4

63.1

2.3

0.188

Total earnings ($)

Year 1 (quarters 3-6)

10,013

9,349

664 *

0.099

Quarter 7

2,845

2,425

420 ***

0.003

Quarter 8

2,748

2,435

313 **

0.025

Sample size (total =1,275)

617

658

Louisville

Ever employed (%)

Year 1 (quarters 3–6)

73.3

73.6

-0.3

0.830

Quarter 7

60.2

62.3

-2.2

0.187

Quarter 8

60.9

61.2

-0.3

0.864

Average quarterly employment, Year 1a (%)

60.1

59.4

0.7

0.578

Total earnings ($)

Year 1 (quarters 3–6)

9,192

9,302

-110

0.707

Quarter 7

2,548

2,800

-252 **

0.015

Quarter 8

2,610

2,770

-160

0.140

Sample size (total = 2,723)

1,376

1,347

San Antonio

Ever employed (%)

Year 1 (quarters 3–6)

71.6

68.4

3.2 **

0.030

Quarter 7

61.4

59.1

2.4

0.162

Quarter 8

60.4

56.6

3.8 **

0.025

Average quarterly employment, Year 1a (%)

59.8

57.5

2.3 *

0.084

Total earnings ($)

Year 1 (quarters 3–6)

9,551

9,116

436

0.142

Quarter 7

2,608

2,435

173 *

0.085

Quarter 8

2,692

2,439

253 **

0.015

Sample size (total = 2,653)

1,347

1,306

(continued)

113

Exhibit C.5 (continued)

New

Existing Difference

Outcomes

Rent Rules Rent Rules

(Impact)

P-Value

Washington, D.C.

Ever employed (%)

Year 1 (quarters 3–6)

59.0

58.9

0.1

0.925

Quarter 7

51.5

51.9

-0.4

0.797

Quarter 8

50.0

49.4

0.6

0.670

Average quarterly employment, Year 1a (%)

41.9

42.3

-0.4

0.733

Total earnings ($)

Year 1 (quarters 3–6)

8,535

8,962

-427

0.147

Quarter 7

2,919

2,961

-41

0.694

Quarter 8

2,818

2,907

-89

0.406

Sample size (total = 3,411)

1,709

1,702

aAverage qua rterly em ployment is cal culated as tot al num

ber of qua

rters wi th em

ployment di vided by

total

numbe

r of quarter s o f followu p, expressed as a perce nta ge

.

* = 10 per

cent. ** = 5 per

cent. ** * = 1 p ercent.

Notes: Estimates were regression adjusted using ordinary least squares, controlling for pre-random assignment character istics of sample m

embers. Roun ding may cau se slight discrepanci es i n calculating sums and diffe rences. A two -ta iled t-tes

t was app lied to diff eren ces bet ween research group s. The p-valu e indicate s the likelihood tha t the diffe rence be

tw een the ne w rent rules group and

the exi sting re nt rul es

group ar ose by cha

nc e. Sample sizes for specific out

comes may

var y bec ause of mis sing val ues.

Source: MDRC calculations, using quarterly wage data from the National Directory of New Hires

114

Exhibit C.6. Average Earnings by Quarter Relative to the Quarter of Random Assignment, by Public Housing Agency (PHA) Control Group: Heads of Households Randomly Assigned in Quarter 2 of 2015

3,500

3,300

3,100

2,900

2,700

2,500

2,300

Earnings ($) 2,100

1,900

1,700

1,500

Quarter 1 Quarter 2 Quarter 3 Quarter 4 Quarter 5 Quarter 6 Quarter 7 Quarter 8

Relative Quarter

Washington, D.C.

Louisville

Lexington

San Antonio

Notes: The sample sizes are Lexington: 255, Louisvil e: 472, San Antonio: 292, and Washington, DC: 965.

Estimates were regression adjusted using ordinary least squares, control ing for pre-random assignment characteristics o

f sam

ple members. Samp le sizes for specific ou tcome s may var y becau

se of m

issing values.

Source: M

DRC calc ulatio ns, u sing quar ter ly wage data from the National Directory of New Hires 115

References

Abraham, Katherine, John C. Haltiwanger, Kristin Sandusky, and James Spletzer. 2009.

“Exploring Differences in Employment Between Household and Establishment Data.”

NBER Working Paper No. 14805. Cambridge, MA: National Bureau of Economic Research. www.nber.org/papers/w14085.

Abt Associates Inc., Urban Institute, and Applied Real Estate Analysis, Inc. 2010. Study of Rents and Rent Flexibility. Report prepared for the U.S. Department of Housing and Urban Development, Office of Public and Indian Housing. Washington, D.C: U.S. Government Printing Office.

Benjamini, Yoav, and Yosef Hochberg. 1995. “Controlling the False Discovery Rate: A Practical and Powerful Approach to Multiple Testing,” Journal of the Royal Statistical Society Series B (Methodological) 57 (1): 289–300.

Government Accountability Office. 2012. Housing Choice Vouchers: Options Exist To Increase Program Efficiencies. Washington, D.C: U.S. Government Printing Office.

Hendra, Richard, James A. Riccio, Richard Dorsett, David H. Greenberg, Genevieve Knight, Joan Phillips, Philip K. Robins, Sandra Vegeris, and Johanna Walter, with Aaron Hill, Kathryn Ray, and Jared Smith. 2011. Breaking the Low-Pay, No-Pay Cycle: Final Evidence From the UK Employment Retention and Advancement (ERA) Demonstration.

Report 765. Leeds, United Kingdom: Department for Work and Pensions.

MDRC. 2016. Rent Reform Data Collection and Analysis Plan. Unpublished design paper submitted to the U.S. Department of Housing and Urban Development. New York: MDRC.

Michalopoulos, Charles. 2005. Does Making Work Pay Still Pay? An Update on the Effects of Four Earnings Supplement Programs on Employment, Earnings, and Income. New York: MDRC.

Nuñez, Stephen, Nandita Verma, and Edith Yang. 2015. Building Self-Sufficiency for Housing Voucher Recipients: Interim Findings From the Work Rewards Demonstration in New York City. New York: MDRC.

Ouellette, Tammy, Nancy Burstein, David Long, and Erik Beecroft. 2004. Measures of Material Hardship: Final Report. Washington, D.C: U.S. Department of Health and Human Services, Office of the Assistant Secretary for Planning and Evaluation.

Public Housing Authorities Directors Association. 2005. Rent Reform: Fair and Simple Solutions. Washington, D.C: Public Housing Authorities Directors Association.

Riccio, James, Victoria Deitch, and Nandita Verma. 2017. Reducing Work Disincentives in the Housing Choice Voucher Program: Rent Reform Demonstration Baseline Report. New York: MDRC.

U.S. Department of Housing and Urban Development (HUD). 2001. “Housing Choice Voucher Program Guidebook.” 7420.10G. https://www.hud.gov/sites/documents/DOC_35611.PDF.

116

U.S. Department of Labor, Bureau of Labor Statistics. 2018. “Civilian Labor Force and Unemployment by Metropolitan Area, Seasonally Adjusted.”

https://www.bls.gov/web/metro.supp.toc.htm.

Verma, Nandita, Edith Yang, Stephen Nuñez, and David Long. 2017. Learning from the Work Rewards Demonstration: Final Findings from the Family Self-Sufficiency Study in New York City. New York: MDRC.

117

118

Document Outline

Contents

Acknowledgments

Contents

List of Exhibits

Foreword

Executive Summary

HUD’s Traditional Rent Policy

Overview of the New Rent Policy

Evaluation Design, Sample Characteristics, and Data Sources

Characteristics of Enrolled Families

Data Sources and Followup Period

Early Findings on Household Heads’ Employment and Earnings

Early Impacts on Outcomes Related to Housing Subsidies

Conclusion

Introduction

HUD’s Traditional Rent Policy

Overview of the New Rent Policy

Administering the New Rent Policy

TTPs and Rent Subsidies at the Start of the New Rent Policy

The Scope of This Report

The Study Sample and Analysis Methods

Building the Research Sample

Characteristics of Enrolled Families

Data Sources and Followup Period

Comparability of the Two Research Groups

Analysis Issues and Strategies

Variation in Rent Policies Across PHAs

Louisville Opt-Outs

Early Impacts on Employment and Earnings

Early Impacts for Heads of Household

Impacts for Other Adults and All Adults in the Household

Conclusion

Early Impacts on Housing-Related Outcomes

Calculating Families’ Contributions to Their Housing Costs

Early Impacts on Families’ Housing Expenditures and Subsidies

Paying the Minimum TTP

Use of Safeguards by the New Rent Rules Group

Interim Recertifications

Hardship Remedies

Impacts on PHA Actions for Families

Conclusion

Summary and Next Steps in the Rent Reform Demonstration

Appendix A: Supplementary Materials for Chapter 1

Appendix B: Supplemental Materials for Chapter 2

Strategy for Addressing Multiple Outcome Measures in the Rent Reform Demonstration

Appendix C: Supplementary Materials for Chapter 3

References