Work, Engagement, and

Wel -Being at the Midpoint:

Findings From the Family

Self-Sufficiency Evaluation

U.S. Department of Housing and Urban Development | Office of Policy Development and Research

Visit PD&R’s website

www.huduser.gov

to find this report and others sponsored by HUD’s Office of Policy Development and Research (PD&R). Other services of HUD USER, PD&R’s research information service, include listservs, special interest reports, bimonthly publications (best practices, significant studies from other sources), access to public use databases, and a hotline (800-245-2691) for help accessing the information you need.

Work, Engagement, and

Wel -Being at the Midpoint:

Findings From the Family

Self-Sufficiency Evaluation

Prepared for

U.S. Department of Housing

and Urban Development

Office of Policy Development

and Research

Submitted by

Nandita Verma

Stephen Freedman

Betsy Tessler

Barbara Fink

with

David Navarro

MDRC

July 2021

Acknowledgments

Acknowledgments

This report reflects contributions from many people. We are especially grateful to all the individuals who volunteered to participate in the evaluation—both those who enrol ed in the programs to receive Family Self-Sufficiency (FSS) services and the members of the control group, who have al owed us to learn from their experiences. We also appreciate the dedication of the many staff members, past and present, at the 18 public housing agencies (PHAs) participating in the evaluation who gave generously of their time, provided us with valuable insights and explanations or prepared necessary data files. Space does not permit us to list everyone who was involved, but we are grateful to PHA administrators, FSS supervisors, case managers, and data managers and programmers at Akron Metropolitan Housing Authority, Baltimore County Housing Office, Columbus Metropolitan Housing Authority, Housing Authority of the City of Dal as, Housing Authority of the City of Deerfield Beach, Housing Authority of the City of Fort Lauderdale, Housing Authority of the City of Los Angeles, Housing Authority of the County of Alameda, Housing Authority of the County of Riverside, Housing Authority of Fort Worth, Housing Authority of Kansas City (Missouri), Housing Opportunities Commission of Montgomery, Houston Housing Authority, Jersey City Housing Authority, Lucas Metropolitan Housing Authority, Orange County Housing Authority, Tarrant County Housing Assistance Office, and the Youngstown Metropolitan Housing Authority.

We appreciate the support of many staff at the U.S. Department of Housing and Urban Development (HUD).

Our HUD government technical representatives, Regina Gray, has been a source of steady guidance on all phases of this work. We also thank HUD Office of Public and Indian Housing (PIH) and the Office of Policy Development & Research (PD&R) staff for their helpful review and feedback on report drafts. Robert Mulderig, Anice Chenault, Amaris Rodriguez, Jogchum Poodt, Jayme Brown, and Tremayne Youmans offered valuable feedback on this report. We are appreciative of Lynn Rodgers, who prepared many administrative data files from the HUD Inventory Management System (IMS)/PIH Information Center (PIC) system and managed MDRC’s requests for data from the National Directory of New Hires Data maintained by the U.S. Department of Health and Human Services (HHS)-Office of Child Support Enforcement (OCSE).

With HHS, we are grateful for support on the HHS-OCSE contracts from Cheri Davidson, Wendi Trant, and Jean Shaw. We are also grateful for the contributions of M. Davis and Company, our survey partner.

At MDRC, we thank many staff members for their important contributions. James Riccio, Richard Hendra, Cynthia Mil er, and Gayle Hamilton offered immense insights in their review of report drafts. We are grateful for the dedicated work contributed by data team members Joshua Vermette, Natasha Piatnitskaia, and Brandon Hawkins who reliably provided high-quality analysis. We are appreciative of MDRC’s Survey Unit oversight on the 36-Month Survey. We thank Galina Farberova and Al a Chaplygina who designed and maintained the FSS service-use and contacts database. Annie Utterback was a steady source of support and managed report coordination and fact-checking processes with assistance from Brandon Hawkins, Katerina Galkin, Affiong Ibok, Gloriela Iguina-Colón, Rachael Metz, Andrew Rock, and Diane Singer. In addition, we are grateful for the efforts that Stephanie Rubino and Simran Vazirani contributed as resource managers. Finally, Ali Tufel and Rebecca Bender offered meticulous feedback and editing across stages of the writing process.

DISCLAIMER

The contents of this report are the views of the contractor and do not necessarily reflect the views or policies of the U.S. Department of Housing and Urban Development or the U.S. Government.

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation

Foreword

Foreword

The Family Self-Sufficiency (FSS) program is administered by over 700 housing authorities across the country, who employ case managers/coaches to work with participant households to develop individualized self-sufficiency plans and access education, training, employment opportunities and wrap-around supportive services. The program promotes savings and asset development through an interest-bearing escrow account.

As families’ rent contribution increases due to earnings increases, housing agencies deposit this increased amount into a savings account which is redeemable upon graduation from the program, or in some instances, can be accessed sooner, if needed. The vast majority of families participating in FSS are Black or Hispanic, potentially positioning FSS as a key program to help these families accrue savings. In 2012, HUD’s Office of Policy Development and Research (PD&R) commissioned the first nationwide study of the Family Self-Sufficiency program that incorporates a randomized control ed trial to assess the effectiveness of the program for increasing earnings, sustaining long-term employment, and reducing families need for public assistance because they’ve grown income and savings.

This interim assessment of the Family Self-Sufficiency program, 3 years after the random assignment of participants, continues to build evidence about the program’s effectiveness in helping participants access education, training, and services that may improve their economic wel -being. The final report, covering the full 5-year standard program and up to 6 to 7 years of fol ow-up, is expected in 2022. The 2019 publication entitled Promoting Work and Self-Sufficiency for Housing Voucher Recipients: Early Findings from the Family Self-Sufficiency Program Evaluation was a precursor to this second interim report. The researchers find as of Year Two, that while FSS increased participation in a range of employment-related and support services by a statistically significant 13 percentage points relative to the control group, participants in the FSS program have not yet increased employment rates or average earnings in a statistically significant way over the control group. They did, however, experience small shifts from part-time to ful -time employment. Given that FSS is typically a 5-year program and that participants can be focused on a range of self-sufficiency activities during that timeframe, it is too early to definitively conclude whether the program is effective at shifting employment and income-oriented outcomes. But this study shows that over half of those enrol ed in FSS accrued some escrow over the 3 years of fol owup and those with a positive escrow balances as of the end of Year 3 had accrued an average of nearly $3,700 ($1,450 among all FSS group members).

In considering the lack of a difference in earnings data between FSS and non-FSS participants highlighted in this report, it may be useful to consider the economics of employment and wages at relatively the same time as the 3-year study, namely, the historically low unemployment nationwide that occurred before the COVID-19 pandemic.

Some of the indicators in the study show longer-term investment on the part of FSS participants in their movement toward economic independence, including the focus on financial literacy, establishing bank accounts, educational and skill attainment, working toward increasing the FSS escrow, and the achievement of other goals. We look to the longer-term analyses to see if this foreshadows greater sustained economic resiliency and self-sufficiency. This might be an important factor to the extent that data representing those FSS

participants in the finals years of their FSS contract are impacted by the severe economic downturn wrought by the coronavirus pandemic—which may significantly impact the final results.

Another factor that may affect future outcomes has been HUD’s efforts over the past 3 years to implement a Performance Measurement Methodology for the FSS program. The Performance Measurement Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation

Foreword

Methodology, when fully implemented, will al ow the Department to assess the effectiveness of the FSS

programs around the nation based on program participation rates, graduation rates, and earnings and income growth for participants. The goal for this new system is to incentivize housing authorities to focus on the most fundamental objectives of the FSS program long after the 5-year study is completed. Future reports wil incorporate additional findings on service delivery, FSS graduation rates, and disbursement of escrow dol ars.

Included in the long-term assessment is a survey administered to a sample of study participants and an in-depth analysis of Experian credit data to study asset creation and debt reduction. This credit analysis strikes us as very important because of FSS participants’ desire for financial counseling and their relatively high take-up rate of that financial education relative to the control group.

Beyond the national evaluation, we plan to continue tracking income data for all Family Self-Sufficiency program participants and monitor program metrics established by the program office. FSS represents a significant investment in family wel -being that may take years to have impact; our research plan has the necessary patience to measure those possible long-term impacts. As more data from subsequent years of the program become available, it will be important to assess the extent to which programs like FSS complement to rental assistance. That is, does it enable enrol ed households to generate wealth and accrue savings and thus promote economic opportunity?

Todd M. Richardson

General Deputy Assistant Secretary for Policy Development and Research

U.S. Department of Housing and Urban Development

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation

Contents

Contents

Executive Summary ......................................................................................................................................xiii

Chapter 1. Introduction .................................................................................................................................... 1

Chapter 2. Program Delivery and Implementation Update ...............................................................18

Chapter 3. Participant Engagement .........................................................................................................27

Chapter 4. Escrow Accruals and Graduation .......................................................................................58

Chapter 5. Impacts on Employment and Earnings ..............................................................................79

Chapter 6. Impacts on Income, Subsidy Receipt, and Wel -Being ..................................................93

Chapter 7. Variation in Program Impacts ...............................................................................................110

Chapter 8. Summary and Looking Forward ........................................................................................129

Appendix A .................................................................................................................................................... 133

Appendix B ..................................................................................................................................................... 141

Appendix C ....................................................................................................................................................166

Appendix D .....................................................................................................................................................177

Appendix E ....................................................................................................................................................182

Appendix F .....................................................................................................................................................211

References .................................................................................................................................................... 232

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation v

List of Exhibits

List of Exhibits

Exhibit ES.1. Selected Participation and Escrow Outcomes, Years 1 to 3 .....................................xvii

Exhibit ES.2. Impacts on Use of Services, Year 3 ................................................................................xix

Exhibit ES.3. Impacts on Employment and Earnings, Years 1 to 3 ..................................................xxi

Exhibit ES.4. Impacts on Household Income, Benefit Receipt, Housing Assistance, and

Financial Well-Being, Year 3 .....................................................................................................................xxiii

Exhibit 1.1. Core Components of the HUD Family Self-Sufficiency Framework ............................ 5

Exhibit 1.2. Simplified Schematic of the Family Self-Sufficiency Program Theory of Change .. 9

Exhibit 1.3. Data Sources for the Family Self-Sufficiency Study ........................................................10

Exhibit 1.4. Public Housing Agencies Participating in the National Family Self-Sufficiency

Evaluation .........................................................................................................................................................12

Exhibit 1.5. Baseline Characteristics of Households in the Family Self-Sufficiency Impact

Sample ...............................................................................................................................................................13

Exhibit 1.6. Baseline Characteristics of Heads of Households in the Family Self-Sufficiency

Impact Sample .................................................................................................................................................15

Exhibit 2.1. Site Engagement Practices ...................................................................................................21

Exhibit 3.1. Family Self-Sufficiency and Housing Choice Voucher Program Statuses, Years

1 to 3, Family Self-Sufficiency Service Use and 36-Month Survey Respondent Samples .......30

Exhibit 3.2. Contacts Between Family Self-Sufficiency Participants and Case Managers,

Year 3, Family Self-Sufficiency Service-Use Sample ..........................................................................33

Exhibit 3.3. Use of Family Self-Sufficiency-Related Services and Recorded Employment

During Years 1 to 3, Family Self-Sufficiency Service-Use Sample ..................................................35

Exhibit 3.4. Indicators of Timing and Duration of Service Use and Program-Recorded

Employment During Years 1 to 3, Family Self-Sufficiency Service-Use Sample .........................37

Exhibit 3.5. Selected Indicators of Contacts and Service Use During Years 1 to 3, by

Self-Reported Employment Status at Random Assignment .............................................................39

Exhibit 3.6. Selected Indicators of Contacts and Service Use During Years 1 to 3, by

Self-Reported Level of Educational Attainment at the Time of Random Assignment ..............40

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation vi

List of Exhibits

Exhibit 3.7. Indicators of Contacts and Family Self-Sufficiency Service Use in Year 3, by

Housing Authority, Family Self-Sufficiency Service-Use Sample ....................................................43

Exhibit 3.8. Use of Family Self-Sufficiency Services and Recorded Employment During

Years 1 to 3 Among Housing Authorities with Similar Levels of Emphasis on Monitoring

and Engagement, Family Self-Sufficiency Service-Use Sample .....................................................45

Exhibit 3.9. Use of Services During Years 1 to 3, by Program Emphasis on Service Type .......46

Exhibit 3.10. Indicators of Engagement with the Family Self-Sufficiency Program, by

Current Enrollment Status, Family Self-Sufficiency Group Members in the 36-Month

Survey Respondent Sample ....................................................................................................................... 47

Exhibit 3.11. Impacts on Use of Services and Employment, Family Self-Sufficiency

36-Month Survey Respondent Sample ...................................................................................................51

Exhibit 3.12. Impacts on Use of Services During Years 1 to 3, by Program Emphasis on

Job Search and Post-Employment Services .........................................................................................54

Exhibit 3.13. Impacts on Use of Services During Years 1 to 3, by Program Emphasis on

Education and Training ................................................................................................................................55

Exhibit 3.14. Impacts on Use of Services During Years 1 to 3, by Program Emphasis on

Financial Services ..........................................................................................................................................56

Exhibit 4.1. Monthly Indicators of Positive Escrow Balance and Credit Accrual, Family

Self-Sufficiency Impact Sample ..................................................................................................................61

Exhibit 4.2. Accrual and Disbursement of Family Self-Sufficiency Escrow Credits in Years

1 to 3, Family Self-Sufficiency Impact Sample .......................................................................................63

Exhibit 4.3. Indicators of Family Self-Sufficiency Escrow Account Accrual in Months 1 to

36 for Selected Subgroups .........................................................................................................................66

Exhibit 4.4. Indicators of Escrow Credits Accrual in Months 1 to 36 by Housing Authority,

Family Self-Sufficiency Service-Use Sample .........................................................................................68

Exhibit 4.5. Baseline Characteristics of the Family Self-Sufficiency Group, by Housing

Choice Voucher and Family Self-Sufficiency Program and Escrow Accrual Statuses ..............70

Exhibit 4.6. Selected Indicators of Contacts and Service Use During Years 1 to 3, by

Housing Choice Voucher and Family Self-Sufficiency Enrollment Status and Escrow

Balance .............................................................................................................................................................75

Exhibit 4.7. Change in Reported Annual Head-of-Household Earnings and Total Household

Earnings During Years 1 to 3, by Housing Choice Voucher and Family Self-Sufficiency

Enrollment Status and Escrow Balance in Month 36 .......................................................................... 77

Exhibit 5.1. How to Read the Impact Tables in This Report ................................................................81

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation vii

List of Exhibits

Exhibit 5.2. Quarterly Employment Rate Among Control Group Members, by Self-Reported

Employment Status at Random Assignment, Family Self-Sufficiency Impact Sample .............83

Exhibit 5.3. Average Quarterly Earnings Among Control Group Members, by Self-Reported

Employment Status at Random Assignment, Family Self-Sufficiency Impact Sample .............84

Exhibit 5.4. Impacts on Employment and Earnings, Years 1 to 3, Family Self-Sufficiency

Impact Sample ................................................................................................................................................86

Exhibit 5.5. Strategies to Support Participants’ Work Efforts: Family Self-Sufficiency

Program Staff Members’ Reflections .......................................................................................................87

Exhibit 5.6. Impacts on Employment in the 12 Months Prior to Interview, Family

Self-Sufficiency 36-Month Survey Respondent Sample ...................................................................88

Exhibit 5.7. Impacts on Current Employment and Job Characteristics, Family

Self-Sufficiency 36-Month Survey Respondent Sample ...................................................................90

Exhibit 6.1. Impacts on Household Income and Income Sources, Family Self-Sufficiency

36-Month Survey Respondent Sample ..................................................................................................95

Exhibit 6.2. Impacts on Material Hardship and Food Insufficiency, Family Self-Sufficiency

36-Month Survey Respondent Sample ..................................................................................................98

Exhibit 6.3. Impacts on Use of Financial Services, Savings, and Debt, Family

Self-Sufficiency 36-Month Survey Respondent Sample ..................................................................101

Exhibit 6.4. Impacts on Financial Strain, Family Self-Sufficiency 36-Month Survey

Respondent Sample ...................................................................................................................................103

Exhibit 6.5. Impacts on Shelter Costs and Housing Subsidies, Years 1 to 3, Family

Self-Sufficiency Impact Sample ...............................................................................................................106

Exhibit 6.6. Impacts on Housing, Family Self-Sufficiency 36-Month Survey Respondent

Sample ............................................................................................................................................................108

Exhibit 7.1A. Impacts on Selected Outcomes, by Employment Status at Random

Assignment..................................................................................................................................................... 113

Exhibit 7.1B. Impacts on Selected Outcomes, by Educational Attainment at

Random Assignment ................................................................................................................................... 114

Exhibit 7.1C. Impacts on Selected Outcomes, by Total Household Income at

Random Assignment ...................................................................................................................................115

Exhibit 7.1D. Impacts on Selected Outcomes, by Disability Status at Random Assignment ..116

Exhibit 7.1E. Impacts on Selected Outcomes, by Percentage of Rent and Utilities Expenses

Paid by Household at Random Assignment ..........................................................................................117

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation viii

List of Exhibits

Exhibit 7.2. Program Clusters: Data and Definitions ..........................................................................120

Exhibit 7.3A. Impacts on Selected Outcomes, by Program Emphasis on Job Search and

Post-Employment Services ........................................................................................................................ 121

Exhibit 7.3B. Impacts on Selected Outcomes, by Program Emphasis on Education

and Training ................................................................................................................................................... 122

Exhibit 7.3C. Impacts on Selected Outcomes, by Program Emphasis on Financial Services 123

Exhibit 7.3D. Impacts on Selected Outcomes, by Program Emphasis on Monitoring

and Engagement ......................................................................................................................................... 124

Appendix Exhibit A.1. Baseline Characteristics of Households in the Family Self-Sufficiency

Impact Sample by Research Group ....................................................................................................... 133

Appendix Exhibit A.2. Baseline Characteristics of Heads of Households in the Family

Self-Sufficiency Impact Sample by Research Group ........................................................................ 135

Appendix Exhibit A.3. Staffing, Escrow, and Graduation Policies .................................................138

Appendix Exhibit B.1. Use of Family Self-Sufficiency Services (Detailed) and Recorded

Employment During Years 1 to 3, Family Self-Sufficiency Service-Use Sample ........................ 141

Appendix Exhibit B.2. Selected Indicators of Contacts and Service Use During Years 1

to 3, by Self-Reported Employment Status at Random Assignment ............................................ 143

Appendix Exhibit B.3. Composite Score Component Values, by Site, Family

Self-Sufficiency Service-Use Sample .................................................................................................... 144

Appendix Exhibit B.4. Impacts on Use of Services in the 12 Months Prior to Interview

by Self-Reported Employment Status at the Time of Random Assignment, Family

Self-Sufficiency 36-Month Survey Respondent Sample ................................................................. 145

Appendix Exhibit B.5. Impacts on Education and Training, by Self-Reported Employment

Status at the Time of Random Assignment, Family Self-Sufficiency 36-Month Survey

Respondent Sample ....................................................................................................................................147

Appendix Exhibit B.6. Impacts on Use of Services in the 12 Months Prior to Interview

by Self-Reported Level of Educational Attainment at the Time of Random Assignment,

Family Self-Sufficiency 36-Month Survey Respondent Sample .................................................... 148

Appendix Exhibit B.7. Impacts on Education and Training, by Self-Reported Level of

Education Attainment at the Time of Random Assignment, Family Self-Sufficiency

36-Month Survey Respondent Sample ................................................................................................150

Appendix Exhibit B.8. Impacts on Use of Services in the 12 Months Prior to Interview for

Housing Authorities with Similar Levels of Emphasis on Monitoring and Engagement,

Family Self-Sufficiency 36-Month Survey Respondent Sample ....................................................152

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation ix

List of Exhibits

Appendix Exhibit B.9. Impacts on Education and Training for Housing Authorities with

Similar Levels of Emphasis on Monitoring and Engagement, Family Self-Sufficiency

36-Month Survey Respondent Sample ................................................................................................154

Appendix Exhibit B.10. Impacts on Use of Services in the 12 Months Prior to Interview,

by Self-Reported Employment Status at the Time of Random Assignment, Family

Self-Sufficiency 36-Month Survey Respondent Sample .................................................................155

Appendix Exhibit B.11. Impacts on Educational Credential Attainment, by Self-Reported

Employment Status at the Time of Random Assignment, Family Self-Sufficiency

36-Month Survey Respondent Sample ................................................................................................156

Appendix Exhibit B.12. Impacts on Use of Services in the 12 Months Prior to Interview

by Program Emphasis on Job Search and Post-Employment Services, Family

Self-Sufficiency 36-Month Survey Respondent Sample ..................................................................157

Appendix Exhibit B.13. Impacts on Educational Credential Attainment, by Program

Emphasis on Job Search and Post-Employment Services, Family Self-Sufficiency

36-Month Survey Respondent Sample ................................................................................................159

Appendix Exhibit B.14. Impacts on Use of Services in the 12 Months Prior to Interview by

Program Emphasis on Education and Training, Family Self-Sufficiency 36-Month Survey

Respondent Sample ...................................................................................................................................160

Appendix Exhibit B.15. Impacts on Educational Credential Attainment, by Program

Emphasis on Education and Training, Family Self-Sufficiency 36-Month Survey

Respondent Sample ...................................................................................................................................162

Appendix Exhibit B.16. Impacts on Use of Services in the 12 Months Prior to Interview

by Program Emphasis on Financial Services, Family Self-Sufficiency 36-Month Survey

Respondent Sample ...................................................................................................................................163

Appendix Exhibit B.17. Impacts on Educational Credential Attainment, by Program

Emphasis on Financial Services, Family Self-Sufficiency 36-Month Survey Respondent

Sample ............................................................................................................................................................165

Appendix Exhibit C.1. Impacts on Employment and Earnings by Quarter, Family

Self-Sufficiency Impact Sample ...............................................................................................................166

Appendix Exhibit C.2. Regression Coefficients for Estimated Impacts on Total Earnings

in Quarters 2 to 13, Family Self-Sufficiency Impact Sample ..............................................................171

Appendix Exhibit C.3. Unadjusted and Adjusted Impacts on Employment and Earnings

in Quarters 2 to 13, Family Self-Sufficiency Impact Sample .............................................................174

Appendix Exhibit C.4. Total Earnings in Quarters 2 to 13 by Level of Exclusion for Outlier

Values, Family Self-Sufficiency Impact Sample ...................................................................................175

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation x

List of Exhibits

Appendix Exhibit C.5. Impacts on Total Earnings in Quarters 2 to 13 by Level of Exclusion

for Outlier Values, Family Self-Sufficiency Impact Sample ..............................................................175

Appendix Exhibit C.6. Impacts on Employment and Earnings During Quarters 2 to 13 by

Weighting Strategy, Family Self-Sufficiency Impact Sample ...........................................................176

Appendix Exhibit D.1. Differences by Research Group in Estimated Annual Income

After 3 Years of Follow-Up, Family Self-Sufficiency Impact Sample Members Currently

Enrol ed in the Housing Choice Voucher Program .............................................................................177

Appendix Exhibit D.2. Differences by Research Group in Estimated Annual Income After

3 Years of Follow-Up, Family Self-Sufficiency Impact Sample Members ...................................180

Appendix Exhibit E.1. Impacts on Selected Employment and Earnings Outcomes in Years

1 to 3, by Selected Baseline Characteristics, Family Self-Sufficiency Impact Sample ............182

Appendix Exhibit E.2. Characteristics, Family Self-Sufficiency 36-Month Survey

Respondent Sample ...................................................................................................................................185

Appendix Exhibit E.3. Impacts on Selected Indicators of Rent and Subsidies in Month

36, by Selected Baseline Characteristics, Family Self-Sufficiency Impact Sample .................192

Appendix Exhibit E.4. Confirmatory Comparisons (Impacts) of Employment and Earnings

Outcomes in Years 1 to 3, by Program Approach, Family Self-Sufficiency Impact Sample ....197

Appendix Exhibit E.5. Impacts on Selected Survey Outcomes by Selected Program

Implementation Features, Family Self-Sufficiency 36-Month Survey Respondent

Sample ............................................................................................................................................................199

Appendix Exhibit E.6. Impacts on Selected Indicators of Rent and Subsidies in Month

36, by Selected Program Implementation Features, Family Self-Sufficiency Impact

Sample ...........................................................................................................................................................204

Appendix Exhibit E.7. Exploratory Comparisons of Employment and Earnings Outcomes

in Years 1 to 3 for Sites with Similar Escrow Accrual and Service Use Outcomes, Family

Self-Sufficiency Impact Sample .............................................................................................................. 207

Appendix Exhibit E.8. Impacts on Employment and Earnings, Years 1 to 3, by Self-Reported

Employment Status at Random Assignment, Family Self-Sufficiency Impact Sample ..........209

Appendix Exhibit E.9. Methodology for Creating Site Clusters .....................................................210

Appendix Exhibit F.1. Family Self-Sufficiency 36-Month Survey Samples and

Response Rates ........................................................................................................................................... 214

Appendix Exhibit F.2. Estimated Regression Coefficients of the Probability of Being a

Respondent to the Family Self-Sufficiency 36-Month Survey, Family Self-Sufficiency

Impact Sample ..............................................................................................................................................215

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation xi

List of Exhibits

Appendix Exhibit F.3. Baseline Characteristics of Households in the Family

Self-Sufficiency Impact Sample by Family Self-Sufficiency 36-Month Survey Respondent

Status ..............................................................................................................................................................218

Appendix Exhibit F.4. Baseline Characteristics of Heads of Households in the Family

Self-Sufficiency Impact Sample by Family Self-Sufficiency 36-Month Survey Respondent

Status .............................................................................................................................................................220

Appendix Exhibit F.5. Estimated Regression Coefficients of the Probability of Being a

Family Self-Sufficiency Group Member, Family Self-Sufficiency 36-Month Survey

Respondents ................................................................................................................................................ 223

Appendix Exhibit F.6. Baseline Characteristics of Households by Research Group,

Family Self-Sufficiency 36-Month Survey Respondent Sample ...................................................226

Appendix Exhibit F.7. Baseline Characteristics of Heads of Households by Research

Group, Family Self-Sufficiency 36-Month Survey Respondent Sample .....................................228

Appendix Exhibit F.8. Comparison of Impacts on Employment and Earnings in Years 1 to

3, Family Self-Sufficiency Impact and 36-Month Survey Respondent Samples ...................... 231

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation xii

Executive Summary

Executive Summary

The U.S. Department of Housing and Urban Development (HUD) Family Self-Sufficiency (FSS) program blends housing assistance with employment-focused supports and services to help improve the economic wel -being of households receiving housing subsidies. A voluntary program, FSS is the main strategy through which HUD helps households receiving federal housing vouchers to increase their earnings and make progress toward economic self-sufficiency. Until recently, little evidence was available about the FSS program’s effectiveness in achieving those goals. To fill that gap, HUD commissioned a national evaluation in 2012 and selected MDRC to lead it. This report presents interim findings from the ongoing evaluation.

HUD provides about 700 public housing agencies (PHAs) with annual grants to hire service coordinators to work with FSS participants to set self-sufficiency goals and to refer them to services in their communities.1

Participants are eligible to receive a lump sum payment when they successfully graduate from the program.

HUD requires FSS participants to meet two goals to graduate: (1) they must be employed, and (2) al household members must not have received cash welfare assistance for the 12 consecutive months leading up to graduation. At enrol ment, participants—typically heads of households—sign a Contract of Participation and develop an Individual Training and Services Plan (ITSP). FSS contracts usually last for 5 years (with 2-year extensions possible). Program staff members work with participants to set and achieve additional goals, such as acquiring an educational or occupational credential or improving credit scores—goals that may be important to the participant and foster employment and higher earnings. A core feature of the program is its escrow account, which is offered to program participants to encourage them to go to work and build long-term savings. Like others receiving housing subsidies, FSS participants pay additional rent when their earnings increase (typically 30 percent of additional earnings), but in the FSS program, the housing agency credits the family’s escrow account with an amount based on their rent increase. Typically, escrow accruals are disbursed to participants when they graduate from the program, and graduates can use their escrow how they please.

If the PHA permits, participants may receive a portion of their escrow accruals earlier, in the form of interim disbursements, to help them achieve their FSS goals.

Using a randomized control ed trial, the national FSS evaluation examines the effects of the program on a range of economic mobility outcomes, including participants’ engagement in and completion of education or training activities, employment, earnings, receipt of housing subsidies and other government assistance, and financial and material wel -being. It compares the outcomes of individuals who were randomly assigned to two groups: an FSS group, whose members were eligible to participate in the FSS program, and a control group, whose members were not eligible to participate in the program for 3 years after study enrol ment. Differences between the two groups’ outcomes represent the program’s impacts. Statistically significant differences between groups indicate with a strong degree of confidence that the impacts can be attributed to FSS rather than to chance.

The national evaluation focuses on housing agencies operating FSS for housing choice voucher (HCV) holders.

Eighteen housing agencies in seven states, operating both small and large FSS programs for HCV recipients, agreed to participate in this evaluation and together enrol ed 2,656 study participants. This report summarizes the findings for the 2,556 study participants (the “impact sample”) who were 18 to 61 years of age at study enrol ment. The second in a series, the report describes longer term (3-year) patterns of program participation and program impacts on employment and housing subsidies. It features findings from a comprehensive survey, conducted roughly 36 months after participants had enrol ed in the study. At this stage in the evaluation, the 1

HUD and program operators refer to this staff position as FSS program coordinator, FSS case manager, or FSS coach.

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation xiii

Executive Summary

interim results point to largely comparable outcomes for the FSS and control groups. Future reports, which will cover 6 to 7 years of fol owup (the full FSS engagement period for those receiving extensions), will speak more conclusively to the program effects.

The Family Self-Sufficiency Program

The FSS program was established in 1990 by Section 554 of the Cranston-Gonzalez National Affordable Housing Act against a backdrop of policy discussions about persistent poverty among participants in government benefits programs. Housing agencies that administer public housing or housing voucher programs operate most FSS programs. In fiscal year 2018, HUD made about $75 mil ion in FSS annual grants available for program coordinator positions (HUD, 2018a). Funding for program management, services, and other related administrative costs is not covered by these grants; these costs are typically absorbed by the housing agency operating the program. Fiscal year 2017 grants show that the size of the programs funded can range from as few as 15 participants in the smal est program to more than 1,000 in the largest (HUD, 2017b).

Overal , FSS enrol s a small fraction of families receiving HCVs, a reflection of the limited congressionally appropriated funding to operate it.

HUD regulations set the basic framework for the FSS program around two central components: (1) case management or case coordination and referrals to employment-related, education and training, and financial management and counseling services, and (2) an escrow savings account (a longer-term financial incentive for households to increase their work and earnings). These program components are expected to help and encourage participants to go to work, increase their earnings from work, reduce their reliance on cash welfare assistance programs, build assets, and set a pathway toward economic self-sufficiency. Although all adults in FSS households are encouraged to seek employment, only the household head—the voucher holder—is required to meet the employment goals of the FSS contract. Housing agencies can design their own case management approaches, an element of flexibility offered by the federal framework. In contrast, the escrow component strictly fol ows HUD specifications, and the rules for escrow calculations are largely uniform across all PHAs.

FSS operators are also required to organize a Program Coordinating Committee (PCC), consisting of service providers in the community. The PCC, which operates more like an informal col aborative group, serves as a mechanism by which FSS operators are made aware of services and resources in the community and service providers can become invested in the success of the FSS program by providing services to FSS participants.

In recent years, HUD has produced a comprehensive resource guide on the program, offering housing agencies practical, hands-on tips for its operation. In 2018, HUD also rol ed out a performance measurement system for assessing programs that receive HUD-FSS funding. Both developments reflect HUD efforts to strengthen program performance.

Mechanisms for Positive Effects

This evaluation posits that the program’s short- and long-term effects are expected to operate via two mechanisms: (1) increase participants’ access to ongoing case management and coordination services that enhance their employability or improve their financial security, and (2) offer a long-term escrow savings account—a potentially generous financial incentive—for participants to start or maintain employment and increase their earnings over time. Through these mechanisms, it is hoped that participants will eventually earn enough to pay for housing without a subsidy and thereby free up PHA resources to serve other households.

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation xiv

Executive Summary

Positive effects may not be realized for a variety of reasons, however. For example, escrow represents a distant and uncertain reward that may not motivate participants to increase their earnings and, per HCV rules, pay higher rent. Also, FSS participants face a variety of barriers that may limit their chances of finding new employment or increasing their earnings, especially if the nature or quality of services delivered do not help address those barriers. In addition, members of the control group may be able to access similar employment-promoting services in their communities, which would suggest that FSS is not adding value over and above what can already be obtained with existing community services. Finally, given the long-term nature of the FSS

program, it is also possible that positive impacts may not be realized in the shorter-term but may be realized later in the fol owup period (closer to the graduation milestone, for example) if more FSS group members take longer to attain education or training credentials, find employment, receive a raise or promotion at their job, or move to a better job.

Evaluation and Sample Characteristics

The HUD-funded FSS evaluation is designed to build rigorous evidence about the effectiveness of the program, examining the types of questions that to date have limited evidence. For instance:

• Does the FSS program help improve the employment, earnings, income, and financial wel -being of program participants compared with what would have occurred in the absence of FSS services and financial incentives?

• How do different groups of FSS participants respond to the program?

• Are some program implementation features more effective at helping participants achieve or make progress toward their self-sufficiency goals?

• Are some types of FSS programs more effective than others?

To address these types of questions, the evaluation includes a comprehensive study of program implementation practices, program impacts, and program benefits and costs. It uses a combination of qualitative and quantitative data to assess the ways in which the FSS program affects the wel -being of program participants.

The evaluation examines the effects for the study sample overall and for certain subgroups—such as those who are not working or those with low levels of educational attainment at study enrol ment. The evaluation also explores whether some FSS programs are more effective than others, whether program features or implementation approaches, such as having small or large caseloads or maintaining more or less frequent contact with participants, appear to be more effective in boosting program participation or improving outcomes.

To select study sites, MDRC examined HUD data, identified candidate PHAs, conducted phone reconnaissance with approximately 60 administrators, visited 27 sites, and developed study agreements with 18 sites in seven states: California, Florida, Maryland, Missouri, New Jersey, Ohio, and Texas. The 18 sites in the evaluation represent a wide range of contexts within which FSS programs operate, such as large and smal housing agencies and FSS programs, in large and small cities and suburban settings.2

In keeping with HUD regulations, the heads of households enrol ing in FSS had to be 18 years of age or older, be in good standing with the housing agency, and have completed a recertification of HCV eligibility within the past 120 days (or be wil ing to complete one at enrol ment, to provide the housing agency with a baseline income for use in determining future escrow credits). Working with each housing agency, MDRC

2

Rural FSS programs, which serve extremely small numbers of participants, are not represented in the study.

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation xv

Executive Summary

developed study recruitment and enrol ment processes. The study agencies did not have to increase or decrease the size of their FSS programs for the purposes of the evaluation, but they did need to double the number of households that signed up for a chance to participate in the FSS program, because one-half were assigned to the control group.

The study sample was enrol ed between October 2013 and December 2014. The study households’

demographic characteristics are broadly similar to those in the national FSS population. The study participants are predominantly (91 percent) female, with an average age of 39 years at random assignment.

At enrol ment, more than 76 percent of participants had a minor child present at home, typically age 12 or younger. About 14 percent did not have a high school diploma or equivalency certificate, suggesting that lack of education may represent a barrier to work for this subset of the population. More than one-half (56 percent) of the sample was working at study enrol ment. Receipt of Supplemental Nutrition Assistance Program (SNAP) benefits, or food stamps, was high (about 70 percent), implying that a large proportion of households had earnings equal to 130 percent of the federal poverty level or less, which is the SNAP benefit eligibility cutoff.

Among the study sample, and similar to national patterns, cash assistance receipt was low (less than one-sixth received cash assistance), and about one-third of the sample reported receiving housing vouchers for 10 years or more. Participants reported that the opportunity to receive financial counseling and management services was a big draw to the program; job-related services were also described as a draw, but to a lesser extent.

Findings on Program Contact and Service Use

Throughout individuals’ enrol ment in the FSS program, FSS coordinators or case managers remain their main point of contact. Through these contacts with program staff, FSS offers participants information about a multitude of services for enhancing their self-sufficiency; case managers and coordinators largely rely on referrals to schools, col eges, and community organizations to provide these services. This report examines participants’ engagement patterns with FSS program staff and use of services over the first 3 years of their 5-year contract period. Do participants stay formally enrol ed in FSS? Do they maintain some type of contact with program staff? What types of services do they use to help them make progress toward attaining their FSS goals? FSS program tenure is likely to affect these patterns, so the summary below starts with a discussion of the extent to which participants are still formally enrol ed in the FSS program at the end of the 3-year fol owup period. A variety of data sources, including program data maintained by the housing agencies, HUD administrative data, and responses to the FSS 36-month survey, are used to examine participation and engagement patterns.

• By the end of Year 3, about 60 percent of FSS group members were still enrolled in the program.

As with any other employment-focused self-sufficiency program, the FSS program in the evaluation witnessed some loss of program participants over the first 3 years of fol owup. While 60 percent of the FSS group remained formally enrol ed in the program, a relatively large proportion (40 percent) had formally exited the FSS program by the end of Year 3. HUD and housing agency administrative data include limited information about exit reasons but show that only about 4 percent of the FSS program exits were related to participants graduating from the program (see exhibit ES.1). A larger proportion of exits involve participants leaving the voucher program or moving to another PHA, being terminated from FSS, or leaving the program voluntarily.

Most sites (all but three) report having policies that al ow them to terminate FSS participants who do not show up for appointments and are not responsive to case managers’ repeated attempts to reach them. Staff members Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation xvi

Executive Summary

at some sites reported feeling strongly that inactive participants should be terminated from the program, while staff members at other sites reported using letters of termination to try to re-engage families.

Exhibit ES.1. Selected Participation and Escrow Outcomes, Years 1 to 3

Outcome

FSS Group

Program Status

Still enrol ed in FSS, Month 36 (%)

61.3

Graduated from FSS (%)

4.2

Sample size

1,285

Program Participation

Used FSS services, Years 1 to 3 (%)

80.4

One or more contact with FSS case manager, Year 3 (%)

50.3

Used FSS services, Year 3 (%)

36.3

Had program-recorded employment, Year 3 (%)

54.8

Service-use and program-recorded employment, Year 3 (%)

28.3

Service-use only, Year 3 (%)

8.0

Program-recorded employment only, Year 3 (%)

26.5

Neither service use nor program-recorded employment, Year 3 (%)

37.3

Sample size

714

Escrow

At least one month of credit accrual, months 1–36 (%)

51.7

Had any escrow balance in Month 36 (%)

40.0

Average escrow balance in Month 36 ($)

1,462

Received escrow disbursement, Months 1–36 (%)

3.5

Received interim disbursement

0.8

Sample size

1,285

FSS = Family Self-Sufficiency.

NOTES: The FSS service-use sample includes all or a randomly selected subsample of Housing Choice Voucher (HCV) heads of household who were randomly assigned to the FSS group between October 18, 2013, and December 22, 2014 in 12 public housing agencies, and were age 18 to 61 at the time of random assignment. The FSS impact sample includes HCV households that were randomly assigned between October 18, 2013, and December 22, 2014, and with a head of household age 18 to 61 at the time of random assignment. Rounding may cause slight discrepancies in calculating sums.

SOURCES: MDRC calculations using housing authority administrative data and HUD Inventory Management System/Office of Public and Indian Housing Information Centzer data.

• On average, in Year 3, roughly halfway through the program, FSS group members who were still enrolled in the program interacted with an FSS case manager about once every 6 months, a decrease in frequency compared with Year 1.

Contact with program staff members declined over 3 years. A “contact” assumes an exchange between the FSS participant and the FSS case manager, either simultaneously, as in a face-to-face meeting or phone cal , or Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation xvii

Executive Summary

sequentially, as during a documented exchange of electronic communications or when the FSS group member mailed or dropped off at the housing agency office a letter or a completed progress report. During the first year of the program, FSS group members communicated with their case manager every 4 or 5 months. Across al housing agencies, FSS group members and FSS case managers averaged less than two exchanges, equivalent to communications occurring every 8 or 9 months—or once every 6 months among FSS group members who were still enrol ed in the program. The average for Year 3 also fal s below the established standard requirement of quarterly or monthly communications that was set by many housing agencies at the start of the evaluation.

• The majority of the FSS group engaged in some type of FSS goal-related activity during the first 3 years of followup. The incidence of participation in FSS-related services decreased over time, however, in part because more FSS group members appear to be were working for pay.

Close to 60 percent of the FSS group members began to participate in FSS-related services within 6 months of random assignment, often by attending short-term workshops on job search skil s, financial security, homeownership preparation, or life skil s training or conducting a job search or education and training-related activity. Over the 3-year fol owup period, though, relatively few FSS group members who had not begun participating in FSS-related services and activities by the end of Month 6 began participation during the remaining 2.5 years of fol owup. According to program records, a little over one-third of FSS group members participated in at least one FSS-related service during Year 3.

The high rate of employment among FSS group members could help explain why they participated in services and activities intermittently or stopped after completing one or two short-term activities. In any month of fol owup, more than twice as many FSS group members were employed as were participating in FSS activities, and in Year 3, a similar proportion of FSS group members worked for pay but did not use FSS-related services, compared with those who combined service use and employment. The high incidence of employment among the FSS group is a positive outcome both in general and because employment is a required graduation goal.

Nonetheless, combining work and service is often difficult for low-income heads of households, particularly those with young children or other caregiving responsibilities. As the months of fol owup proceed, FSS group members who maintain employment but have ceased participating in FSS activities or, as occurred for 12

percent of FSS group members, had not yet participated in FSS activities by the end of Year 3, may find it increasingly difficult to connect with the program and complete all goals listed in their ITSP.

• The FSS program led to increases of 10 to 23 percentage points above the control group levels in participation in employment- or self-sufficiency-related activities.

The 36-month survey respondents in both research groups were asked whether they had received a broad range of services and financial supports in the 12 months prior to their survey interview. Responses to these questions provide important information about the control group’s engagement in similar services and for interpreting the estimates of the FSS program’s effects on employment, earnings, and other self-sufficiency outcomes. As shown in exhibit ES.2, nearly 70 percent of the control group members reported using at least one service in the 12 months preceding the interview. Control group respondents most often sought out assistance for enrol ing in or paying for healthcare coverage, attended education or training programs, or participated in job search activities. Relatively few control group members reported receiving assistance to help manage their family finances or for homeownership preparation. The FSS program realized moderate-level increases (between 6 and 14 percentage points) above control group levels in the domains of job search, homeownership preparation, post-employment services, and education and training but had a much larger effect (greater than 20 percentage points) on the use of financial counseling services. FSS group respondents Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation xviii

Executive Summary

were also more likely than their control group counterparts to participate in multiple activities or combine employment and services. Higher levels of participation in education and training activities were accompanied by a small increase above the control group level in earning an occupational credential or license within the 36-month fol owup period, but do not appear to have increased attainment of academic credentials (such as conferral of a degree or a diploma, not shown).

Exhibit ES.2. Impacts on Use of Services, Year 3

FSS

Control

Difference

Outcomes

Group

Group

(Impact)

p-Value

In the Past 12 Months

Used any services (%)

79.5

69.0

10.5***

0.000

Job search

38.8

24.4

14.4***

0.000

Post-employment services

22.4

15.7

6.6***

0.000

Financial counseling

40.6

17.9

22.8***

0.000

Education or training

30.6

24.9

5.7***

0.005

Homeownership preparation

17.5

6.2

11.3***

0.000

Health coverage or health assistance

25.9

27.7

-1.8

0.383

Social services

19.3

19.1

0.3

0.892

Supportive services

20.2

17.1

3.1*

0.077

Sample size (total = 1,936)

988

948

FSS = Family Self-Sufficiency.

NOTES: The FSS 36-Month Survey respondent sample includes Housing Choice Voucher heads of household who were randomly assigned between October 18, 2013, and December 22, 2014, were age 18 to 61 at the time of random assignment and responded to the FSS 36-Month Survey. Sample sizes for specific outcomes may vary because of missing values. Estimates were regression-adjusted using ordinary least squares, controlling for differences in sample member characteristics recorded at the time of random assignment.

No special weights were applied to responses to adjust for differences in sample size by housing authority. Rounding may cause slight discrepancies in calculating differences. A two-tailed t-test was applied to differences between the FSS group and the control group. The p-value indicates the likelihood that the difference between the FSS group and control group arose by chance. Statistical significance levels are indicated as: * = 10 percent; ** = 5 percent; and *** = 1 percent.

SOURCE: MDRC calculations using responses to the FSS 36-Month Survey.

Findings on Escrow Accrual and Balances

The FSS program’s escrow incentive is intended to motivate participants to increase their earnings, reduce their reliance on cash welfare assistance, and build long-term savings. The likelihood of accruing escrow, however, can be influenced by several factors. One is FSS participants’ employment status at program enrol ment. FSS participants who are not working at program entry could potentially benefit the most from the FSS escrow because all their future earnings would be included in the calculation of escrow credits.

Unemployed participants may also face the most severe barriers to finding and maintaining employment, however, which is required for graduation and earning the escrow. In contrast, FSS participants who enter the program working full time or with relatively high earnings may still have the best prospects of increasing their earnings by finding a better job or advancing with their current employer. They may also be the most likely to maintain their employment after they start accruing escrow credits. They may experience smal er increases in their earnings, however, and accrue only a small amount of escrow.

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation xix

Executive Summary

• Nearly 52 percent of the FSS group accrued some escrow over 3 years of followup. Those with positive escrow balances as of the end of Year 3 had accrued an average of nearly $3,700 ($1,450 among all FSS group members).

FSS group members began accruing escrow credits at a fairly rapid pace fol owing their enrol ment in the program. At the end of Year 1, nearly one-fourth of FSS group members had a positive escrow balance. A year later, that proportion had increased to 39 percent. Few additional FSS group members began accruing escrow during Year 3, however, as this proportion stood at 40 percent at the end of the 3-year fol owup period. FSS

group members who were underemployed at random assignment—that is, who were working part-time hours or receiving relatively low annual earnings—were more likely to accrue escrow credits compared with those who were working ful -time or receiving relatively high earnings.

During the 3 years of fol owup covered in this report, only about 4 percent of FSS group members had graduated or received an interim escrow disbursement. Further, exits from the FSS program accounted for a fairly large drop-off in escrow accrual, as about one in five FSS group members who had accrued escrow no longer maintained a balance in Month 36.

A larger proportion of FSS group members with a positive escrow balance at the end of Year 3 maintained contact with their FSS case manager and participated in an FSS-related service during Year 3, compared with FSS group members with no balance. Nonetheless, as mentioned above, a relatively large proportion of FSS

group members with some escrow were working during Year 3 but did not participate in FSS services, per program records.

Impacts on Employment and Earnings

This evaluation draws on the National Directory of New Hires (NDNH) quarterly wage records and survey responses to examine the program’s early effects on work outcomes—that is, whether the FSS group experienced a greater incidence of employment or earned more on average than members of the control group in the first 3 years fol owing program enrol ment. FSS participants can take up to 5 years to achieve their program goals, and given their highly individualized goals and pathways to attain them, positive effects on work outcomes may occur after the first 3 years of program enrol ment tracked in this report.

• Over the first 3 years of followup, based on NDNH wage data, the FSS and control groups experienced comparable quarterly employment levels and average earnings.

Quarterly wage data show high levels of employment for FSS and control group members. More than 80

percent of both groups worked for pay at some point during the 3 years of fol owup; on average, about 63

percent were employed in any given quarter. Members of both groups averaged a bit more than $41,000 in total earnings during the fol owup period (nearly $14,000 per year). As shown in exhibit ES.3, there are only small and (with one exception) not statistically significant differences in employment or earnings outcomes between the two study groups. These outcomes were also examined for subgroups of participants based on their work, education, and disability statuses at enrol ment. Exhibit ES.3 presents the results for subgroups defined by baseline employment status and shows that no statistically significant differences exist in impacts by employment status at random assignment. For that matter, for the range of subgroups examined so far, the FSS program led to only small and not statistically significant differences (or impacts) on these measures when average values for each research group are compared.

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation xx

Executive Summary

Exhibit ES.3. Impacts on Employment and Earnings, Years 1 to 3

FSS

Control

Difference

Outcome

Group

Group

(Impact)

p-Value

Impact Sample

Ever employed (%)

83.4

81.7

1.7

0.170

Average quarterly employment rate (%)

63.5

63.0

0.5

0.679

Employed in all quarters (%)

32.7

35.5

-2.8*

0.081

Total earnings ($)

41,650

41,487

163

0.884

Sample size (total = 2,548)

1,282

1,266

Not Employed at Random Assignment

Ever employed (%)

68.6

66.5

2.1

0.378

Average quarterly employment rate (%)

42.0

41.2

0.8

0.667

Employed in all quarters (%)

8.9

11.2

-2.3

0.178

Total earnings ($)

20,923

20,109

814

0.572

Sample size (total = 1,126)

575

551

Employed at Random Assignment

Ever employed (%)

95.2

93.7

1.5

0.164

Average quarterly employment rate (%)

80.5

80.3

0.2

0.898

Employed in all quarters (%)

51.6

54.7

-3.1

0.209

Total earnings ($)

58,060

58,403

-343

0.835

Sample size (total = 1,422)

707

715

FSS = Family Self-Sufficiency.

NOTES: The FSS impact sample includes Housing Choice Voucher households that were randomly assigned between October 18, 2013, and December 22, 2014, and with a head of household age 18 to 61 at the time of random assignment. Average quarterly employment rate is calculated as total quarters employed divided by total quarters of followup, expressed as a percentage. Estimates were regression-adjusted using ordinary least squares, controlling for pre-random-assignment characteristics of sample members. Rounding may cause slight discrepancies in calculating sums and differences. A two-tailed t-test was applied to differences between research groups.

The p-value indicates the likelihood that the difference between the FSS group and control group arose by chance. Statistical significance levels are indicated as: * = 10 percent; ** = 5 percent; and *** = 1 percent. The H-statistic test was used to test for statistically significant differences in impact estimates across different subgroups. Statistical significance levels are indicated as: † = 10 percent; †† = 5 percent; and ††† = 1 percent. No statistically significant differences in impact estimates were found across subgroups.

SOURCES: MDRC calculations using baseline data and quarterly wage data from the National Directory of New Hires.

The 36-month survey was used to examine program impacts on job characteristics (for example, hourly wages and hours worked), data that are not captured in NDNH wage records. Most respondents in each research group who were working at the time of their interview reported working in a regular job (self-employment or temporary or seasonal jobs was rare). The largest proportion of respondents worked ful -time and worked regular daytime shifts. Respondents tended to work in relatively low-paying jobs with few or no benefits. FSS group members reported working full time for close to 6 months during the 12 months before interview, compared with 5 months for the control group, a difference that is statistically significant. FSS group members were more likely to report they were looking for ful -time work, compared with their control group Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation xxi

Executive Summary

counterparts (31 percent versus 27 percent). FSS group respondents also reported working slightly more hours per week, an increase of about 1 hour (a statistically significant difference).

• The FSS program appears to have positive effects on self-reported measures of employment and financial well-being among more disadvantaged subgroups—a pattern is not supported by NDNH quarterly wage data.

For most subgroups, the survey data show that the FSS group reported a higher incidence of working ful -time hours at the interview compared with control group members. Statistically significant increases above the control group ranged from 4 to 12 percentage points. Impacts were recorded for four subgroups with greater disadvantages and for three additional subgroups.3 These findings suggest that the FSS program may have facilitated more disadvantaged study participants to realize slightly better employment outcomes (more hours of work but not higher earnings) compared with the control group.

Given that FSS is a 5-year program and that participants can focus on a range of self-sufficiency goals during this timeframe, it is important to continue monitoring their employment behaviors over the full 5-year term of the FSS program to assess whether it is effective at shifting work outcomes. The longer-term evaluation wil continue to track the employment and earnings trajectories for both study groups and assess whether bigger differences begin to emerge as FSS participants approach the end of their FSS contracts and possibly access their escrow balances.

Impacts on Income, Benefit Receipt, and Material Wel -Being

• In the short term, the FSS program had few, and mostly inconsistent, effects on indicators of self-reported income and financial well-being.

As shown in exhibit ES.4, the FSS program increased participants’ access to mainstream banking, but FSS

group members, on average, incurred greater amounts of debt. Consistent with programs that emphasize the use of financial security and homeownership preparation counseling and workshops, FSS led to positive effects on attitudinal outcomes. FSS group members, when surveyed, were more likely than control group respondents to indicate that they had improved their financial situation in the past year and were better able to plan for the future.

Survey responses also suggest that FSS led to a smal , statistically significant decrease (of 2 percentage points) in the receipt of Temporary Assistance for Needy Families (TANF) cash welfare benefits. FSS households need to remain off TANF for at least 12 months to graduate from the program. Reductions in the receipt of TANF

benefits will possibly continue or become larger in later years, as FSS group members approach the end of their FSS contract. No differences in the receipt of SNAP benefits were observed between groups.

The FSS program does not require families to give up their housing assistance once they graduate. They may be required by the voucher program to exit if their income exceeds the maximum al owed for eligibility for a housing subsidy. Based on HUD administrative data, 81 percent of the FSS group and 79 percent of the control group continued to receive a housing subsidy at the end of the 3-year fol owup period, a 2 percentage-point difference that is not statistically significant. Both groups averaged receiving roughly similar levels of housing subsidies. In the absence of employment and earnings effects in the first 3 years, these results are not surprising. Future analysis will examine whether effects on housing outcomes emerge during the longer term.

3

The more disadvantaged subgroups are those who are not employed at random assignment, have no educational degree or credential, have an annual household income of up to $10,000, and are paying up to 25 percent of their housing expenses out of pocket.

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation xxii

Executive Summary

Exhibit ES.4. Impacts on Household Income, Benefit Receipt, Housing Assistance, and Financial Wel -Being, Year 3

FSS

Control

Difference

Outcome

Group

Group

(Impact)

p-Value

Survey responses

Income and financial wel -being

Average total household income ($)

1,686

1,633

53

0.281

Hardship paying regular monthly bil s (%)

50.7

47.0

3.7*

0.089

Currently has bank account (%)

67.5

61.3

6.2***

0.001

Average savings ($)

145

170

-26

0.344

Average debt ($)

15,403

13,778

1,625**

0.045

Financial situation is better than last year (%)

54.3

47.4

6.9***

0.003

Benefit receipt in month before interview (%)

Received SNAP/food stamps

57.3

58.3

-1.0

0.622

Received TANF

9.6

11.9

-2.3*

0.081

Sample size (total = 1,936)

988

948

Administrative Data

Housing assistance

Enrol ed in HCV program in Month 36 (%)

80.9

78.6

2.3

0.136

Total housing subsidy in year 3 ($)

8,313

8,511

-198

0.309

Sample size (total = 2,548)

1,281

1,267

FSS = Family Self-Sufficiency. HCV = Housing Choice Voucher. SNAP = Supplemental Nutrition Assistance Program. TANF =

Temporary Assistance for Needy Families.

NOTES: The FSS 36-Month Survey respondent sample includes HCV heads of household who were randomly assigned between October 18, 2013, and December 22, 2014, were age 18 to 61 at the time of random assignment and responded to the FSS 36-Month Survey. The FSS impact sample includes HCV households that were randomly assigned between October 18, 2013, and December 22, 2014, and with a head of household age 18 to 61 at the time of random assignment. Sample sizes for specific outcomes may vary because of missing values. Estimates were regression-adjusted using ordinary least squares, controlling for differences in sample member characteristics recorded at the time of random assignment. No special weights were applied to responses to adjust for differences in sample size by housing authority. Rounding may cause slight discrepancies in calculating differences. A two-tailed t-test was applied to differences between the FSS group and the control group for continuous variables and selected outcomes expressed as proportions. The p-value indicates the likelihood that the difference between the FSS group and control group arose by chance. Statistical significance levels are indicated as: * = 10 percent; ** = 5 percent; *** = 1 percent.

SOURCES: MDRC calculations using responses to the FSS 36-Month Survey and HUD Inventory Management System/Office of Public and Indian Housing Information Center data.

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation xxiii

Executive Summary

Variation Across Sites in Program Impacts

An important element of this evaluation is to examine how program effects vary across the types of FSS

programs. These effects could vary across PHAs because of differences in how FSS administrators design program policies and set expectations for program participants. Most PHAs operate relatively small FSS

programs, making it difficult, however, to conduct an independent assessment of the effects of any single FSS

program.4 This report, as with the last one, continues to explore patterns of variation for clusters of programs, a topic for continued exploration through the end of the evaluation.

To examine PHA-level variation in participation and impacts, PHAs with similar implementation features were grouped into a series of “site clusters,” and the magnitude of employment and earnings impacts estimated for each cluster. For instance, the data used to group PHAs into low, medium, and high monitoring and engagement emphasis clusters are based on three program implementation features and practices at study launch: caseload sizes, expectations about program contact, and the program’s focus on short-term goals. Sites classified as high monitoring and engagement sites have smal er caseloads, expect participants to maintain more frequent contact with FSS coordinators and focus on having participants establish short-term goals.5

The percentage of FSS group members who ever participated in one or more FSS-related service in Years 1

to 3 was highest in housing agencies that most strongly emphasize monitoring and engagement. Similarly, FSS

group members in these housing agencies were more likely to have a record of service use and employment during Year 3. These PHAs also record the highest rate of having three contacts or more during Year 3.

So far, only one test shows statistically significant variation in impacts among site clusters. Participants affiliated with PHAs with a strong emphasis on monitoring and engagement averaged $8,000 less in total earnings over 3 years compared with the control group. In contrast, the FSS program led to slightly positive but not statistically significant effects on earnings in PHAs with a low and medium emphasis on monitoring and engagement. The negative impact for PHAs with a high emphasis on monitoring and engagement possibly reflects the tendency of these programs to encourage FSS group members to participate in FSS activities, which may decrease their hours or weeks of employment. If so, this negative impact could diminish or turn positive after FSS group members complete their participation. The longer-term data will be important to more fully analyze and interpret this pattern.

Additional findings based on survey data show that FSS programs with a strong emphasis on monitoring and engagement did not lead to better outcomes in other measures of financial or personal wel -being compared with other types of programs. FSS programs with a high emphasis on monitoring and engagement led to relatively large and positive impacts on having a savings or checking account, but other types of FSS programs also led to increases in this outcome that were nearly as large. Likewise, all three types of programs increased the proportion of study participants who assessed their financial situation as better than the previous year, but only the largest increase, recorded for programs with a low emphasis on monitoring and engagement, is statistically significant.

At present, the researchers can only conjecture as to what implementation features are associated with the patterns of impacts on employment and earnings. Additional tests of variation in impacts by PHAs will be examined in future years as FSS group members reach the end of their FSS contract.

4

The national evaluation includes 18 PHAs and clusters them on various dimensions of program practice to assess variation in outcomes and impacts.

5

Similar tests were run for PHAs that varied in the types of services that they emphasized during the goal-setting stage (job search and post-employment support, education and training, or financial counseling and workshops) and also for PHAs with similar performance levels as measured by HUD for an earlier (pre-evaluation) cohort of FSS participants.

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation xxiv

Executive Summary

Looking Forward

This report presents extensive information on program participation patterns and outcomes of study participants roughly halfway through the FSS program. The remainder of the evaluation, slated to end in 2022 and covering 6 to 7 years of fol owup for the study sample, will al ow the evaluation to track participant outcomes through the end of their FSS contracts and will also examine post-FSS circumstances for former participants. Two additional reports will be published during this timeframe, relying largely on PHA, HUD, and NDNH data. The long-term housing and employment and earnings analyses will also be supplemented with a final round of interviews with program staff members to understand how they work with families to help them prepare for graduation and their exit from FSS. In addition, these reports will incorporate findings from two new study components that will extend the scope of the current evaluation and inform our understanding of the FSS program and its effects: (1) a longer-term fol owup survey of study participants, including FSS participants who graduated from the program (the successful exits) as well as those who were terminated from FSS, voluntarily exited the FSS program, or are no longer receiving HCVs, and (2) an analysis of a range of financial security and wel -being outcomes, based on credit data acquired from Experian. The FSS program’s emphasis on financial literacy and homeownership preparation may lead to improved credit scores, lower levels of accumulated debt, and better management of household finances, compared with the control group. The longer timeframe and more comprehensive data will support a more complete assessment of FSS and the difference it makes in helping families make progress toward self-sufficiency.

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation xxv

Chapter 1. Introduction

Chapter 1. Introduction

Since the mid-1980s, the U.S. Department of Housing and Urban Development (HUD) has supported efforts to improve the economic wel -being of public housing residents or households receiving rental assistance under the federal Housing Choice Voucher (HCV) program (also known as “Section 8,” after Section 8 of the Housing Act of 1937).6 The Family Self-Sufficiency (FSS) program, a voluntary program, is one such effort. In this approach, the provision of the rental subsidy is augmented by services and a long-term escrow account to support participants’ economic mobility. Evidence on the ways in which the FSS program affects the economic mobility of program participants is generally limited, and until recently, little of that evidence was based on random assignment experiments or strong quasi-experimental research. To build rigorous evidence about the effectiveness of the FSS program, HUD commissioned the first national random assignment evaluation of this program in 2012. This report, the second in a series, documents interim results on program participation and effects around Year 3 of fol owup, roughly halfway through the 5-year program.

Annually, about 700 public housing agencies (PHAs) are awarded grants to implement FSS in public housing and HCV rental assistance programs.7 The funding provides PHAs modest resources to hire coordinators (or case managers or coaches) to work with participants to set self-sufficiency goals and to refer them to services in their communities. At program enrol ment, participants sign a Contract of Participation (COP) and complete an Individual Training and Services Plan (ITSP). The typical FSS contract can last for up to 5 years, during which participants are expected to achieve all agreed-upon goals. FSS also includes an escrow account, or an asset-building feature, that is designed to encourage participants to go to work, increase their earnings, and build savings in an interest-bearing account, which the housing agency maintains. Like others receiving housing assistance, FSS participants see their rents increase when their earnings increase, but in the FSS

program, the housing agencies credit the family’s escrow account with an amount based on their rent increase.

These escrow accruals are paid to participants once they graduate from the program—that is, when they are employed and have met all goals outlined in their COP and ITSP. Nongraduates forfeit their escrow accruals.8

PHAs may al ow participants to withdraw accrued escrow funds while they are in the program in order to pursue their goals.

The national evaluation uses a randomized control ed trial to test whether FSS produces its intended effects of increasing employment and earnings and improving a broad range of quality-of-life outcomes for HCV

households. Randomized control ed trials employ an experimental design that compares the outcomes of a program group whose members are eligible to participate in the intervention with those of a control group whose members are not eligible to participate in the intervention. Random assignment of study participants to either a program group or a control group is designed to ensure that the individuals in the program and control groups are similar at the start of the study.9 Differences between the program and control groups’

outcomes reflect the program’s impacts. Statistically significant differences indicate that the impacts can be attributed with a high degree of confidence to the intervention rather than to chance.

The evaluation focuses on non-Moving to Work (MTW) housing agencies that are operating FSS for HCV holders. At the time of study design, MDRC and HUD agreed to exclude MTW PHAs, which 6

Project Self-Sufficiency, Operation Bootstrap, and the Jobs Plus demonstration are some examples of the efforts to use housing assistance as a platform to support the work outcomes of housing-assisted households.

7

This report uses the terms public housing agencies and public housing authorities, both PHAs, interchangeably.

8

For FSS participants in the HCV program, any escrow forfeiture becomes part of housing assistance payment (HAP) equity and is restricted to HAP activities. See HUD (2015).

9

Randomization does not affect—or limit—study participants’ use of resources that are available in the general community.

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 1

Chapter 1. Introduction

have congressionally authorized administrative flexibility to modify their FSS programs without legislative or regulatory change, and FSS programs for public housing residents, which serve considerably fewer participants. Eighteen non-MTW housing agencies operating FSS for tenant-based housing voucher recipients agreed to participate in this evaluation and together enrol ed 2,656 voucher holders in the study.10

The first report from this evaluation introduced the study design and detailed the findings on the first 18 to 24 months of fol owup (Verma et al., 2019). It documented FSS implementation strategies across the 18 sites, participants’ engagement, and program impacts on labor force outcomes and receipt of government benefits.

That report showed that PHAs have substantial discretion over FSS program implementation, leading to broad variation in how case management services are delivered. The early data also showed that the program increased participants’ engagement in a range of employment-related services, relative to the control group.

The earlier report documented that FSS had produced small shifts from part-time to ful -time employment among FSS participants, relative to the control group, but there was no evidence that the program increased employment or average earnings over the first 24 months of fol owup. This report, the second in a series of reports on the ongoing evaluation, extends the above analysis through the end of Year 3. Since FSS is a 5-year program, this report examines whether bigger differences between the program and control groups begin to emerge over a longer-term fol owup period. The report draws on a new data source, the 36-month survey, to look at the program’s effects on a broader range of outcomes, including household material, financial, and economic wel -being. Additional reports slated as part of this evaluation will look at outcomes in Years 6 and 7, well after the end of the program for most participants.11

The Housing Assistance and Employment Context

Low-income renters receive federal housing assistance by three primary means: HCV, project-based rental assistance, and public housing assistance. The HCV program is the nation’s largest rental assistance program, providing subsidies to slightly more than 2 mil ion low-income households. Administered by 2,150 local housing agencies, the HCV program al ows families to rent a unit in a neighborhood of their choice, if the housing meets HUD inspection standards and the landlord is wil ing to accept housing vouchers. Households contribute 30 percent of their monthly income to their rent (minus certain adjustments to defray childcare expenses or for other reasons), and the HCV program covers the rest of the rent and utilities expenses, up to a locally determined maximum (also referred to as the payment standard).

Housing vouchers became part of U.S. housing policy in the 1970s.12 Eligibility for housing vouchers is limited to households with income under 80 percent of the median income for the metropolitan area or county in which they choose to live. However, the program gives priority to extremely low-income households by reserving at least 75 percent of available vouchers each year for households with income at or below 30 percent of the area median income. No time limits are imposed on the duration of subsidy receipt, but the program does restrict the amount of income a household may receive and remain eligible for this benefit. If a housing voucher holder’s household income exceeds the limit of 80 percent of the area median income for 6 consecutive months, the household eligibility for the subsidy ends. Congress provides funding annually for all current voucher holders, although no statutory guarantee of permanent renewals exists.

10

In fiscal year 2014, funding streams for HCV and public housing FSS programs were merged, and housing authorities could submit one application for their annual grants. This consolidation of funding streams also meant that PHAs could use the funding to serve both public housing and HCV FSS programs if applicable.

11

In 2018, HUD extended the national FSS evaluation through 2021, allowing 6 to 7 years of followup for the study sample. With the exceptions of participants who receive an extension, most FSS participants will reach the 5-year term of the program by 2020. This longer-term study will also include a special analysis of credit data and related financial well-being outcomes.

12

See Schwartz (2006) for additional background information on the HCV program.

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 2

Chapter 1. Introduction

The HCV program serves large numbers of households with elderly members and members with disabilities, and nonelderly and nondisabled households account for only about 49 percent of all voucher-assisted households (CBPP, 2017).13 A 2016 analysis of HUD administrative data shows that a large majority of the voucher population works: roughly 69 percent of nonelderly, nondisabled households using vouchers were working or had worked recently. The employment and earnings trajectories of nonelderly, nondisabled housing voucher holders, however, have been a long-standing policy concern. Given both the potential employment advantage that voucher receipt may offer and the potential work disincentives inherent in various government assistance programs (because higher earnings generate higher rent or reduced benefits), researchers and policymakers have raised questions about the expected effects of employment-focused programs like FSS.

As with any means-tested program, the provision of a government benefit has the potential to affect the recipients’ work effort. Some analysts have argued that the provision of housing subsidies not only improve access to decent housing but may also—in and of itself—promote work.14 This view holds that the housing stability that comes from rent subsidies may enable recipients to focus on employment or build human capital, and that when housing assistance takes the form of vouchers, households are able to move to better quality neighborhoods that offer more or better employment opportunities.15 This view, however, is chal enged by evidence that seems to suggest that although many households undoubtedly do benefit in selected ways, housing assistance alone may not, on average, improve employment outcomes (Jacob and Ludwig, 2008; Mil s et al., 2006; Shroder, 2010).16 In this case, voucher holders may feel less pressure to work when their housing expenses are subsidized and their remaining income is adequate to sustain the family without the cost of seeking work (because of transportation expenses, for example) or finding adequate childcare while working.

Similarly, the HCV program’s rent rules could also discourage work. Voucher holders must pay 30 percent of any earnings for rent, up to the point that they are no longer eligible for this subsidy. Thus, their participation in the HCV program subjects them to an implicit “tax” on additional earnings that could negatively affect their inclination to work (Popkin et al., 2000, 2010; Popkin, Cunningham, and Burt, 2005). The FSS

program’s interest-bearing escrow account is intended to address this issue: Program participants may be permitted to receive interim disbursements or access their full escrow savings on graduation from the program.

The Family Self-Sufficiency Program

Established in 1990 by Section 554 of the Cranston-Gonzalez National Affordable Housing Act, the FSS

program emerged against a backdrop of policy discussions about persistent poverty among participants of government benefit programs. Mostly operated by housing agencies administering public housing or HCV programs, FSS programs reach a small fraction of all voucher families—a reflection of the funding appropriated to operate this program.17 Beyond the program size constraints imposed by funding, there is generally limited information about the reasons why many voucher households do not enroll in FSS. Early research on this topic has shown that, at least from the program operators’ perspectives, the potential fear on the part of voucher recipients that they may lose their housing assistance and other public assistance benefits 13

Households that have elderly members or members with disabilities make up the remaining 51 percent.

14

See Sard and Waller (2002) for one discussion on this perspective.

15

Recent research has also shown positive long-term effects for young children of the original Moving to Opportunity demonstration and, in a separate nationwide study, the benefits that low-income children experience in adulthood when they move to high-opportunity neighborhoods as children (Chetty and Hendren, 2017a, 2017b).

16

For example, the findings from the Welfare-to-Work program conducted in the early 2000s found that having and using a voucher reduced employment rates and earnings amounts in the first year or two after random assignment, but the small negative effect of vouchers disappeared over time, and vouchers had no significant effect overall on employment and earnings during 3.5 years of followup. The most rigorous evidence from the United States suggests a loss of 10 to 20 cents in earnings per dollar of assistance (see Shroder, 2010).

17

HUD makes funding available for FSS programs through the annual grants, but the funding is limited to the amount that Congress appropriates.

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 3

Chapter 1. Introduction

for not fulfil ing the FSS contract is a crucial factor that might keep participants from enrol ing in the program (Rohe and Kleit, 1999). These program operators also saw issues like family responsibilities, potential lack of motivation, childcare or transportation problems, and distrust of social programs as possible barriers (Rohe, 1995; Rohe and Kleit, 1999). MDRC’s own interviews with program staff members during the site recruitment process for the national evaluation evoked similar responses.

In the most recent round of annual grants, HUD made available about $75 mil ion in funding for the FSS

program (HUD, 2018a).18 The FSS grants offer support for coordinator positions, with no provisions for program management or other related administrative costs.19 Fiscal year 2017 grants show that the size of the programs funded can range from as few as 15 participants in the smal est program (funded for only a part-time coordinator position) to more than 1,000 in the largest.20 Thus, although FSS is the only HUD program aimed at helping voucher holders improve their work outcomes and reduce their need for housing subsidies and other government benefits, it remains a small program at the federal and local levels.21

In 2017, HUD published its first comprehensive resource guide for program operators. Without enforcing a particular service delivery framework, the guide provides practical, hands-on tips for operating the FSS

program (HUD, 2017b). Housing agencies operating the FSS program are required to prepare an FSS Action Plan and have it approved by HUD. This document is expected to detail program parameters—for instance, size and population served, types of services that will be offered, and program rules and policies. Once HUD

approves an action plan, the PHA does not need to resubmit it to HUD for approval unless the PHA makes policy or other changes to the program.

Starting in 2018, HUD also rol ed out a performance measurement system for assessing programs that receive HUD FSS funding.22 The performance score, a composite, is based on three measures: the extent to which the earnings of FSS participants increase over time after joining the FSS program, the FSS graduation rate, and the portion of expected participants served.23 A 2018 announcement in the Federal Register informed housing agencies of the criteria for evaluating FSS programs. HUD plans to use the performance measures to identify high-performing and low-performing FSS programs, which could inform its understanding of best practices and its delivery of technical assistance.24 The introduction of the FSS performance measurement system establishes a new monitoring context for FSS programs nationwide.

Core Features

Guided by statutory requirements and HUD regulations, the FSS program is structured around two basic components: an escrow savings account with a behavioral condition for its release (a longer term financial incentive for households to increase work and earnings, described in more detail in the fol owing sections) and coordination of supportive services (exhibit 1.1). Except for the escrow account, local housing agencies can decide how to structure their case management and case coordination services—an element of flexibility captured in this evaluation.

18

A small fraction of HUD’s budget is reserved for self-sufficiency efforts.

19

HUD funds the FSS programs through the FSS Notice of Funding Availability (NOFA). Housing authorities apply for this funding on an annual basis.

20

These findings are from an MDRC analysis of HUD FSS grant awards.

21

Changes to rent policies could also be used to encourage work among HCV participants, as is being tested as part of HUD’s Rent Reform Demonstration (see Riccio, Deitch, and Verma, 2017).

22

See HUD (2018b).

23

The three measures are weighted as follows: earnings (50 percent), graduation rate (30 percent), and participation rate (20 percent).

24

Toward these goals, at least once per year, HUD will analyze data collected through the Public Housing Information Center to calculate FSS performance scores for each FSS program that received an FSS coordinator grant in one or more of the past three fiscal year NOFA competitions.

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 4

Chapter 1. Introduction

Exhibit 1.1. Core Components of the HUD Family Self-Sufficiency Framework

Program Offers

Graduation Requirements

Head of Household Employeda

Tenants volunteer for FSS

No cash welfare for

and complete a Contract of

Case Management

Interest-Bearing

any household member

Participation and Individual

and

Escrow Savings

in the 12 months leading

Training and Services Plan

Referrals to Services

Account

up to graduation

Other Agreed-Upon Goals

Program Coordinating Committee

composed of service providers,

offers guidance

Up to 5 years to achieve goals

and graduate from the program

aRequirements can vary.

and earn escrow

FSS = Family Self-Sufficiency.

Participation in the FSS program is voluntary. Housing agencies promote the program through various means, including flyers and program brochures in housing application packets; notifications by way of community partners, PHA newsletters, and websites; and group-orientation sessions at the housing agency. Informal channels, such as referrals from friends and relatives, also help spread the word about the program. Once participants enrol , the case coordination services offered by the program are designed to help participants access services that will help them achieve their goals. Although all adults in FSS households are encouraged to seek employment, only the household head—the voucher holder—is required to meet the employment goals of the FSS contract in order to graduate and col ect escrow. On meeting these goals (usually within 5 years), household heads graduate from the FSS program and can access the escrow savings their household accrued (unless participants qualify for an interim withdrawal (see below), they can only access their escrow once they graduate from FSS).

The FSS program’s escrow account is intended to serve both as an incentive for households to go to work and to increase their earnings and as a long-term savings vehicle to help them build financial assets. Housing agencies place no restrictions on participants’ use of escrow funds (unusual for asset-building programs), but they report that households most commonly use their resources to start a new business, repair credit, buy a home, or pay for education.25 Some programs also consider interim disbursements, as long as participants use the funds to meet approved expenses related to their self-sufficiency goals. Partial payments of the escrow 25

The 42-month survey conducted as part of the Work Rewards demonstration sheds some light on the desired uses of the escrow. Administered before the escrow funds had been disbursed to graduates, the Work Regards survey showed that about one-third of the respondents indicated that they would save their escrow money for an emergency. Other uses included saving for children’s future educational expenses, paying for basic necessities, and buying a house. See Verma et al., 2017.

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 5

Chapter 1. Introduction

before program graduation can be approved for expenditures such as tuition, car purchase, credit repair, uniforms, tools, homeownership, or business startup.

Little published data is available on the extent to which participants accrue escrow and, among program graduates, how much escrow they have at graduation. A recent HUD report to Congress, focusing on the July 2017 to June 2018 period, shows that 47 percent of graduates during this period had escrow savings averaging approximately $7,700.26 The Opportunity NYC‒Work Rewards (hereafter, Work Rewards) demonstration tested the effects of regular FSS (FSS-only) and an enhanced version of the FSS program that offered special cash incentives to encourage work (FSS-plus-incentives) against a control group. This demonstration provides the first complete evidence about the extent to which cohorts of participants graduate from the program and earn an escrow disbursement. Fol owing the FSS participants over 6 years, the study showed that about 45

of the FSS participants graduated and received escrow disbursements. Households in the FSS-only group received an average of about $3,800 (the FSS-plus-incentives group, which received FSS and two additional special workforce incentives, received nearly $700 more in escrow disbursements on average than the FSS-only group). Escrow disbursements covered a wide range of amounts, with the bottom quartile of payments averaging less than $1,000, and the top quartile averaging more than $15,000 (Verma et al., 2017).

To graduate from FSS, a HUD measure of program success, the head of household must complete all the goals and activities listed in the ITSP, be employed, and become independent of public cash assistance. The welfare receipt requirement applies to all members of the households, and no member of the household should receive Temporary Assistance for Needy Families (TANF) cash assistance for at least 12 months before graduation.27

Some FSS programs also require participants to work toward additional goals (related to financial security, for example) to graduate.28 The welfare requirement, in contrast, is implemented uniformly across all sites. If the head of household is not employed and someone other than the head of household achieves increases in earned income, the family is not eligible to receive escrow at the time of graduation, a potentially problematic aspect of the escrow component for households with multiple adults. In addition, it is also possible for participants to graduate from the program and not receive any escrow funds; this could happen for various reasons, including not having had the earned income increases that are necessary to trigger escrow accumulation. Thus, it cannot be assumed that all FSS participants who graduate from the program do so with some amount of escrow.29

One other HUD requirement that applies to all FSS programs relates to the Program Coordinating Committee (PCC). With referrals being central to the FSS service delivery model, the intent of the PCC is to create a mechanism by which the service providers in the community can become invested in the success of the FSS program. The PCC, which operates as a col aborative group, is intended to provide both guidance to the housing agency administering the FSS program and direct services to clients; it usually comprises some or al of the service providers that accept FSS referrals.

26

See HUD, n.d.a. “Family Self-Sufficiency 2020 Summary of Resources.”

27

According to HUD rules, receipt of the following is not considered welfare assistance: food stamps through the Supplemental Nutrition Assistance Program (SNAP), medical assistance, childcare assistance, work supports such as transportation assistance or short-term benefits under TANF, or disability benefits for the Head of Household or another family member.

28

Setting additional graduation requirements is not consistent with HUD regulations or guidance on the program. As discussed in chapter 2, it is unclear whether FSS staff members set these additional benchmarks and actually terminate participants if they do not achieve them or whether these are used as ways to motivate participants to achieve what the staff members see as necessary goals for participants to truly become self-sufficient. This topic will be examined as part of the ongoing evaluation.

29

In Work Rewards, which followed a cohort of participants for 6 years after they enrolled in FSS, 30 percent of FSS participants who graduated did not receive an escrow disbursement. According to a recent HUD analysis, 63 percent of participants earn some escrow while in the program; 47 percent of graduates earn some amount of escrow. See HUD, n.d.a., “Family Self-Sufficiency 2020 Summary of Resources.”

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 6

Chapter 1. Introduction

How Family Self-Sufficiency Might Help Participants Advance

Exhibit 1.2 offers a simplified schematic to identify the main pathways by which the FSS program might increase participants’ work and earnings and improve their financial wel -being. At the most basic level, the program’s hypothesized short- and long-term effects might operate via two mechanisms.

Mechanism 1: Increase the Payoff Through Case Management

Although FSS programs may vary in their service delivery approaches, they all have some dimension of goal setting and case management (or coordination) that includes needs assessments and referrals to services that may help participants address some of the chal enges that might come in the way of work. Typically, case managers work with each participant (and sometimes other members of the household) to identify goals the participant will aim to achieve during the 5 years of program participation. During this process, case managers and participants discuss the types of supports participants might need to advance toward their goals.

The supports might include, for example—

• Securing quality, low-cost childcare to make balancing work and home life commitments more feasible.

• Engaging in and completing education and training to improve employment prospects and create pathways for advancement.

• Finding and maintaining stable employment.

• Establishing, repairing, or improving the participant’s credit score to increase employment prospects and decrease reliance on high-cost alternative credit sources like pawn, automobile-title, and payday loans.

Progress along each of these pathways would make it easier and more remunerative to work. Furthermore, some of these pathways, such as credit score improvement, may also help participants manage their financial resources and thus improve material hardship irrespective of the program’s impact on employment and earnings.

Improving outcomes, such as education or credit, for example, may be difficult with a program that offers less intense case management or coordination services (as in a “light-touch” services approach, which may be less structured, require infrequent contact with staff, and lack a strong monitoring and engagement focus).

In general, outcomes will depend on several factors, including the strength of the service providers (and the service providers’ models) in the local community, the case management model (including the type and frequency of fol owup), and the capacity and wil ingness of the participant to fol ow through on a course of action—something that is directly targeted in other interventions that apply a more behavioral science–

informed coaching approach (Guare and Dawson, 2016) but is somewhat weakly targeted in FSS given the fairly far-off possible reward of the escrow disbursement.30 Because some of these factors vary among the housing agencies in this study, the evaluation will explore the ways in which different program practices affect participant outcomes, if at al .

30

These efforts, which focus on “executive skills”—or roughly, the capacity to plan, manage, and cope—attempt to achieve larger impacts than are typically achieved with conventional case management. MDRC’s MyGoals Demonstration is testing the effects of a structured, executive skills–informed coaching and incentives program on work outcomes.

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 7

Chapter 1. Introduction

Mechanism 2: Provide an Incentive to Work Through Escrow

As described, the escrow account is designed, in part, to counteract the disincentive effect of the implicit “tax”

built into HCV rent rules; specifically, 30 percent of a recipient’s earnings must be contributed to rent, so 30 percent of any earnings gains are diverted to increased rent payment. This rent policy could discourage additional work by decreasing the marginal gain for any added hour of work done. It may also discourage any work effort among those not working by reducing the effective wage rate below the “reservation wage,” the wage rate necessary to induce potential workers to enter or reenter the labor market.

The degree to which this implicit tax on wages acts as a disincentive to work, or to work harder or find a better paying job, is not well established. Consequently, it is difficult to estimate the potential impact of a program like FSS—and specifically its escrow component that is intended to cancel out that disincentive.

Two additional aspects are to be considered: the effectiveness of escrow as an incentive and the factors that may constrain participants’ response. First, escrow represents an incentive to work, but the incentive cannot be earned until graduation requirements are fulfil ed, so it is a distant and uncertain reward. It may not, therefore, effectively (or completely) counteract any disincentive effect of the HCV rent rules because those costs are immediate and certain. The current structure of the escrow account was the impetus for testing an offer of more immediate, work-related cash incentives alongside the normal FSS program escrow incentive as part of the Work Rewards demonstration.

Second, FSS participants face a variety of barriers that may limit their employment prospects and increase the costs of work (beyond the contours of the local job market and transportation-spatial infrastructure). Like most low-income households in the country, participants may receive multiple means-tested benefits, including Supplemental Nutrition Assistance Program, or SNAP, and TANF benefits. Aspects of these benefits work in ways similar to the HCV rent rules, and uncertainty or fear about decreases in or loss of these benefits could discourage work effort. Poor educational attainment, criminal history, and poor or no credit history may limit the types of jobs participants can qualify for and obtain, thus reducing the payoff from work or increased hours. Likewise, family obligations and responsibilities (such as taking care of children or loved ones with disabilities or who are sick), being sick or having disabilities, and the need to secure employment-related transportation may further discourage work or additional hours by increasing the costs associated with employment (by reducing the effective wage, potentially below zero). For some, a cost-benefit calculation of minimum wage with uncertain hours may conclude that, at least in the short run, not working is a better choice for the family. Thus, in isolation, the HCV rules may constitute only a small part of the decision not to work or not to work more. For this reason, case management and case coordination services, described previously, may be important components.

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 8

Chapter 1. Introduction

Exhibit 1.2. Simplified Schematic of the Family Self-Sufficiency Program Theory of Change FSS

Interim Outcomes

Long-Term Outcomes

Increased Financial Security

Initial and ongoing coordination—

Increased Participation in

and referral to—range of services:

Education and Training / Supportive

Improved Material Well-Being

Services / Workshops / Classes

• Employment

Reduced Reliance on

• Education and Training

Government Assistance

• Financial Literacy/Management

• Homeownership

• Health-Related Services

• Social Services

• Supportive Services

Increased Employment and Earnings

Incentive to Increase Earnings

Through the Long-Term Saving

Escrow Received

Escrow Account

Escrow Accrual

at Graduation

FSS = Family Self-Sufficiency.

Drawing on this conceptual framework, this ongoing evaluation assesses how a self-sufficiency program like FSS affects a range of economic mobility outcomes for program participants. It also examines the effects for certain subgroups to better understand what works best for whom. For example, it is possible that the program may have larger effects for participants who are not employed at study enrol ment, because it is often easier for individuals to advance to higher wage jobs once they are already employed than to get a job in the first place. Given the case management supports, the FSS program may also have different effects depending on a participant’s barriers to work or preparation for work. Based on program theory, prior evidence, or policy interest in a given subgroup, the evaluation focuses on subgroups defined by participant characteristics at enrol ment: work status and educational attainment.31 In addition, given the variation in FSS implementation practices across sites, the subgroup analysis also considers program impacts for participants who are exposed to different program engagement and implementation strategies or program “types.”

31

These subgroups were identified during the design stage. Additional subgroups examined are mainly considered exploratory analyses.

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 9

Chapter 1. Introduction

The National Family Self-Sufficiency Impact Evaluation

Until recently, questions about the FSS program’s effectiveness had not been investigated using methods that would support unambiguous causal inferences.32 Thus, an overarching goal of the national evaluation is to build conclusive and comprehensive evidence about the program’s effects. In other words, does FSS improve the employment, earnings, income, and financial wel -being of participants, over and above the levels for a control group with similar characteristics and work histories and similar interest in receiving employment-related or financial security-related services but who are not al owed to enroll in FSS? How do different groups of participants respond to FSS? Do the effects vary across types of people participating in the program or in response to different types of implementation practices? Does the intervention produce positive benefit-cost results from the perspective of study participants or HUD and participating housing authorities? To address this broad range of questions, the evaluation includes a comprehensive, multisite study of FSS programs, consisting of three core study components: implementation research, impact analysis, and benefit-cost analysis.33 Exhibit 1.3 lists the data sources and the fol owup included in this report.34

Exhibit 1.3. Data Sources for the Family Self-Sufficiency Study

Data

Data Period

Length of Followup

Baseline characteristics

October 2013–December 2014

At random assignment

Wage records

April 2013–December 2017

36 months (12 quarters)

Program participation and escrow data

October 2013–December 2017

36 months

Housing subsidy data

October 2013–December 2017

36 months

36-month survey

February 2017–August 2017

29–46 months

Eighteen housing authorities in seven states—California, Florida, Maryland, Missouri, New Jersey, Ohio, and Texas—were selected to participate in the FSS study (see exhibit 1.4). These sites are broadly representative of the contexts within which FSS programs operate.35 The sites participating include those running smal , mid-sized, and large FSS programs and smal , mid-sized, and large voucher programs. Although no data were available to distinguish typical or higher quality FSS programs during site recruitment, MDRC and HUD sought to include a broad range of sites, including ones with different program sizes, staff caseloads, and case management or coordination practices and unique program-implementation features.36 The site-specific enrol ments, including program and control group members, ranged from 50 to 350, reflecting varying enrol ment targets based on the sizes of the programs.37

32

Nationally, the Work Rewards demonstration was the first to use a random assignment design to test the effects of an FSS program (Nuñez, Verma, and Yang, 2015; Verma et al., 2012, 2017). Results from that study, which focused on a single site, showed that the program was effective in enrolling participants in education and training activities or linking them to financial literacy programs, but there were few gains in the range of economic and material outcomes tracked for the sample. A small quasi-experimental analysis of the Compass Working Capital FSS programs in Lynn and Cambridge, Massachusetts, showed that the Compass FSS programs were associated with an average gain in annual household earnings of $6,305 between the fourth quarter of 2010 and the first quarter of 2016 (Geyer et al., 2017).

33

Verma et al. (2019) includes additional information about the study components.

34

The 36-month survey achieved a 77 percent response rate, and there is little evidence of bias or potential problems in generalizing the results to the larger impact sample (see appendix F). Additional details on the study data sources are provided in the following chapters.

35

See Verma et al. (2019) for site recruitment details. The site selection approach considered various factors, such as program size, the possibility of building clusters of sites within states, regional and local diversity, and varying program approaches. The research team examined HUD

data from 2010 to 2012, creating a list of potential sites; conducted phone reconnaissance with about 60 program administrators; visited 27

sites; and ultimately negotiated agreements with 18 sites.

36

HUD’s performance management system ranking for FSS programs was not available until 2018. The ranking system classifies 20 percent of FSS programs as high performing, 60 percent as standard, 10 percent as low, and 10 percent as troubled.

37

Seven PHAs agreed to enroll under 100 study participants, five agreed to enroll between 100 and 200, and another six agreed to enroll between 200 and 350. PHAs operating larger FSS and HCV programs were more likely to agree to larger enrollment targets.

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 10

Chapter 1. Introduction

The Study Sample

From October 2013 to December 2014, the 18 public housing agencies enrol ed and randomly assigned 2,656

households. This figure includes a small number of households that later withdrew voluntarily from the study or that program staff members determined to have been ineligible for FSS at their time of random assignment and removed from the study, as well as households headed by elderly individuals (62 years or older), who are not the focus of the main impact analysis—excluding these individuals reduced the sample to 2,556. These 2,556 study participants compose the sample for the entire impact analyses in this report (also referred to as the

“impact sample” in relevant exhibits).

Exhibits 1.5 and 1.6 present sample characteristics from a survey (also referred to as the Baseline Information Form, or BIF) that participants completed at the time of study enrol ment.38 As shown, the sample is largely composed of households with children, and 76 percent of households included a minor child. Nearly 34 percent of participant households included another adult. Although labor market outcomes for other adult household members are not analyzed here, their earnings do contribute to household escrow accumulation, affect the household’s subsidy, and may benefit directly or indirectly from FSS case management. Approximately 70

percent of study households reported receiving SNAP benefits. The TANF benefit receipt rate was around 16 percent. The FSS program is designed to help participants move off cash assistance, such as TANF, and reduce reliance on public assistance in general.39 Slightly more than one-half of the sample (54.2 percent) reported having received Section 8 housing assistance for 6 years or less. About 31 percent reported having received Section 8 housing assistance for 10 years or more.

38

See appendix exhibits A.1 and A.2 for additional baseline characteristics on the program and control groups.

39

The Work Rewards data suggest more families may enter and exit the TANF system over the followup period.

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 11

Chapter 1. Introduction

Exhibit 1.4. Public Housing Agencies Participating in the National Family Self-Sufficiency Evaluation

OHIO

• Akron Metropolitan Housing Authority

MISSOURI

• Columbus Metropolitan Housing Authority

• Housing Authority of Kansas City

• Lucas Metropolitan

Housing Authority

• Youngstown Metropolitan

Housing Authority

NEW JERSEY

• Jersey City

Housing Authority

MARYLAND

• Baltimore County

Housing Office

• Housing Opportunities

Commission of

Montgomery County

CALIFORNIA

FLORIDA

• Housing Authority of the County of Alameda

• Housing Authority of the

• Housing Authority of the City of Los Angeles

TEXAS

City of Deerfield Beach

• Orange County Housing Authority

• Housing Authority of the City of Dallas

• Housing Authority of the

• Housing Authority of the County of Riverside

• Housing Authority of Fort Worth

City of Fort Lauderdale

• Houston Housing Authority

• Tarrant County Housing Assistance Office

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 12

Chapter 1. Introduction

Exhibit 1.5. Baseline Characteristics of Households in the Family Self-Sufficiency Impact Sample

Characteristic

Impact Sample

Average number of household membersa

3.2

Average number of adults in householda

1.5

Households with more than 1 adult (%)

33.7

Average number of children in household

1.8

Number of children in household (%)

0

23.8

1

22.7

2

24.7

3 or more

28.8

For households with children, age of youngest child (%)

0–2 years

20.8

3–5 years

20.4

6–12 years

41.3

13–17 years

17.5

Primary language spoken at home is English (%)

92.2

Receives TANF (%)

15.8

Receives food stamps/SNAP (%)

69.6

Length of time receiving Section 8 Housing Choice Voucher (%)

Less than 1 year

5.0

1–3 years

27.6

4–6 years

21.6

7–9 years

15.2

10 years or more

30.6

Total annual household income (%)

$0

4.5

$1–$4,999

17.0

$5,000–$9,999

18.7

$10,000–$19,999

31.9

$20,000–$29,999

19.3

$30,000 or more

8.5

(continued)

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 13

Chapter 1. Introduction

Exhibit 1.5. Baseline Characteristics of Households in the Family Self-Sufficiency Impact Sample continued

Characteristic

Impact Sample

Payment for rent and utilities (%)

$0

1.9

$1–$199

15.0

$200–$399

24.3

$400–$599

21.3

$600–$799

15.1

$800 or more

22.4

During the past 12 months, household experienced at least one financial hardship (%) 59.0

Not able to buy prescription drug

13.3

Not able to buy food

28.9

Not able to pay telephone bil

28.2

Not able to pay rent

18.5

Not able to pay utility bil

43.4

Sample size

2,556

SNAP = Supplemental Nutrition Assistance Program. TANF = Temporary Assistance for Needy Families.

aMaximum response option for number of adults in a household is four.

NOTES: The FSS impact sample includes Housing Choice Voucher households that were randomly assigned between October 18, 2013, and December 22, 2014, and with a head of household age 18 to 61 at the time of random assignment. Sample sizes for specific measures may vary because of missing values. Rounding may cause slight discrepancies in calculating sums. Detail may sum to more than total for questions that allow more than one response.

SOURCE: MDRC calculations from Baseline Information Form data.

In terms of their individual characteristics (see exhibit 1.6), the sample is predominantly female (90.6 percent), with an average age of 39 years at study enrol ment. About 14 percent of the sample did not have a General Educational Development (GED) certificate or high school diploma, suggesting that lack of education may represent an important barrier to address for this subset of the population. To explore the effect of baseline education, the evaluation examines how starting educational status affects program participation, employment, and other outcomes. Around 41 percent of the sample reported some other barriers to employment. Among these study participants, physical health (18.8 percent) and access to affordable childcare (17.8 percent) represent the most common difficulties. Approximately 56 percent of study participants were working at study enrol ment (with around 30.5 percent working full time). Work status and earnings are primary outcomes of interest for this study because FSS is designed to enable and encourage (more remunerative) work and because employment is one of the requirements for program graduation and access to accumulated escrow.

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 14

Chapter 1. Introduction

Exhibit 1.6. Baseline Characteristics of Heads of Households in the Family Self-Sufficiency Impact Sample

Characteristic

Impact Sample

Sample Member Characteristics

Female (%)

90.6

Age (%)

19–24 years

2.2

25–34 years

33.9

35–44 years

35.6

45–59 years

27.3

60–61 years

1.1

Average age (years)

39

Marital status (%)

Married, living with spouse

7.7

Married, not living with spouse

6.8

Cohabitating

1.4

Single, widowed, or divorced

84.0

Citizenship status (%)

U.S.-born

87.7

Naturalized

8.1

Noncitizen

4.2

Race/ethnicity (%)

Black, non-Hispanic/Latino

73.3

Hispanic/Latino

15.8

White, non-Hispanic/Latino

6.7

Other

4.2

Education

Highest degree or diploma earned (%)

GED certificate

3.0

High school diploma

10.6

Some col ege or received technical/trade license

55.0

Associate’s or 2-year col ege degree

10.8

4-year col ege or graduate degree

6.5

None of the above

14.0

(continued)

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 15

Chapter 1. Introduction

Exhibit 1.6. Baseline Characteristics of Heads of Households in the Family Self-Sufficiency Impact Sample continued

Characteristic

Impact Sample

Has trade license or training certificate (%)

47.0

Employment status

Currently employed (%)

56.2

Regular job

48.4

Self-employed

4.2

Temporary or seasonal job

3.5

Currently working 35 hours or more per week (%)

30.5

Average hours worked per week

18.3

Average weekly earnings ($)

213

Barriers to Employment

Has any problem that limits work (%)

41.2

Physical health

18.8

Emotional or mental health

7.6

Childcare access or cost

17.8

Need to care for disabled household member

7.3

Previously convicted of a felony

6.3

Limited English-speaking ability (%)

3.8

Does not have access to transportation for employment (%)

No access to public transportation

17.8

No access to an automobile

18.2

FSS Program

Heard of escrow before random assignment (%)

44.0

Interest in FSS services related to (%)

Job-related services

70.5

Social services

32.4

Financial services

95.5

Sample size

2,556

FSS = Family Self-Sufficiency. GED = General Educational Development.

NOTES: The FSS impact sample includes Housing Choice Voucher households that were randomly assigned between October 18, 2013, and December 22, 2014, and with a head of household age 18 to 61 at the time of random assignment. Sample sizes for specific measures may vary because of missing values. Rounding may cause slight discrepancies in calculating sums. Detail may sum to more than total for questions that allow more than one response.

SOURCE: MDRC calculations from Baseline Information Form data.

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 16

Chapter 1. Introduction

Overal , study households and heads of households are broadly similar to those in the FSS national population, with some notable differences.40 Sample members are somewhat more likely to have no children present (23.8

percent in the study sample have no children versus 17.6 percent in the national FSS population).41 Study households are less likely to report no income (4.5 percent versus the national FSS figure of 6.5 percent) but are also less likely to report income of $30,000 or more (8.5 versus 13.9 percent). Sample members also report higher levels of TANF and SNAP benefit receipt than the averages for the national FSS population (15.8 versus 10.0 percent for TANF; 69.6 versus 37.5 percent for SNAP). Study sites tend to run larger HCV and FSS

programs and spend more on rent and utilities per participant than the national population of PHAs running FSS programs, a consequence of the need to select sites that would al ow for sample recruitment within the required 1-year window.

Structure of This Report

The report is structured in eight chapters, each examining results over the 3-year post-random assignment fol owup period. Chapter 2 provides a brief program implementation update and sets the context for understanding longer term participant engagement patterns with case managers, service referrals, and use of services (chapter 3). Chapter 4 updates the analysis related to FSS group members’ accumulation of escrow credits and average account balances over the fol owup period. Chapter 5 turns to the program’s 3-year impacts on employment and earnings—assessments based on comparing average outcomes for FSS

and control group members. Using the same approach, chapter 6 examines whether the FSS program led to positive effects on household income and on a series of self-sufficiency and financial security outcomes.

Next, chapter 7 investigates variation in impacts and outcomes for subgroups defined by baseline individual characteristics and program features. The final chapter summarizes the essential takeaways from the interim findings and describes the future work on this evaluation.

40

See Verma et al. (2019). To assess whether individuals and households in the study were broadly similar to their site- and national-level counterparts, MDRC compared sample members with the broader FSS population in the study sites and with the national population of FSS

participants and housing voucher holders. The team also compared the study’s participating PHAs with all PHAs operating FSS programs.

41

Children are defined here as individuals under the age of 18.

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 17

Chapter 2. Program Delivery and Implementation Update

Chapter 2. Program Delivery and

Implementation Update

By design, Family Self-Sufficiency programs provide local public housing agencies with a broad framework within which to design and implement an FSS program tailored to their local population and service environment. As a result, and not surprisingly, local FSS policies, approaches, priorities, and staffing arrangements vary across housing authorities. A previous report, which focused on early case management practices, provided an overview of how the PHAs structured their FSS programs to help families make progress toward self-sufficiency. It described how participants in the 18 FSS study sites generally interacted with the program, how the programs were staffed, how the programs typically worked with different population groups such as employed and unemployed clients, and the various policies and approaches to implementing FSS that the 18 study sites adopted.42 Chapter 2 provides an update on ongoing program implementation practices since the previous report.

Between May and July 2018, the evaluation team conducted another round of interviews with program staff members at the FSS study sites.43 These interviews were conducted roughly 3.5 years after study enrol ment ended, built on previous rounds of interviews, and were structured to gather information about the PHAs’

current FSS policies and approaches.44 The goal of this round of interviews was to understand changes over time, how programs were working with participants who were more than midway through the 5-year program, and programs’ strategies for keeping participants engaged.45

As described in chapter 1, in recent years, HUD has funded close to 700 PHAs each year to implement the FSS program. The 18 FSS study sites represented a broad range of programs that vary by size, staffing configuration, program practices, community resources, and other factors and reflect the variation that exists in the types of programs funded by HUD. This report documents their operational practices that were in place across the FSS study sites and how program operators and staff members describe their implementation practices.

Staffing and Caseloads

HUD funds the FSS programs each year through the annual FSS Notice of Funding Availability (NOFA), and the funding is limited to the amount appropriated by Congress. Housing authorities have to apply for this funding on an annual basis. Staff positions are funded according to program size but are also limited to the number of coordinators for which they had been funded in recent years. Most programs have not had an increase in the number of FSS coordinators for which they were funded (regardless of increase in program size) since before 2012. The 2016 NOFA stated, “Eligibility for funding will be based on the number of FSS program participants in your entire FSS program during the target period, according to a formula that requires 15 families to support one part-time position, 25 families to support one ful -time position, and an additional 50 families to support each additional position beyond the first ful -time position.”46 The 42

Data for the previous report on program implementation came from interviews with FSS supervisors and case managers and observations at all 18 FSS study sites between November 2015 and January 2016, roughly a year after sample enrollment ended (Verma et al., 2019).

43

This round included in-person interviews with FSS supervisors and case managers at 7 sites and phone interviews with FSS supervisors at 10

sites. One site was unavailable for either in-person or phone interviews at the time the data collection was under way.

44

The study sample was enrolled between October 2013 and December 2014.

45

Appendix exhibit A.3 presents additional information on FSS program features and policies of the sites in the study.

46

Per communication with HUD Office of Public and Indian Housing (PIH) staff, this funding formula has now been written into the FSS

statute (HUD, 2016c).

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 18

Chapter 2. Program Delivery and Implementation Update

FSS study sites varied in the amount of funding they received: Four sites received less than $50,000, six sites received between $50,000 and $99,999, five sites received between $100,000 and $299,999, and three sites received $300,000 or more.47 FSS study sites also varied in how many FSS staff members they had and how they deployed them: Some solely had FSS-related functions, while others played multiple roles, including conducting HCV recertifications.48

For most sites in the evaluation, the FSS program team included one or two supervisors and from one to eight case managers, depending on the size of the program.49 In the period since the first round of interviews, several sites expanded their staff by hiring additional case managers.50 In contrast, a handful of sites reduced the number of case managers, consequently increasing caseload sizes. At the time of the 2018 staff interviews, staff members at two sites reported working with an average of 50 or fewer cases, eight sites had averages of 51

to 99, four had between 100 and 149, and three had more than 150, with two of those three having more than 200. Some sites saw their caseload sizes shift drastically, while others managed modest changes.51

Contact Expectations

The study sites set expectations about the frequency with which case managers contact clients beyond the mandated annual notification of clients’ escrow account balances, either at in-person meetings or by phone, e-mail, or asking participants to return a completed written progress report form. Staff members at many of the study sites reported that they maintained their original contact expectations, as documented in an earlier report. Most sites still mandated a minimum of quarterly contact, and all sites encouraged clients to get in touch with a case manager if they experienced difficulties, had questions, or wanted further assistance. Chapter 3 explores the actual contact that occurred, which differed in some cases from the expectations about the frequency of contact.

Staff members at a small number of sites stated that case managers were required to contact their participants on a monthly basis. Some were reportedly taking a more targeted approach and requiring monthly contacts with specific subgroups of participants (for instance, nonworking and nondisabled participants, so that case managers could better connect them to job-search services). One site restructured the frequency of contact to strongly encourage its staff to communicate with their participants every month. This change was not a strict requirement, as the PHA-mandated frequency was still quarterly. Case managers at this site stayed in touch with most of their clients each month, however, which enabled them to more quickly learn of and address any issues that may have adversely affected participants’ program participation or continued employment. See exhibit 2.1 for examples of site engagement practices.

In addition to the quarterly (or monthly) contact mandate, all sites required at least one annual in-person meeting with each participant. These in-person meetings were part of either the annual HCV rent recertification meetings (especially in those sites that had combined HCV/FSS staff) or the annual FSS

program review meeting. Sites continued to be flexible and creative about the ways in which they permitted 47

The funding can be used only for coordinators’ salary and fringe benefits. The amount each site receives per coordinator is based on local comparable salaries.

48

HUD intentionally leaves this decision to local discretion.

49

According to communication with PIH program staff members, nationally, most FSS programs have one coordinator, and that individual may be the only resident services staff member at the PHA. In programs with two supervisors, one is usually a mid-level case manager who manages the day-to-day operations, and the other is a more senior manager or supervisor. PHAs with a resident services department may be better positioned to be more supportive of programs like FSS, and the PHA senior staff can support the relationships needed to secure service commitments.

50

These sites, however, did not report lower caseload sizes.

51

Average caseload sizes were calculated by dividing the total number of FSS participants by the number of staff members with an FSS

caseload, regardless of whether those staff members were full time or part time or had other roles, such as HCV or homeownership program responsibilities. Averages do not include supervisors, who sometimes carry a small caseload.

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 19

Chapter 2. Program Delivery and Implementation Update

clients to maintain contact with the program: Participants could communicate in person during scheduled appointments at the office or the individual’s home or during onsite events or workshops, by submitting a completed progress report, by phone, or by e-mail. The overarching principle was for participants to stay in touch using the method that was easiest for them. Using FaceTime as a mode of contact was reported by one site. Meeting clients outside of the PHA (for example, visiting them at home, meeting them at their place of work, or meeting them at a local coffee shop) was also an option at two programs. In 2017, one site decided to use one-way texting to stay in touch with program participants. Typical messages sent via text included alerts of upcoming events, workshops, and other activities.52

52

The texts go out only to those participants who agreed to sign up for these messages.

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 20

Chapter 2. Program Delivery and Implementation Update

Exhibit 2.1. Site Engagement Practices

“The more connected people are to the program the bet er overall they do.”

One site with high expectation for engagement emphasizes the importance of a successful start to participation and of building a relationship with the participant. Coordinators work especially closely with participants immediately after they join the program. To establish rapport and the routine of being in touch, monthly contact is expected for the first 3 months. Goals are written using the “SMART” goal format meaning that they are achievable and measurable. For instance, when a participant wants to buy a home, the goal on the ITSP says “home ownership readiness” rather than “home ownership,” since the former may be more achievable within the FSS program’s five-year timeframe than the latter.

After the 3 months, coordinators check in quarterly—usually by phone or email. In addition, coordinators plan and hold a variety of workshops throughout the year. These workshops engage participants with relevant content and information and provide additional opportunities for participants and coordinators to connect in person.

“As we’re checking in, we’re going over their progress, based on what’s in their ITSP… Then, a year prior to graduation, we’re definitely doing a check-off to make sure they’re on their way to graduation.”

At another site that aims for monthly contact throughout the duration of FSS, goals on the ITSP are intentionally short-term (that is, they can be achieved in less than one year) to keep people motivated and feeling like they’re accomplishing something. Goals are broken down into incremental steps, and each step is articulated on the ITSP. Frequent contact al ows coordinators to change goals as they are achieved.

In another site, cases are prioritized so that more help and support is purposefully given to those who need it. There is an explicit expectation that coordinators will spend more time and focus on unemployed participants, who receive monthly contact, compared with quarterly contact for employed participants.

Other groups needing more support include those who are on probation (such as at risk of being terminated) and those who are nearing graduation. A report that is prepared monthly and shared with the supervisor categorizes all participants and lets coordinators know which clients need more frequent contact.

The extra effort made by coordinators at this site to reach participants is manifested not just in the frequency of the contact but also the type of contact. Coordinators are required to make field visits where they meet participants in person somewhere other than the housing authority office. Program staff believe field visits are better than “typical” communication in identifying underlying issues and barriers and providing an “extra push” for those needing additional support, such as the unemployed and those cycling through different jobs. Coordinators at this site have HCV responsibilities as well as the FSS coordinator duties. Building rapport and frequent contact remain central despite having these dual roles.

FSS = Family Self-Sufficiency. HCV = Housing Choice Voucher. ITSP = Individual Training and Services Plan. SMART = Specific, Measurable, Achievable, Realistic, and Timely.

SOURCES: 2018 field research interviews with program staff in study sites.

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 21

Chapter 2. Program Delivery and Implementation Update

Working with Unemployed or Underemployed Participants

Supervisors and case managers in nearly all sites emphasized that line staff spent more of their time working with unemployed participants than with employed participants. This is not surprising, given that clients without jobs usually had a number of barriers and issues that needed to be addressed, such as low educational levels, spotty employment histories, support service needs (such as childcare and transportation), and family and landlord issues. Chapter 5 (see exhibit 5.4) provides more details about some of the strategies staff members used to work with unemployed or underemployed participants.

Re-Engaging Participants Who Are Not Making Progress

FSS case managers described using a variety of approaches to identify and reengage participants who were no longer making progress toward their program goals. They used client-submitted progress reports, agency-level and personal-level tracking systems (including spreadsheets and case notes), face-to-face meetings, phone cal s, and e-mails for this purpose. For those participants whom staff members identified as not making progress, case managers generally intensified case management outreach, including multiple face-to-face meetings to rebuild camaraderie, trust, and confidence. Part of this approach involved reminding clients of their original program goals, stressing that participants wanted to better their and their children’s lives, and identifying and resolving barriers to participation as much as the program could. Some programs terminated participants if they did not stay engaged; chapter 3 describes the efforts they made before taking that action.

Modifying Goals and Facilitating Graduation

Goal setting is central to the FSS program. At enrol ment, participants work with FSS case managers to set goals for the duration of their FSS contract and must achieve all goals that are included on ITSP to graduate from the program. HUD requires that the ITSP include two mandatory goals: to be employed and for al household members to be free from cash assistance for 12 consecutive months. Participants usually agree to pursue additional goals that are added to the ITSP that reflect their ambitions and hopes. These initial goals can be updated, provide a starting point for mapping various pathways toward self-sufficiency such as finding and securing a job or a promotion that pays a living wage, gaining education or occupational credentials that will lead to a better job, improving credit scores, saving money, or acquiring other financial literacy skil s that make homeownership possible. Participants’ goals also influence the types of services case managers offer.

HUD provides housing authorities with considerable leeway in terms of how they approach goal setting with their clients, including in the number, types, and timeframes of goals and whether and how often goals are updated.

Preparing for graduation was equated with reviewing clients’ goals and making sure participants were progressing and “on track” in achieving their goals. Case managers said that they reviewed the ITSP and clients’ goals at least annually and in many cases more often. For instance, in a site with quarterly check-ins, the case manager said she will have a copy of the ITSP in front of her “so that we’re going over that to see how their progress is.” Because making sure clients achieve their goals is so central to case managers’ work with clients throughout the 5 years, when asked when preparation for graduation begins, more than one case manager said, “The day that they enrol . Really, the day that they enrol .”

Goals that are included on the ITSP are additional requirements to graduate. However, programs helped participants meet the graduation requirements by al owing participants to update their goals on the ITSP, including changing details of their employment goals, usually up until 6 to 12 months before their expected FSS program graduation date; the decision about how close to graduation a participant can change goals Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 22

Chapter 2. Program Delivery and Implementation Update

was left to the discretion of each PHA (for example, four sites al owed the ITSPs to be updated right up to a participant’s graduation date). Changing goals was a common occurrence, but as clients got closer to the FSS

contract end date, case managers were often even more diligent in discussions with clients to make sure the goals on the ITSP were ones that the participants were still interested in and that they were realistic within the time remaining. While these types of discussions happened throughout the duration of the FSS program, it happened toward the end “even more.” Case managers described having conversations about “lingering”

goals—that is, goals that participants said they were interested in but had not made progress toward achieving.

These types of discussions sometimes took place even earlier in the sites where goals could not be changed for a year (or more) before the contract end date.

Long-term goals such as buying a home or completing a col ege degree were ones that often were not likely to be completed within the program’s 5 years, though participants could make progress toward those goals during that time. Case managers usually told clients they could keep working on those goals even if they took them off the ITSP. For instance, one case manager explained:

“I will say to them, you know, look, we either rewrite this goal so that I can graduate you, or drop the goal.

Just focus on being employed. But that doesn’t mean you can’t still be working on this.”

Some sites did not just eliminate goals, but rather required participants to replace or modify a particular goal with a new, more realistic goal. One case manager stated:

“I don’t change goals in the middle like that. Now, if it’s…feasible to revise a goal, I’ll do that, but changing it just because they can’t do it just so they can graduate? I don’t do that.”

In the final year or two before the end of the contract, case managers described an added scrutiny about the goals as part of the annual review. As one staff member put it:

“I’d say to you that if they have about 2 years left, because per our action plan [the ITSPs] are fluid up until the last 6 months. So, if we’ve got 2 years we need to…refocus everything.… But it is something that if I know they’re relatively new it’s not as imperative to know [about the participants’ goals], but if it’s third year, fourth year, then yeah it’s important.”

Programs that imposed other participation requirements, such as attendance at workshops, tried to be flexible in how participants could meet those requirements.

At a minimum, staff members said they let clients know at the annual meeting what they needed to do to graduate. About 1 year from the contract end date, some sites sent out a letter or series of letters reminding participants about their goals and the timing or date when goals were final and could not be changed. In one site, case managers did not send a letter until 60 days before the contract end date, which was after the date that clients could change goals but gave them time to provide the documentation necessary to graduate. One site sent a form for participants to complete at the fourth year—putting the onus on participants to let the staff members know whether they had met the goals and were ready to graduate.

Both case managers and clients initiated discussions about graduation. Typically, case managers made sure the file was complete, which meant the goals and their completion were documented, and then supervisors signed off and approved the graduation. One site reported that staff members voted on whether the client was ready to graduate.

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 23

Chapter 2. Program Delivery and Implementation Update

In explaining the graduation requirements, staff members said they included information about escrow and what clients needed to do to get that money. Sites also put this information in writing. One case manager noted:

“And I do make it a point of saying to them, look, you got another year and a half left in the program. You know, you’ve got escrow money sit ing here. Remember, this is what needs to happen if you want to graduate

[with] this money.”

Some case managers also reported that they put in extra effort to keep someone with escrow on track to graduate. One case manager observed:

“I will say that [if] people that have escrow, I’m bound and determined that you weren’t gonna do something stupid to lose one quarter of this money because this money is more money than you had in the past, you know, so I’m not gonna let you…miss an appointment from me and not cal . I’m gonna harass you to the end of the earth in order to make sure that doesn’t happen.”

Employment Requirements for Graduation

Although HUD requires FSS clients to be employed in order to graduate, some program staff members reported that additional criteria not required by HUD—such as earning either “enough to support themselves” or a pay rate “deemed reasonable” by the PHA, working a certain number of hours per week, or being employed a certain number of months before graduation—were sometimes added at the discretion of the PHAs. These additional criteria varied quite a bit across the 18 FSS study sites (HUD, 2017c: 26).53 The majority of sites reported that they required participants to meet several of these more specified benchmarks to graduate. It is unclear whether FSS staff members held participants to meeting these higher benchmarks, however, or whether they used these additional criteria simply as ways to motivate participants to achieve what the staff members thought was necessary for participants to truly become self-sufficient. While staff members reported that these criteria were “requirements” for graduation, it is possible that staff members used these additional criteria primarily as motivators and never planned to enforce achievement of these higher benchmarks. Future rounds of research will explore this further.

Two sites reported that they accept any employment to satisfy graduation requirements. The rest, according to program staff members, set the bar higher.54 Some sites reported that they specified the number of weekly hours of employment required to graduate (such as 30 or 32 hours per week), and some required 6 or 12

months of continuous employment to graduate. Sites also offered extensions on a case-by-case basis, so clients could reach the required number of months. Likewise, sites that said they required employment at more than minimum wage tended to be in regions with higher living expenses. In those sites, many staff members held the view that clients needed to earn enough to sustain themselves in order to be self-sufficient. Ten sites specified a rate of pay (for example, wages that are “sustainable”—that is, wages that are high enough to maintain a reasonable standard of living without assistance), and two required people who came into the program employed to increase their income. Rather than set a minimum number of hours or wage rate, two sites reported trying to accommodate different employment situations by multiplying the minimum wage by 53

According to HUD staff members, these additional criteria are not permitted, and HUD clarified its policy and reminded sites of this in the 2017 NOFA by stating, “PHAs shall not require or define a certain number of hours or rate of pay as ‘suitable’ for all FSS participants”

(HUD, 2017b).

54

At the time of the prior research visits to the FSS study sites, four sites had reported that they accepted any employment to satisfy graduation requirements. Since then, two more sites added requirements by adding a specified number of hours of employment and by being more specific about earnings requirements (one of which reported that it would determine if wages were appropriate on a case by case basis, and the other of which required that the wage “improve” over the initial wage if someone came into the program employed) (Verma et al., 2019).

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 24

Chapter 2. Program Delivery and Implementation Update

a certain number of hours (30 in one site, 40 in the other) and used that as the minimum required earnings amount. A majority of the study sites reported making changes to the employment requirements for graduation between January 2016 and July 2018. Many of these changes increased the requirement (that is, raised the bar even higher). In a few cases, the change was small (for example, a change from simply requiring ful -time employment to noting that ful -time employment is defined as at least 30 hours per week). Most of the changes were related to hours and earnings requirements; most sites did not change requirements for how many months a participant had to be continuously working in order to graduate. One site removed the requirement that specified the number of hours a participant must be working. The staff members realized they were making it more difficult for clients to graduate. As one staff member put it:

“And we made that to where it’s not as hard. It’s like, why (are) we making this harder than we have to? For instance, a lot of it came from having a certain amount of hours that people needed to work and stuff like that… It’s like, why are we requiring them to work 32 hours, HUD doesn’t require that. You know what I mean?”

Some policy changes also added some leeway for program staff members so that they could use their own judgment in determining “reasonable” wages.

Reaching Goals Before the End of the 5-Year Contract Period

Most of the FSS study sites describe their approach to working with clients as being “client-driven”—that is, they take the clients’ lead when working with them to identify goals and to create a plan that will help them meet those goals, rather than suggesting goals or services to the clients. One of the aspects of being client-driven is that programs al ow clients to proceed at their own pace. Some participants complete all their goals and graduate early, and others take the full 5 years or even need to get an extension. When asked if they encouraged participants to use the full 5 years, some staff members reported that they encouraged clients to do that so that they could use the program’s resources for as long as possible. Others said they did not necessarily encourage participants to stay but let them know the program al owed them to stay for 5 years if they needed that support.

Clients who moved out of housing because they earned too much to qualify for the subsidy were automatically graduated from FSS when they were no longer eligible for the subsidy, even if they did not complete all their goals.55

Sites had different practices when participants completed all goals on their ITSP before the end of the contract.

Some programs automatically graduated clients at that time. In other programs, clients were encouraged to set new goals and continue in the program and keep earning escrow until they achieve their new goals and reached the contract end date. In still others, staff members said that staying in the program to build their escrow account and work toward homeownership was a satisfactory reason for some to stay in the program.

55

HUD regulations state that FSS participants are eligible to graduate from the program and receive the full amount that has accrued in the escrow account when household earnings have increased to the point where 30 percent of the monthly adjusted income equals or exceeds the published fair market rent for the size of the unit for which the FSS family qualifies based on the PHA’s occupancy standards. This rule does not apply to FSS participants in public housing, who are not the subject of this study (HUD, n.d.b.; “Outreach and Goal Setting—Module 2.3: Contract of Participation and Individual Training and Services Plan”).

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 25

Chapter 2. Program Delivery and Implementation Update

Conclusions

Overal , FSS program delivery did not change much between the first year and the midpoint of the program.

Contact expectations remained the same, for the most part. While most programs reported adding criteria to the employment requirements for graduation, it is unclear whether this was done primarily to motivate participants or whether there were actual consequences for not meeting those extra requirements. Meanwhile, case managers also reported paying special attention to participants’ progress toward their goals as they approached the final years of their FSS contracts and al owing participants to update their goals so that they were more likely to achieve them in time to graduate.

Chapter 3 explores the FSS programs’ practices related to terminating participants, particularly as those practices pertain to FSS enrol ment status midway through the 5-year program. It also examines longer-term participant engagement patterns with case managers, service referrals, and use of services. Additional field research in late 2019 and early 2020 will explore FSS program practices in the final year of FSS for many study participants.

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 26

Chapter 3. Participant Engagement

Chapter 3. Participant Engagement

The Family Self-Sufficiency (FSS) programs offer participants information about a multitude of services for enhancing self-sufficiency and largely rely on referrals to schools, col eges, and community organizations to provide these services. Nonetheless, throughout participants’ enrol ment in the program, FSS case managers remain their main point of contact.56 Chapter 3 examines participant engagement patterns with case managers, service referrals, and use of services during the first 3 years of fol owup. Analyses of FSS group members’ engagement and program participation are based on quantitative data from housing agencies’

electronic and paper notes and records, FSS and Housing Choice Voucher (HCV) program reporting forms, program management information records and tracking spreadsheets, supportive services payment records, HUD administrative data, and responses to the FSS 36-month survey.57

The analysis of program data, administrative data, and survey responses suggests the fol owing:

• Levels of program engagement declined over 3 years. By the end of Year 3, about 40 percent of FSS

group members had exited from the program. On average, FSS group members who were still enrol ed in the program interacted with an FSS case manager about once every 6 months, a decrease in frequency compared with Year 1.

• The vast majority of FSS group members engaged in some type of goal-related activity during the first 3

years of fol owup: About 80 percent participated in at least one FSS-related activity, and most of those who do not have a program activity recorded appear, per program records, to have been working for pay during at least part of the fol owup.

• Typically, FSS group members participated in short-term activities, such as job search or financial counseling sessions or workshops. Most FSS group members who engaged in an FSS-related service began participation soon after enrol ment in FSS and participated during 6 months or less of the fol owup period.

• Programs with stronger monitoring and engagement practices (smal er caseload sizes, an expectation of more frequent contacts, and a focus on establishing short-term goals) tended to have higher rates of exiting from the FSS program, compared with programs without these attributes, but they also worked more intensively with FSS group members who remained in the program.

• Based on survey responses, in Year 3 of fol owup, the FSS program led to a moderate increase of about 10 percentage points above the control group level in overall use of services. The FSS program increased receipt of financial counseling by a much larger margin: 23 percentage points.

• On average, however, the cluster of FSS programs that strongly emphasized monitoring and engagement did not experience the largest impacts on service use in Year 3, in part because participation rates were relatively high for control group respondents in these localities.

56

Some programs refer to this role as case coordination, while others view it as case management. In this and the following chapters, the terms housing authority, housing agency, PHA, FSS program, and site are used interchangeably. Similarly, the terms client and participant are used interchangeably.

57

The research team collected quantitative data from paper notes and records or electronic records from PHA management information systems in 12 of the 18 housing agencies. For details about the 36-month survey, see appendix F. As noted earlier, the survey achieved a high response rate (77 percent), and there is little evidence of bias or potential problems in generalizing the results to the larger impact sample.

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 27

Chapter 3. Participant Engagement

Program Participation in the Middle Years of the 5-Year Family

Self-Sufficiency Contract

The FSS program offers participants a broad array of services, plus access to an escrow account, that are designed to work together to promote earnings growth and financial self-sufficiency over 5 years. At program enrol ment, participants work with case managers to complete an ITSP that lists the participant’s program goals, the specific actions required to achieve these goals, and the expected completion dates of goals and actions. A typical ITSP includes an employment goal, such as working at the same job for at least 12 months; an education or training goal, such as getting an associate’s degree at a community col ege; and either a financial security goal or a homeownership-related goal. Most FSS group members committed to participating in a job search or post-employment activity. In addition, many agreed to pursue several related goals within a domain; for example, in the financial security domain, they might agree to receive financial counseling, prepare a family budget, and raise their credit score (Verma et al., 2019: 71–73).

During the first years of the 5-year FSS program, the primary chal enges for case managers involve referring participants to appropriate services that would move them into work or increase their earnings and thus help them begin accruing escrow as soon as possible. As participants enter the middle years of their FSS contract period, their relationship to the program and their prospects for attaining their program goals become more varied. As shown in chapter 3, some have made significant progress toward meeting their goals, others are pursuing activities toward attaining their goals, and still others are in a more tenuous situation. Thus, the chal enges for case managers during the middle phase of the FSS contract period vary by participant and involve (1) how often and how actively to contact participants in various situations of engagement, (2) whether to attempt to keep participants engaged in the program by referring them to additional services (including, in some housing agencies, mandatory workshops on job search skil s, workplace issues, or life skil s) or to consider employment with occasional check-ins as sufficient engagement, (3) whether to modify participants’ ITSPs by changing or dropping goals to make it easier to for them to graduate from the program, and (4) whether to actively encourage participants who have become disconnected from the program to reengage, or to drop them from the program and open a place for a new enrol ee. Using administrative data and survey responses, chapter 3 considers how different case management strategies were pursued.

Family Self-Sufficiency Enrol ment Status

By the end of the 3-year fol owup period covered in this report, a relatively large proportion of FSS group members had exited the program. Exhibit 3.1 shows estimates based on HUD and PHA administrative data and the 36-month survey. These data provide exit rates and reasons for exit. The top panel of the table, which is based on administrative data, shows that about 61 percent of the FSS group remained enrol ed in FSS at the end of the 3 years. The administrative data include limited information about exit reasons, and only about 4 percent of the FSS participants appear to have graduated from FSS.58 A higher proportion of exits appears to be related to participants leaving the voucher program or moving to another PHA, being terminated from FSS, or leaving the program voluntarily.59

58

The measure of continued enrollment in the FSS program is calculated with data from local housing agencies and federal data from HUD.

The FSS group member was considered to have left the FSS program if either data source indicated departure from the program. An alternative measure of program enrollment, based solely on local housing agency data, shows that 61 percent of FSS group members were still enrolled in Month 36.

59

Those who exited from the program include about 6 percent of FSS group members who never enrolled (see exhibit 4.2). These sample members are included in the impact analysis presented in this report. See chapter 4 for additional information on FSS group members who left the program during Years 1 to 3.

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 28

Chapter 3. Participant Engagement

The bottom panel of exhibit 3.1 shows termination outcomes based on the 36-month survey data. Nearly 80

percent of FSS group survey respondents reported that they were still enrol ed in FSS, a larger proportion than was recorded with administrative data. Yet, among these enrol ees, nearly half of the FSS group respondents described themselves as not using or participating in any FSS-related services.60 These participants were stil enrol ed in the program but, in their perspective, were not in contact with program staff members or engaged in any FSS-related services.

The staff member interviews conducted in 2018 provide some insights into the termination policies and processes for taking this action. PHAs can terminate the FSS Contract of Participation and hence remove participants from the FSS program if case managers determine that participants have not fulfil ed their responsibilities under their contract. If the contract is terminated, the PHA closes the escrow account, and the family forfeits any accumulated escrow funds. The FSS household also no longer has access to FSS program services.

The majority of sites (all but three) continue to have policies that al ow them to terminate FSS participants who do not show up for appointments and are not responsive to case managers’ attempts to reach them.

Commonly, the policy cal s for at least three staff member attempts to contact participants; these efforts are often made by phone, e-mail, and at least one letter. There may be one or two warnings, and then the last letter will give a deadline by which the person must respond, or his or her case will be terminated.

60

The calculation is as follows: 37.2 percent enrolled and not using FSS services / 79.0 percent enrolled = 47.1 percent.

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 29

Chapter 3. Participant Engagement

Exhibit 3.1. Family Self-Sufficiency and Housing Choice Voucher Program Statuses, Years 1 to 3, Family Self-Sufficiency Service Use and 36-Month Survey Respondent Samples Outcomes and Statuses (%)

FSS Group

Administrative Data

Still enrol ed in FSS program

61.3

Exited from FSS program after enrol ment

38.8

Left HCV program or moved to another HCV program

26.5

Graduated from FSS program

4.2

Left FSS program voluntarily

11.8

Asked to leave FSS program

9.0

Sample size

1,285

Survey Responses

Current FSS program status

Enrol ed and using FSS services

41.8

Enrol ed and not using FSS services

37.2

No longer enrol ed in the FSS program

21.0

Reasons for leaving the FSS program, if no longer enrol ed

Left HCV program

45.5

Left FSS program voluntarily

38.4

Moved out of area

14.1

Graduated

12.1

Asked to leave FSS program

12.1

Purchased a home

5.6

Sample size

988

FSS = Family Self-Sufficiency. HCV = Housing Choice Voucher.

NOTES: The FSS impact sample includes HCV households that were randomly assigned between October 18, 2013, and December 22, 2014, and with a head of household age 18 to 61 at the time of random assignment. The FSS 36-Month Survey respondent sample includes FSS impact sample members who responded to the FSS 36-Month Survey. Rounding may cause slight discrepancies in calculating sums. Detail on program statuses may not sum to totals because some FSS group members exited from both the HCV

program and the FSS program. Detail on reasons for leaving the FSS program may not sum to totals because FSS group survey respondents could report more than one reason.

SOURCES: MDRC calculations using housing authority administrative data, HUD Inventory Management System/Public and Indian Housing Information Center data, and responses to the FSS 36-Month Survey.

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 30

Chapter 3. Participant Engagement

The final letter gives clients a certain number of days to appeal or come back into compliance. As one case manager put it:

“If we see that the quarterly event, workshop, and service event was missed, or the participant’s quarterly progress report hasn’t been sent for the last three quarters. If that individual isn’t corresponding to our cal s or our let ers—because we’re gonna write let ers and start cal ing after the second miss.”

Another case manager noted:

“It’s not based on just the caseworker’s judgment. It’s based on the documentation that we have that shows that we have repeatedly at empted to make contact with the family and the family is unresponsive. So, before the staff closes any cases, we have to prove that we made numerous at empts and the family’s not responsive.”

Some sites do not have clear-cut policies about the termination process, but supervisors want to see that case managers attempted to reach clients over the course of many months. Even where policies seem clear, staff members appear to give participants an “extra try” before proceeding with termination. A staff member at the site that tries to use escrow as an incentive to bring clients back into compliance said:

“[W]e let them know how much money they have, because sometimes that will motivate them to come back.

‘Oh, I have all this money in my escrow?’ Then they come back, and if that one doesn’t work out, then we’l terminate them.”

Some sites also use termination letters as warnings—not intending to terminate the participant right away but expecting that the letter might prompt a response. In this way, case managers may be able to accommodate clients and do what they can to avoid terminations. As one case manager related: “If they communicate with us, we’re pretty flexible with them. We’ll try to accommodate them. There have been cases where we give them three and four appointments sometimes, just because.” When case managers are deciding how to proceed with terminating clients, however, they look to the person’s history in the program—whether the person has been responsive in the past, whether the person is employed, or if the person has earned escrow.

Some amount of reluctance to terminate is also based on thinking that some clients are making progress on their own or that they may be dealing with life crises. One case manager said:

“Because, you know, sometimes they’re out there doing things. They just don’t make you aware that they’re doing things until you say, well, I don’t have—I haven’t heard from you, please send me this, I need this and then they send me all this stuff and you say, well why didn’t you tell me.”

Some terminations are voluntary: Participants are given the option to leave the program. They are told (depending on the site policy) that they can come back and start the program anew after waiting a year or two.

As one case manager expressed it, “Maybe now is not the time to focus on FSS.… So, just take some time off and focus on you. You have to have that conversation.”

Some case managers also described how staff turnover affects participant engagement in the program. When new staff members are hired, they may not be able to establish a relationship with some clients, or it may take them time to “get up to speed.” New staff members often “clean” files and terminate participants after long periods of inactivity.

Some sites may have been reluctant to terminate because of HUD requirements to maintain minimum caseload sizes to sustain their funding for case manager positions. Some case managers felt strongly that inactive participants should be terminated from the program. As one case manager put it: Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 31

Chapter 3. Participant Engagement

“If I had the power, I’d close people because they just are doing nothing. Absolutely nothing. They feel that we’re just babysit ing them. Just babysit ing. Not doing something. And those are the ones that give the biggest fight, and they’ve done nothing.”

Contact with Case Manager

To analyze contact patterns between FSS participants and case managers, the evaluation draws on information recorded by the housing agencies. These data are used to create measures of the timing and mode of communications between FSS staff members and participants (in-person meeting, phone cal , exchange of e-mails or other electronic documents, or mailing of paper forms or documents). For each FSS participant, the data focus on Year 3 (about Months 25 to 36) of fol owup.

For this analysis, a “communication” is counted if it involved an exchange between the FSS participant and the FSS case manager, either simultaneously, as in a face-to-face meeting or phone cal , or sequentially, as during a documented exchange of electronic communications or when the FSS group member mailed or dropped off at the housing agency office a letter or a completed progress report.61 To analyze the incidence of communications, the month of fol owup was chosen as the unit of analysis. Indicators of the number of months with at least one communication were created.62 Separate averages were calculated for each mode of communication. An additional analysis uses responses from the FSS 36-month survey. Respondents were asked when they had last communicated with an FSS case manager since random assignment, a somewhat longer fol owup period than was covered by the analysis of housing agency records.

61

Not counted as communications are “attempted contacts,” such as phone messages left by the FSS coordinator, e-mails or paper forms sent by the FSS coordinator when there was no subsequent reply by the FSS group member, letters and notices mailed by the FSS coordinator, and recorded communications between the FSS coordinator and other members of the FSS group member’s household. For several housing authorities, this analysis also excludes the routine monthly mailing or e-mailing to all or to large numbers of FSS participants of fliers from service providers, newsletters, notices of upcoming job fairs or housing agency events, and job listings.

62

Thus, for example, months with five phone conversations were treated similarly in the analysis to months with one phone conversation. This analysis strategy acknowledges the difficulty of creating an equivalent metric for measuring the incidence of communications when two parties in communication can routinely exchange multiple e-mails or text messages in rapid succession.

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 32

Chapter 3. Participant Engagement

Exhibit 3.2. Contacts Between Family Self-Sufficiency Participants and Case Managers, Year 3, Family Self-Sufficiency Service-Use Sample

Contacts

FSS Group

Enrolled in Month 36

Enrol ed in FSS in Month 36 (%)

65.3

100.0

Had one or more contacts (%)

50.3

68.0

In-person

35.3

49.1

By phone

23.5

31.3

By e-mail, text, social media, or fax

8.7

11.8

By mail or delivery of form or letter

22.0

30.3

Average number of months with one or more contacts

1.5

2.0

Number of months with one or more contacts (%)

0

49.7

32.0

1–3

36.7

48.7

4–6

9.7

13.7

7–9

2.9

4.1

10–12

1.0

1.5

Sample size

714

466

FSS = Family Self-Sufficiency.

NOTES: The FSS service-use sample includes all or a randomly selected subsample of Housing Choice Voucher heads-of-household who were randomly assigned to the FSS group between October 18, 2013, and December 22, 2014 in 12 public housing agencies, and were age 18 to 61 at their time of random assignment. “Contacts” are defined as interactions between FSS group members and FSS case managers in which the FSS group member was actively involved. Contacts include conversations that took place in person or by phone; communications by email, text, social media, or fax that were initiated by the FSS group member; and completed forms or letters that were mailed or delivered in person by the FSS group member. Rounding may cause slight discrepancies in calculating sums. Detail may sum to more than total because FSS group members may have had more than one type of contact.

SOURCE: MDRC calculations using housing authority administrative data.

Exhibit 3.2 summarizes the findings on the incidence and types of communications between FSS group members and FSS case managers, based on housing agency records (paper documents or electronic data) for 12 of the 18 PHAs that were the focus of this analysis. Exhibit 3.2 presents information for all FSS group participants in the 12 sites targeted for this analysis and for those still enrol ed in FSS in Month 36. As exhibit 3.2 shows, one-half of the FSS group (and nearly 70 percent of FSS group members still enrol ed in Year 3) had at least one contact with an FSS case manager, most often in face-to-face meetings at the housing agency office. Nearly one-fourth of FSS group members had at least one phone conversation with their FSS case manager, and about 9 percent communicated with their FSS case manager by email, text, social media, or fax—typically by returning a progress report.

Across all housing authorities, FSS group members and FSS case managers averaged less than two exchanges, equivalent to communications occurring every 8 or 9 months—or once every 6 months among FSS group members still enrol ed in the program. The incidence of communications with FSS case managers decreased over time. As noted in a previous report on the FSS evaluation, during the first year of the program, FSS

group members communicated with their case manager every 4 or 5 months (Verma et al., 2019). The average Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 33

Chapter 3. Participant Engagement

for Year 3 also fal s below the established standard requirement of quarterly or monthly communications that most housing agencies had set at the start of the evaluation.63

Participation in Services and Referrals

HUD provides funding for case management and coordination, but housing agencies rely on local community organizations to provide services to FSS participants. The study sites offer participants information about or connections to a broad range of services through both formal and informal partnerships with service providers.

This section examines FSS participants’ patterns of engagement in FSS program activities during the first 36 months of fol owup.64 Program data from housing agency records offices, supplemented by HUD

administrative data and by participant responses to the FSS 36-month survey, were used to create measures of FSS participation: use of FSS-related services and recorded employment (per HUD guidelines, an important measure of FSS program success).65

As discussed below, the vast majority of FSS group members demonstrated at least a minimal level of engagement in the program during the first 3 years of fol owup: they participated in at least one goal-related activity, started a job, or maintained employment that they began before enrol ing in the FSS program. As measured, levels of engagement appear to have diminished over time, although much more so for the use of FSS services than for employment recorded by the program.

Service Use and Recorded Employment

Exhibit 3.3 and appendix exhibit B.1 summarize the patterns of FSS group members’ service use and employment documented in PHA records over the first 3 years of fol owup and, separately, in Year 3.66 As exhibit 3.3 shows, according to housing agency data, about 80 percent of FSS group members used at least one FSS-related service during Years 1 to 3, and three-fourths of FSS group members worked for pay. Nearly two-thirds of the FSS group combined service use with employment, whereas only about 8 percent of FSS group members enrol ed in the program but never used an FSS service and had no recorded employment through Month 36.

Use of FSS-related services fol owed no set pattern during Years 1 to 3. At program enrol ment, when fil ing out their ITSP, nearly all FSS group members committed to pursue at least one financial security goal. Not surprisingly, the largest proportion of FSS group members participated in financial security–related activities (45 percent), most often by attending at least one financial management workshop or by receiving individual financial counseling. (See appendix exhibit B.1 for detailed findings on service use.) A little more than one-third of FSS group members participated in a job search, self-employment preparation, or post-employment services activity—with job search activities (individual or group) as the most frequently used services. A slightly smal er proportion of FSS group members attended an education or training program, mostly post-secondary education or vocational training. Although use of social services was rarely included as a goal in FSS group members’ ITSP, a sizable proportion (35 percent) of FSS group members received social services from the 63

See chapter 4, page 82, for analysis of contacts and engagement of FSS group members by program eligibility status and accrual of escrow credits. As noted in chapter 4, on average, FSS group members with the largest balances in their escrow account also had highest incidence of contacts with their FSS case managers during Year 3.

64

For this analysis, the terms used services and participated in activities are used interchangeably.

65

Data collected from housing authorities included case notes, service referral forms, workshop attendance sheets, communications from service providers, participant progress reports, management information system records, service-use tracking spreadsheets, communications between FSS group members and FSS case managers, and HCV program status forms.

66

The rest of this analysis focuses on the experiences of the FSS sample members in the 12 sites targeted for this analysis (or the “service-use sample”), all of whom enrolled in the FSS program following random assignment.

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 34

Chapter 3. Participant Engagement

FSS program, usually by attending at least one workshop in life skil s, parenting, or building self-esteem. In contrast, whereas most FSS group members had agreed to pursue at least one homeownership-related goal, only about 16 percent of FSS group members had taken steps toward doing so by the end of Year 3. Most of these FSS group members attended at least one workshop that covered the financial requirements for purchasing a home or received individual counseling. More closely reflecting the pattern of goal setting in ITSPs, about one in five FSS group members received some type of support service, most often in the form of transportation assistance or financial aid, and few received health coverage assistance or health-related services from the FSS program.

Exhibit 3.3. Use of Family Self-Sufficiency-Related Services and Recorded Employment During Years 1 to 3, Family Self-Sufficiency Service-Use Sample

Outcome (%)

Years 1 to 3

Year 3

Used FSS services (%)

80.4

36.3

Financial security

45.2

10.6

Job search or post-employment

37.4

9.8

Post-employment only

2.4

0.3

Social services

35.3

12.5

Education or training

31.4

17.9

Supportive services

21.7

5.0

Homeownership preparation

15.8

3.8

Health coverage and services

7.0

2.1

Had program-recorded employment (%)

75.4

54.8

FSS service use and employment status (%)

Service-use and program-recorded employment

63.3

28.3

Service-use only

17.1

8.0

Program-recorded employment only

12.0

26.5

Neither service use nor program-recorded employment

7.6

37.3

Sample size (total = 714)

FSS = Family Self-Sufficiency.

NOTES: The FSS service-use sample includes all or a randomly selected subsample of Housing Choice Voucher heads of household who were randomly assigned to the FSS group between October 18, 2013, and December 22, 2014 in 12 public housing agencies, and were age 18 to 61 at the time of random assignment. Rounding may cause slight discrepancies in calculating sums. Detail may sum to more than 100 percent because FSS group members could use more than one service.

SOURCE: MDRC calculations using housing authority administrative data.

The “Year 3” column in exhibit 3.3 demonstrates the decrease in service use over time. According to agency records, a little over one-third of FSS group members used at least one FSS-related service during Year 3, and fewer than one in five FSS group members participated in any specific activity. Even financial counseling and workshops, which can be organized as a series of monthly meetings, engaged only about 10 percent of FSS

group members. Only longer-term education and training activities (often in pursuit of a license or credential) involved nearly one-fifth of FSS group members.

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 35

Chapter 3. Participant Engagement

The relatively high rate of employment among FSS group members helps explain why FSS group members participated intermittently or stopped after completing one or two short-term activities. According to housing agency data, about 45 percent of FSS group members entered the program already employed (not shown in exhibit 3.3). In any month of fol owup, more than twice as many FSS group members were employed as were participating in FSS activities, and in Year 3, almost a similar proportion of FSS group members worked for pay but did not use FSS-related services, compared with those who combined service use and employment.

The high incidence of employment among FSS group members is a positive outcome both in general and because employment is a required graduation goal. Nonetheless, combining work and service receipt is often difficult for low-income heads of households, particularly those with young children. As the months of fol owup proceed, FSS group members who maintain employment but have ceased participating in FSS activities or, as occurred for 12 percent of FSS group members, have not yet participated in FSS activities may find it increasingly difficult to reconnect with the program and complete all goals listed in their ITSP.

Most FSS group members began to participate in FSS-related services within the first 6 months of random assignment (see exhibit 3.4). Many of these FSS group members started their involvement with the FSS

program by attending short-term workshops on job search skil s, financial security, homeownership preparation, or life skil s soon after enrol ing in the program. In addition, about 7 percent of FSS group members continued attending a post-secondary education or vocational training program that they had started before enrol ing in the FSS program (see appendix exhibit B.1). Relatively few FSS group members who had not begun participating in FSS-related services and activities by the end of Month 6 began participation during the remaining 2.5 years of fol owup.67

Most FSS group members participated in FSS-related services on a short-term basis. The majority of FSS

group members participated in FSS activities 6 months or less out of the 3 years of fol owup, and about 30

percent of FSS group members participated during 7 months or more. (See exhibit 3.4.) 67

As exhibit 3.4 shows, 22 percent of FSS group members first participated in an FSS activity after Month 6 of followup, and 20 percent never participated during the followup.

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 36

Chapter 3. Participant Engagement

Exhibit 3.4. Indicators of Timing and Duration of Service Use and Program-Recorded Employment During Years 1 to 3, Family Self-Sufficiency Service-Use Sample

Outcome

FSS Group

Service Use

First month with service use (%)

1–3

39.4

4–6

18.8

7–12

9.9

13–36

12.0

No service use

19.9

Average number of months with service use

6.7

Months with service use (%)

0

19.9

1–3

37.3

4–6

13.7

7–12

11.2

13–36

17.9

Employment

First month with program-recorded employment (%)

1–3

51.3

4–6

4.1

7–12

8.4

13–36

11.6

No recorded employment

24.6

Average number of months with program-recorded employment

17.3

Months with program recorded employment (%)

0

24.6

1–3

3.2

4–6

3.5

7–12

10.8

13–36

57.8

Sample size

714

FSS = Family Self-Sufficiency.

NOTES: The FSS service-use sample includes all or a randomly selected subsample of Housing Choice Voucher heads of household who were randomly assigned to the FSS group between October 18, 2013, and December 22, 2014 in 12 public housing agencies, and were age 18 to 61 at the time of random assignment. Rounding may cause slight discrepancies in calculating sums.

SOURCE: MDRC calculations using housing authority administrative data.

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 37

Chapter 3. Participant Engagement

Variation in Program Engagement

The FSS program coordinates access and referrals to a wide range of services to help participants find jobs, advance in careers, and improve their financial security. Use of these services may vary among study participants who enter the program with different levels of employment, education, or financial circumstances.

Variation in how housing authorities operate the FSS program may also affect levels of service use. This section explores these relationships.

Program Engagement by Baseline Employment and Education

Exhibits 3.5 and 3.6, respectively, show comparisons of indicators of FSS program engagement by self-reported employment and educational attainment at study enrol ment.68 Among subgroups defined by these baseline characteristics, most FSS group members maintained at least a minimal connection to the program through the end of the fol owup period. For example, for the employment-based subgroups, between 61 and 68

percent of FSS group members continued enrol ment in the program through Month 36 (see exhibit 3.5).69

Across all subgroups, a large majority engaged in FSS-related services during Years 1 to 3—with rates of service use varying from 72 percent (for participants with ful -time employment at random assignment) to 84

percent (for participants without employment). Subgroups varied more substantially in their incidence of use of specific FSS-related services. For example, as shown in exhibit 3.5, considerably larger proportions of FSS

group members who reported no employment or part-time employment at their time of random assignment (44 and 38 percent, respectively) engaged in job search or post-employment activities during Years 1 to 3, compared with FSS group members who were working full time at random assignment (26 percent). These more disconnected-from-work subgroups also averaged higher rates of using social and supportive services compared with FSS group members who were employed full time. Among educational attainment subgroups (exhibit 3.6), FSS group members with no degree or credential at random assignment recorded much lower attendance at financial security and homeownership preparation counseling and workshops (29 and 8 percent, respectively) compared with other FSS group members with a high school diploma, GED, or post-secondary degree.

Frequency of contact was relatively low in Year 3 for all subgroups but varied somewhat among educational attainment subgroups. For example, across these subgroups, the incidence of having at least one contact with an FSS case manager in Year 3 varied between 41 percent (for participants with a high school degree or GED

at random assignment) and 56 percent (for participants with some col ege attendance).

Subgroups also differed in the extent to which FSS group members combined services and employment during Year 3 of fol owup (see the bottom panels of exhibits 3.5 and 3.6). For instance, FSS group members who reported being unemployed at random assignment had the highest incidence (46 percent) of no engagement (neither service use nor program-recorded employment) during Year 3 compared with other employment status subgroups. In addition, among educational attainment subgroups (exhibit 3.6), there is a relatively strong association between educational attainment level and level of engagement in the program. The subgroup with a 2-year col ege degree or higher had the highest rate of combining employment and service use in Year 3 and the lowest rate of no service use nor program-recorded employment.

68

As noted in chapter 1, this study prioritizes these individual-level subgroups to assess variation in outcomes.

69

The enrollment measure is based on housing agency data, which, as noted above, yields slightly higher rates of enrollment than measures based on data from HUD.

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 38

Chapter 3. Participant Engagement

Exhibit 3.5. Selected Indicators of Contacts and Service Use During Years 1 to 3, by Self-Reported Employment Status at Random Assignment

Not

Employed

Employed

Outcome

Employed

Part Time

Full Time

Years 1 to 3

Services and Employment

Used FSS-related service

83.7

83.2

72.1***

Job search or post-employment services

43.9

38.2

25.9***

Financial counseling

42.6

52.4

42.6*

Education or training

32.5

32.5

31.0

Homeownership preparation

14.4

16.2

17.8

Health coverage or health assistance

6.1

8.4

7.1

Social services

39.3

34.6

29.4*

Supportive services

24.5

22.5

16.2*

Had program-recorded employment

55.5

92.1

91.9***

Year 3

Contacts with Case Managers

Had 1 contact or more (%)

49.4

55.5

46.7

Had contact during 3 months or more (%)

18.4

24.1

19.8

Average number of months with contacts

1.3

1.8

1.4**

Service Use and Employment (%)

***

Service use and program-recorded employment

24.5

31.9

31.0

Service use only

12.9

4.2

3.6

Program-recorded employment only

16.9

39.3

29.9

Neither service use nor program-recorded employment

45.7

24.6

35.5

Still enrol ed in FSS program in Month 36

66.3

67.5

61.4

Sample size (total = 714)

326

191

197

FSS = Family Self-Sufficiency.

NOTES: The FSS service-use sample includes all or a randomly selected subsample of Housing Choice Voucher heads of household who were randomly assigned to the FSS group between October 18, 2013, and December 22, 2014 in 12 housing agencies, and were age 18 to 61 at the time of random assignment. Rounding may cause slight discrepancies in calculating sums and differences. A chi-square test was performed on subgroup differences in frequency distributions and an F-test was performed on subgroup differences in means.

Statistical significance levels are indicated as: * = 10 percent; ** = 5 percent; *** = 1 percent.

SOURCES: MDRC calculations using baseline data and housing authority administrative data.

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 39

Chapter 3. Participant Engagement

Exhibit 3.6. Selected Indicators of Contacts and Service Use During Years 1 to 3, by Self-Reported Level of Educational Attainment at the Time of Random Assignment High

2-Year

No Degree

school

college

or

degree or

Some

degree or

Outcome

credential

GED

college

higher

Years 1 to 3

FSS-Related Service Use and Employment (%)

Used FSS-related service

73.1

81.9

82.1

83.0

Job search or post-employment services

32.4

34.5

38.5

44.4

Financial counseling

29.0

52.0

47.9

48.9***

Education or training

27.6

28.2

33.5

39.3

Homeownership preparation

8.3

16.4

16.3

22.2**

Health coverage or health assistance

6.9

6.8

7.4

6.7

Social services

32.4

31.1

35.8

43.0

Supportive services

16.6

18.1

28.0

20.0**

Had program-recorded employment

71.0

70.6

80.2

77.0*

Year 3

Contacts with Case Managers

Had 1 contact or more (%)

49.0

41.2

55.6

53.3**

Had contact during 3 months or more (%)

12.4

18.6

24.5

23.0**

Average number of months with contacts

1.1

1.2

1.7

1.9***

Service Use and Employment (%)

***

Service use and program-recorded employment

18.6

21.5

33.5

37.8

Service use only

9.7

8.5

7.4

6.7

Program-recorded employment only

25.5

28.8

26.5

24.4

Neither service use nor program-recorded

employment

46.2

41.2

32.7

31.1

Still enrol ed in FSS program in Month 36

62.1

65.0

66.1

67.4

Sample size (total = 714)

145

177

257

135

FSS = Family Self-Sufficiency. GED = General Educational Development.

NOTES: The FSS service-use sample includes all or a randomly selected subsample of Housing Choice Voucher heads of household who were randomly assigned to the FSS group between October 18, 2013, and December 22, 2014 in 12 housing agencies, and were age 18 to 61 at the time of random assignment. Rounding may cause slight discrepancies in calculating sums and differences. A chi-square test was performed on subgroup differences in frequency distributions and an F-test was performed on subgroup differences in means.

Statistical significance levels are indicated as: * = 10 percent; ** = 5 percent; *** = 1 percent.

SOURCES: MDRC calculations using baseline data and housing authority administrative data.

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 40

Chapter 3. Participant Engagement

Program Engagement Across Study Sites

Levels of engagement with the program vary considerably by site. This analysis, shown in exhibit 3.7, uses one program enrol ment measure and four participation outcome measures: (1) enrol ment in FSS during the last month of Year 3; (2) having participant–case manager contacts during at least 1 month of fol owup; (3) having participant–case manager contacts during at least 3 months of fol owup; (4) using FSS-related services during at least 1 month of fol owup; and (5) using FSS-related services during at least 3 months of fol owup.

The third and fifth of these measures represent relatively high levels of participation and engagement by FSS

group members. These measures are created for each housing agency, and all the measures focus on Year 3 of fol owup.

The second row of exhibit 3.7, the median value for all housing authorities, shows that about one-half of the FSS group members from any housing agency had at least one contact with an FSS case manager during Year 3 of fol owup. The extremes were housing authorities in which two-thirds of FSS group members had at least one contact and housing authorities in which only about one-third of FSS group members had at least one contact. Similarly, among the housing agencies with the highest incidence of service use, about one-half of FSS group members used at least one FSS-related service during Year 3 of fol owup, compared with one in six FSS group members in housing authorities with the lowest incidence. Averages for housing authorities also fel along a wide continuum for measures of more extensive engagement with the FSS program, having contacts with an FSS case manager within 3 months or more and using FSS-related services during 3 months or more.

Implementation, using data from interviews with FSS administrators and case managers and data col ected from FSS group members’ ITSP forms. As the middle panel of exhibit 3.7 shows, the implementation features tested were (1) maintaining small FSS caseloads, (2) expecting relatively frequent contacts, and (3) setting Year 1 program goals for a large majority of FSS group members. The research team examined similar associations for indicators of sites’ tendency to emphasize types of service use when preparing ITSPs, as measured by the percentage of FSS group members with an FSS goal in the domain of (4) job search or post-employment services, (5) education or training, or (6) financial security.

It was also hypothesized that housing agencies with implementation features that suggest a stronger commitment to close monitoring of and frequent engagement with FSS participants would demonstrate a higher incidence of contacts and service use compared with housing authorities without these features. With some exceptions, the findings from this analysis confirm this hypothesis. Tests of association between indicators of service type and participation outcomes were more exploratory. Housing authorities could plausibly attain relatively high rates of contacts or service use by encouraging FSS group members to participate in a series of short-term employment-related activities, such as job search workshops, or in longer term education or training activities with frequent check-ins.

One simple way to examine whether FSS program implementation features are likely to affect participants’

contacts and service-use outcomes involves calculation of Pearson’s product moment correlation coefficients.

Site averages for each of the six implementation features listed previously were correlated with site averages for each of five outcomes, resulting in a matrix of 30 correlation coefficients. The middle panel of exhibit 3.7

shows the results of these tests. Correlation coefficients with a negative coefficient suggest a negative association between an indicator of program implementation and an indicator of contacts or service use.

As exhibit 3.7 shows, three implementation features that likely reflect each site’s commitment to monitoring and engagement—maintaining small FSS caseloads, expecting relatively frequent contacts, and setting Year 1 program goals for a large majority of FSS group members—have at least a moderately strong and positive association with indicators related to contacts and service use, as evidenced by correlation coefficients that Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 41

Chapter 3. Participant Engagement

range between 0.65 and 0.90 for having had contacts in 3 months or more and between 0.56 and 0.63 for using FSS-related services.70 At the same time, housing authorities that strongly emphasized monitoring and engagement also tended to have higher-than-average rates of exiting from FSS. One possible way to summarize these findings is that these programs were not highly successful at keeping participants engaged in FSS (or more consistently terminated participants who stopped engaging in the program) but worked more intensively with participants who remained in the program.

The three implementation features related to monitoring and engagement also have moderately strong and positive associations with each other. These results suggest that these separate indicators are attributes of a general implementation feature, which will be referred to as a housing agency’s level of “emphasis on monitoring and engagement.” To use this concept in additional analyses, the three indicators were combined into a composite measure and average values were calculated for each housing agency.71 The bottom row of exhibit 3.7 shows the results of tests of association between housing authorities’ emphasis on monitoring and engagement and indicators of incidence of FSS group members’ contacts and service use. The correlation coefficients for the measure of continued enrol ment in FSS (-0.67), having contacts during 3 months or more (0.83), and using FSS services (0.64) are consistent with the findings for each component measure.

Exhibit 3.7 also shows that in some instances, a housing agency’s emphasis on a type of goal or activity has a moderately strong and positive association with program engagement outcomes. Housing authorities that emphasized job search or post-employment goals tended to have higher-than-average incidences of FSS

services use during Year 3 (correlation coefficient = 0.43). Similarly, programs that emphasized education and training goals had a higher-than-average incidence of contacts between participants and FSS case managers during Year 3 (correlation coefficient = 0.48).

70

Correlation coefficients of between 0.700 and 1.000 are considered to indicate a relatively strong and positive association between an indicator of program implementation and a participation outcome, whereas values of between 0.400 and 0.699 suggest a moderately strong and positive association, and values of between 0.000 and 0.399 suggest little or no association.

71

See appendix exhibit B.3 for additional details on the measure creation.

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 42

Chapter 3. Participant Engagement

Exhibit 3.7. Indicators of Contacts and Family Self-Sufficiency Service Use in Year 3, by Housing Authority, Family Self-Sufficiency Service-Use Sample

Enrolled

Had

Used FSS

in FSS

Had One Contact in

Used

Services in

Program in Contact 3 Months

FSS

3 Months

Outcome

Month 36

or More

or More

Services

or More

Average value for the 3 highest-

ranking housing authorities (%)

81.0

66.9

41.9

50.4

32.5

Median value for all housing

authorities (%)

69.0

47.5

20.3

38.1

17.4

Average value for the 3 lowest-

ranking housing authorities (%)

40.9

36.0

3.1

16.7

7.3

Correlation Coefficients (Housing Authority Averages)

Average FSS caseload sizea

-0.63

0.34

0.71

0.63

0.41

Expected number of contacts per

year

-0.52

0.35

0.90

0.56

0.56

Proportion of FSS group with a

Year 1 goal

-0.68

0.16

0.65

0.57

0.35

Proportion of FSS group with a job

search or post-employment goal

-0.09

0.23

0.00

0.43

0.37

Proportion of FSS group with an

education or training goal

0.12

0.48

0.06

0.25

0.36

Proportion of FSS group with a

financial security goal

0.16

0.25

0.12

0.18

0.33

Emphasis on monitoring and

engagement (composite score)

-0.67

0.31

0.83

0.64

0.49

b

Number of housing authorities (total = 12)

FSS = Family Self-Sufficiency.

aAverage caseload sizes were multiplied by -1 to test whether small caseload sizes are positively correlated with greater incidence of contacts and service use.

bThe FSS monitoring and engagement composite score incorporates three components: average caseload size, expected number of contacts per year, and the proportion of FSS group members with a Year 1 goal. A Z-score for each component was calculated using the site value and the mean of all 18 sites. The Z-scores were summed to create the composite value.

NOTES: The FSS service-use sample includes all or a randomly selected subsample of Housing Choice Voucher heads-of-household who were randomly assigned to the FSS group between October 18, 2013, and December 22, 2014, in 12 housing agencies and were age 18 to 61 at the time of random assignment.

SOURCES: MDRC calculations from housing authority service-use data and information provided by FSS administrators and case managers.

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 43

Chapter 3. Participant Engagement

Exhibit 3.8 looks more closely at the association between housing agencies’ program implementation approach (or emphasis on monitoring and engagement) and FSS group members’ service-use outcomes. As exhibit 3.8

shows, the percentage of FSS group members who used at least one FSS-related service in Years 1 to 3 was highest by a wide margin in housing agencies that most strongly emphasize monitoring and engagement.

Similarly, FSS group members in these housing agencies were more likely to have a record of service use and employment during Year 3. These PHAs also record the highest rate of having three contacts or more during Year 3.

Exhibit 3.9 displays variation in service use among housing agencies that differed in their emphasis on encouraging participation in (1) job search and post-employment activities, (2) education and training activities, and (3) financial counseling and workshops.72 As exhibit 3.9 shows, a PHA’s level of emphasis on specific types of services had scattered effects on FSS group members’ overall levels of participation. For example, about 70

percent of FSS group members in PHAs with a low emphasis on job search and post-employment activities used any FSS service in Years 1 to 3, about 13 percentage points below the levels for PHAs with a medium or high emphasis. The percentage of FSS group members who used any services varied to a lesser extent (by 8

percentage points) among PHAs that differed in their emphasis on participation in financial counseling and workshops. Finally, PHAs’ level of emphasis on participation in education and training activities had no effect on group members’ participation rates in any FSS services.

In contrast, rates of participation for specific types of FSS activities were strongly affected by PHAs’ level of emphasis. For instance, about 47 percent (with rounding) of FSS group members in PHAs with the strongest emphasis on education and training attended an education or training course during Years 1 to 3—a rate of participation about 20 percentage points higher than for the PHAs with a low or medium emphasis. Similarly, the rate of participation in job search activities (41 percent) was highest in PHAs with the strongest emphasis on job search and post-employment services, and the level of participation in financial counseling or workshops was highest in PHAs that most strongly emphasized financial services (51 percent).

72

See appendix E for an explanation of how PHAs were categorized as having a low, medium, or high emphasis on participation in each of these three types of activities.

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 44

Chapter 3. Participant Engagement

Exhibit 3.8. Use of Family Self-Sufficiency Services and Recorded Employment During Years 1 to 3 Among Housing Authorities with Similar Levels of Emphasis on Monitoring and Engagement, Family Self-Sufficiency Service-Use Sample

Monitoring and Engagement

Outcome (%)

Low

Medium

High

Years 1 to 3

Services and Employment

Used FSS-related service

75.2

73.9

93.9***

Job search or post-employment services

20.4

37.8

58.4***

Financial counseling

47.8

33.0

55.1***

Education or training

27.0

27.8

43.0***

Homeownership preparation

13.3

13.5

21.5**

Health coverage or health assistance

1.9

5.2

15.4***

Social services

23.0

20.4

66.8***

Supportive services

7.0

25.7

36.0***

Had program-recorded employment

71.1

74.3

81.8**

Year 3

Contacts with Case Managers

Had 1 contact or more (%)

50.0

48.3

52.8

Had contact during 3 months or more (%)

10.7

17.4

35.5***

Average number of months with contacts

1.0

1.2

2.3***

Service Use and Employment (%)

***

Service use and program-recorded employment

21.9

25.7

39.3

Service use only

8.9

5.7

9.3

Program-recorded employment only

34.8

27.4

15.0

Neither service use nor program-recorded employment

34.4

41.3

36.4

Still enrol ed in FSS program in Month 36

74.8

60.4

58.4***

Sample size (total = 714)

270

230

214

FSS = Family Self-Sufficiency.

NOTES: The FSS service-use sample includes all or a randomly selected subsample of Housing Choice Voucher heads of household who were randomly assigned to the FSS group between October 18, 2013, and December 22, 2014 in 12 housing agencies, and were age 18 to 61 at the time of random assignment. Rounding may cause slight discrepancies in calculating sums and differences. A chi-square test was performed on subgroup differences in frequency distributions and an F-test was performed on subgroup differences in means.

Statistical significance levels are indicated as: * = 10 percent; ** = 5 percent; *** = 1 percent. The FSS monitoring and engagement composite score incorporates three components: average caseload size, expected number of contacts per year, and the proportion of FSS

group members with a Year 1 goal. A Z-score for each component was calculated using the site value and the mean of all 18 sites. The Z-scores were summed to create the composite.

SOURCES: MDRC calculations using baseline data and housing authority administrative data.

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 45

Chapter 3. Participant Engagement

Exhibit 3.9. Use of Services During Years 1 to 3, by Program Emphasis on Service Type 100

90

83.2

82.9***

83.9

81.1

80.0

79.1

79.8

80

76.1

70.3

70

60

51.2***

50

47.5***

47.5

41.0**

40

31.0

32.8

31.9

30

27.6

26.8

20

10

0

Low

Medium

High

Low

Medium

High

Low

Medium

High

Emphasis on

Emphasis on

Emphasis on

Job Search and

Education and

Financial

Post-Employment Services

Training

Services

Used FSS-related services

Job search

Education and training

Financial

FSS = Family Self-Sufficiency.

NOTES: The FSS service-use sample includes all or a randomly selected subsample of Housing Choice Voucher heads of household who were randomly assigned to the FSS group between October 18, 2013, and December 22, 2014, in 12 housing agencies, and were age 18 to 61 at the time of random assignment. A chi-square test was performed on subgroup differences in frequency distributions and an F-test was performed on subgroup differences in means. Statistical significance levels are indicated as: * = 10 percent; ** = 5

percent; *** = 1 percent. The FSS monitoring and engagement composite score incorporates three components: average caseload size, expected number of contacts per year, and the proportion of FSS group members with a Year 1 goal. A Z-score for each component was calculated using the site value and the mean of all 18 sites. The Z-scores were summed to create the composite.

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 46

Chapter 3. Participant Engagement

Participants’ Assessment of the Family Self-Sufficiency Program

FSS group respondents to the 36-month survey, both current and former enrol ees, answered a series of questions concerning their involvement with the FSS program. Exhibit 3.10 summarizes their responses.

As exhibit 3.10 shows, current enrol ees reported a relatively positive assessment of their engagement in the program. About three-fourths of current enrol ees stated that they were satisfied with the FSS program, and a similar proportion reported that they had made at least some progress toward meeting their program goals in the past year. Most current enrol ees related that they expected to end their involvement in FSS within 3 years, although relatively few of these respondents accurately listed all the requirements for graduation. Current enrol ees listed a wide range of goals as their primary objective, with homeownership as the most frequently named goal (21 percent). Current enrol ees also cited purchasing a home as the most likely use of their escrow dol ars (30 percent).

Exhibit 3.10. Indicators of Engagement with the Family Self-Sufficiency Program, by Current Enrol ment Status, Family Self-Sufficiency Group Members in the 36-Month Survey Respondent Sample

Currently No Longer

Outcome (%)

Enrolled

Enrolled

Total

Contacts During the Past 12 Months

Had a contact with an FSS case manager

82.8

84.4

83.1

Assessed sufficiency of contacts

***

No contacts

17.6

15.6

17.2

Too few

24.1

34.5

26.3

Right number

56.4

45.5

54.2

Too many

1.8

4.5

2.4

Expected Number of Years Left in FSS Program

Less than one year

10.9

–

–

1–1.99 years

27.8

–

–

2–2.99 years

41.1

–

–

3 years or more

20.1

–

–

Knowledge of FSS Graduation Requirements

Did not name any requirements

11.6

29.1

15.3***

Correctly named as requirement

Head of household employment

73.6

51.9

69.0***

No receipt of welfare cash benefits

44.9

18.9

39.4***

Attaining all program goals

74.6

57.8

71.1***

Incorrectly named as requirement

No receipt of food stamp/SNAP benefits

34.7

16.0

30.8***

Leaving Housing Choice Voucher program

29.8

22.3

28.2**

(continued)

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 47

Chapter 3. Participant Engagement

Exhibit 3.10. Indicators of Engagement with the Family Self-Sufficiency Program, by Current Enrol ment Status, Family Self-Sufficiency Group Members in the 36-Month Survey Respondent Sample continued

Currently No Longer

Outcome (%)

Enrolled

Enrolled

Total

Most Important Program Goal

Homeownership

21.2

–

–

Employment or career advancement

16.5

–

–

Attainment of education or training credential

15.1

–

–

Improvement in personal or family wel -being

14.5

–

–

Better management of household finances

13.5

–

–

Other

4.9

–

–

All goals equally important

14.4

–

–

Amount of progress toward goal attainment in the past 12 months

A lot

30.8

–

–

Some

42.3

–

–

Very little

17.4

–

–

None

9.5

–

–

Planned or actual use of escrow dol ars

To buy a house

30.2

2.0

24.7

To pay off loans, debts, or bil s

3.0

7.2

3.8

To pay for education

2.4

0.0

1.9

To make a big purchase

2.0

2.0

2.0

To start a business

0.8

0.0

0.6

Since Enrollment in FSS Program

Communications with case managers were

Important

82.1

87.3

83.1

How often referrals to services were helpful

***

Always

31.0

22.1

29.2

Often

20.2

20.6

20.2

Sometimes

26.4

23.1

25.8

Never

17.7

22.6

18.7

No referrals

4.7

11.6

6.1

(continued)

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 48

Chapter 3. Participant Engagement

Exhibit 3.10. Indicators of Engagement with the Family Self-Sufficiency Program, by Current Enrol ment Status, Family Self-Sufficiency Group Members in the 36-Month Survey Respondent Sample continued

Currently No Longer

Outcome (%)

Enrolled

Enrolled

Total

Most often cited obstacles to attainment of FSS goals

Health or disability

10.5

12.7

10.9

Transportation problems

9.9

10.2

10.0

Needed services unavailable

4.6

25.0

8.9***

Childcare problems

8.7

3.0

7.6***

Communications

5.4

10.7

6.5***

Satisfaction with FSS program

***

Satisfied

76.8

51.3

71.5

Neither satisfied nor dissatisfied

11.2

17.6

12.5

Dissatisfied

12.1

31.2

16.0

Sample size

781

207

988

FSS = Family Self-Sufficiency.

NOTES: The FSS group survey respondent sample includes all Housing Choice Voucher heads of household who were randomly assigned to the FSS group between October 18, 2013, and December 22, 2014, were age 18 to 61 at their time of random assignment., and responded to the FSS 36-Month Survey. A chi-square test was applied to differences between current enrollees and FSS group members who reported that they had left the program. Statistical significance levels are indicated as: * = 10 percent; ** = 5 percent;

*** = 1 percent. Rounding may cause slight discrepancies in calculating sums and differences.

SOURCE: MDRC calculations using responses to the FSS 36-Month Survey.

Nevertheless, current enrol ees communicated a more mixed view on some aspects of their involvement with the FSS program. Nearly all current enrol ees related that they had communicated with their FSS case manager in the past year, but only a little more than one-half of these respondents judged that they had had the minimum required number of contacts. Similarly, only about one-half of current enrol ees assessed their referrals to FSS services as being always or often helpful to them, and a somewhat larger proportion related that they experienced at least one obstacle to the attainment of their goals, such as transportation or childcare problems.

As might be expected, former enrol ees, on average, expressed a less positive view of their involvement with the FSS program. For example, only about one-half of former enrol ees declared themselves as satisfied with the FSS program, about 25 percentage points below the average for current enrol ees. In addition, by margins of about 10 percentage points, former enrol ees were less likely to report having had the right number of contacts with FSS case managers or helpful referrals to FSS services. Finally, a much larger proportion of former enrol ees reported that their FSS program had not offered them services that they needed. Former enrol ees were also more likely to cite “communications problems” (a possible indicator of dissatisfaction with FSS case management).

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 49

Chapter 3. Participant Engagement

Effects of Family Self-Sufficiency on Use of Services

This section compares the use of services by members of the FSS and control groups during the first 3 years of fol owup based on responses to the FSS 36-month survey. As chapter 1 discusses, control group members could not receive services from the FSS program and access to FSS escrow accounts for 3 years after random assignment, but they could seek alternative services in their community. Their reported levels of service use represent what would be expected to occur in the absence of an FSS program among voucher holders who were interested in FSS. The average differences between the two research groups in total services received or in receipt of specific types of services represent the effects, or impacts, of the FSS program on service receipt.73

Survey respondents in both research groups were asked whether they had received any of 14 types of services and financial supports in the past 12 months. Using ordinary least squares regression to estimate differences between research groups in service receipt, the analysis controls for possible differences in respondent characteristics measured around the time of random assignment. Exhibit 3.11 displays the results of these comparisons.74 These findings provide important context for interpreting the estimates of the FSS program’s effects on employment, earnings, and other self-sufficiency outcomes that are presented in the chapters that fol ow.75

As exhibit 3.11 shows, respondents in both research groups reported relatively high levels of participation in employment-related or self-sufficiency-related activities. According to survey responses, nearly 70 percent of control group members reported using at least one service in the previous 12 months, and more than three-fourths of the control group were employed for at least 1 month. When participating in activities, control group respondents most often sought out assistance in enrol ing in or paying for healthcare coverage, attended education or training programs, or participated in job search activities. However, compared with the FSS

group relatively few control group members reported receiving assistance in managing their family finances or in preparing to purchase a home.

The FSS program led to a moderate overall increase of 10 percentage points (with rounding) above the control group level in participating in any employment-related or self-sufficiency-related activities.76 The FSS

program realized moderate-level increases above control group levels in the domains of job search activities, homeownership preparation, post-employment services, and education and training but had a much larger effect (of more than 20 percentage points) on the use of financial counseling services. FSS group respondents were also more likely than their counterparts in the control group to combine employment and service use.

Higher levels of participation in education and training activities do not, however, appear to increase the attainment of academic credentials (such as conferral of a degree or a diploma) within the 36-month fol owup period. There are no differences across study groups in these outcomes. There is a small and statistically significant impact, however, on members earning occupational credentials or licenses.

73

Some FSS group members may also report their use of services that they found on their own initiative.

74

For this analysis, statistically significant differences of less than 5 percentage points are small, from 5 to 14.99 percentage points are moderate, and of 15 percentage points or higher are large.

75

When conducting a randomized controlled trial of employment-related or other types of services, it is generally assumed that the program will lead to at least a moderate increase in service receipt above the level for the control group. If service levels are similar between the two research groups, the evaluation may not provide a fair test of the services in question.

76

This pattern was also observed at 18 months of followup (see Verma et al., 2019).

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 50

Chapter 3. Participant Engagement

Exhibit 3.11. Impacts on Use of Services and Employment, Family Self-Sufficiency 36-Month Survey Respondent Sample

FSS

Control

Difference

Outcomes

Group

Group

(Impact)

p-Value

In the past 12 months

Used any services (%)

79.5

69.0

10.5***

0.000

Job search

38.8

24.4

14.4***

0.000

Post-employment services

22.4

15.7

6.6***

0.000

Financial counseling

40.6

17.9

22.8***

0.000

Education or training

30.6

24.9

5.7***

0.005

Homeownership preparation

17.5

6.2

11.3***

0.000

Health coverage or health assistance

25.9

27.7

-1.8

0.383

Social services

19.3

19.1

0.3

0.892

Supportive services

20.2

17.1

3.1*

0.077

Ever employed (%)

78.0

76.4

1.6

0.342

Average number of domains of service use or

employment

3.1

2.4

0.7***

0.000

3 or more domains (%)

55.6

39.3

16.3***

0.000

Earned academic credential

9.1

7.3

1.8

0.140

High school diploma or equivalency certificate

2.4

1.7

0.7

0.286

Post-secondary degree

6.7

5.6

1.2

0.285

Earned occupational credential or license

15.1

12.5

2.6*

0.096

Sample size (total = 1,936)

988

948

FSS = Family Self-Sufficiency.

NOTES: The FSS 36-Month Survey respondent sample includes Housing Choice Voucher heads of household who were randomly assigned between October 18, 2013, and December 22, 2014, were age 18 to 61 at the time of random assignment and responded to the FSS 36-Month Survey. Sample sizes for specific outcomes may vary because of missing values. Estimates were regression-adjusted using ordinary least squares, controlling for differences in sample member characteristics recorded at the time of random assignment.

No special weights were applied to responses to adjust for differences in sample size by housing authority. Rounding may cause slight discrepancies in calculating sums and differences. A two-tailed t-test was applied to differences between the FSS group and the control group. The p-value indicates the likelihood that the difference between the FSS group and control group arose by chance. Statistical significance levels are indicated as: * = 10 percent; ** = 5 percent; and *** = 1 percent.

SOURCE: MDRC calculations using responses to the FSS 36-Month Survey.

The evaluation also examined FSS program impacts on service use according to sample members’ baseline employment and educational attainment. These findings are shown in appendix exhibits B.4 through B.6.

Among these subgroups, large majorities of respondents in both research groups reported using at least one service during the 12 months before their 36-month survey interview. The FSS program generally increased service use above the levels for the control group, but for only one subgroup—respondents with a high school diploma or equivalency certificate—did the FSS program lead to large impacts (of 15 percentage points or more) in the use of any services. Among all subgroups, the FSS program increased the use of assistance with job search or post-employment services, financial counseling, and homeownership preparation services by Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 51

Chapter 3. Participant Engagement

moderate to large margins above the levels reported by the control group. FSS led to little or no effect on attendance in education and training courses, nor on receipt of supportive, social, or health services.

Educational attainment subgroups showed greater variation in the incidence and magnitude of impacts on the use of specific types of services, compared with subgroups based on self-reported employment at baseline. For example, among survey respondents with a 2-year col ege degree or higher, the FSS program led to especially large increases (of 31 percentage points and 17 percentage points, respectively) above control group levels in the use of financial counseling and homeownership preparation services. These effects were nearly three times larger than the increases above the control group for the FSS group among respondents with no degree or educational credential at baseline. In contrast, FSS group members with a 2-year col ege degree or higher were the only subgroup among the four educational attainment subgroups with a lower incidence of receipt of supportive services compared with control group.77

As appendix exhibits B.5 and B.7 show, around 10 to 20 percent of survey respondents in the FSS group reported having earned an academic degree or occupational credential after random assignment. The FSS program led to small or moderate-level increases above control group levels in the attainment of an academic degree or occupational credential among the more disadvantaged subgroups—respondents without employment and respondents without a high school degree or equivalent at baseline. Among other subgroups, similar proportions of respondents in each research group reported the attainment of an academic degree or occupational credential.

Effects of Program Implementation Features on Service Use

As noted above, the incidence of FSS service use was highest among FSS group members in housing agencies with the strongest emphasis on monitoring and engagement. Nonetheless, programs with unusually high rates of service use may not have the largest increases in service use compared with what would have happened in the absence of the program. For example, in a relatively service-rich locality, a similarly large proportion of control group members could participate in alternative services on their own initiative. To test for correlations between possible effects of housing authorities’ emphasis on monitoring and engagement and levels of service use, housing authorities were grouped into three clusters, representing relatively low, medium, or high levels of emphasis on monitoring and engagement. Then, separate estimates of program effects on service use for each cluster were calculated.

As shown in appendix exhibit B.8, according to survey responses, each type of implementation practice led to at least a moderate increase in overall service use above the levels reported by control group respondents, although the FSS-control group difference of 5 percentage points for housing agencies with a high emphasis on monitoring and engagement is not statistically significant. The impact on the use of any services was about twice as large among the housing agencies with a low or moderate emphasis on monitoring and engagement.

(These differences in impacts among the housing agency clusters are not statistically significant.) Variation in control group participation patterns account for most of this difference in impacts. Whereas about 80

percent of FSS group respondents in all three types of housing authorities reported participating in at least one activity during the 12 months before their 36-month survey interview, the incidence of service use for control group respondents ranged from 65 percent in housing agencies with the least emphasis on monitoring and engagement to 76 percent in housing authorities with the strongest emphasis. Only for homeownership 77

Variation among educational attainment subgroups in the magnitude and direction of impacts on these three outcomes is statistically significant at the 10 percent level or less.

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 52

Chapter 3. Participant Engagement

preparation and, to a lesser extent, the provision of financial counseling is where we witness the greatest impact the among housing authorities with the strongest emphasis on monitoring and engagement.

As appendix exhibit B.9 shows, none of the program approaches had much effect on the incidence of earning an academic degree or occupational credential. The one exception was the group of programs with a medium-level of emphasis on monitoring and engagement. Within this group, about 11 percent of respondents in the FSS group reported having earned an academic degree since random assignment, an increase of 5 percentage points above the control group level.

Other implementation strategies could potentially affect impacts on service use. One possibility is the type of services that public housing agencies emphasized. Exhibits 3.12 to 3.14 show how impacts on service use vary, depending on how strongly the PHA emphasized job search and post-employment services, education and training, or financial services. As these exhibits show, the FSS program consistently increased overal service use above control group levels (from 7 to 14 percentage points), irrespective of the PHA’s emphasis on service type. As expected, the impact on the use of job search services was largest (18 percentage points) among PHAs with the strongest emphasis on job search and post-employment services. Similarly, PHAs with the strongest emphasis on education and training averaged the largest impact (12 percentage points) on attendance in education and training courses among all PHAs. In contrast, the FSS program led to consistently large increases of more than 20 percentage points above control group levels in the use of financial counseling and attendance at financial education workshops. Also shown in exhibits 3.12 to 3.14, FSS programs, including those with a strong emphasis on education and training, led to only small and scattered increases above the control group in earning an academic or occupational credential.

These results are relevant when considering the possible impacts of FSS on employment and earnings and other financial outcomes, presented in chapter 6. Given that FSS programs with different types of implementation practices all led to increases in service use above control group levels, it is reasonable to expect that most programs would eventually lead to positive effects. This report examines whether these types of effects are realized within a 3-year period.78

78

A final report will estimate the effects of the FSS program on a variety of outcomes calculated with administrative records over an additional 2

to 3 years of followup—beyond the time when most FSS group members would be expected to complete their FSS Contract of Participation.

Impacts on other aspects of service use, such as the duration and quality of services and the attainment of academic or occupational credentials, could also contribute to program effects on employment, earnings, and other financial outcomes.

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 53

Chapter 3. Participant Engagement

Exhibit 3.12. Impacts on Use of Services During Years 1 to 3, by Program Emphasis on Job Search and Post-Employment Services

20

18.3***

15

10.5*** 10.6**

10.9*

10

9.0*

8.3*

5.3*

5

4.0

4.3*

1.6

-0.4

-0.1

0

-5

Used Any

Job Search or

Earned

Earned

Services

Post-Employment

Academic

Occupational

Services

Credential

Credential or

License

Low

Medium

High

NOTES: The FSS 36-Month Survey respondent sample includes Housing Choice Voucher heads of household who were randomly assigned between October 18, 2013, and December 22, 2014, were age 18 to 61 at the time of random assignment and responded to the FSS 36-Month Survey. Estimates were regression-adjusted using ordinary least squares, controlling for differences in sample member characteristics recorded at the time of random assignment. No special weights were applied to responses to adjust for differences in sample size by housing authority. A two-tailed t-test was applied to differences between the FSS group and the control group. Statistical significance levels are indicated as: * = 10 percent; ** = 5 percent; and *** = 1 percent. The H-statistic test was used to test for statistically significant differences in impact estimates across subgroups. Statistical significance levels are indicated as: † = 10 percent; †† = 5 percent; and ††† = 1 percent.

SOURCES: MDRC calculations using responses to the FSS 36-Month Survey, data collected from Individual Training and Services Plan forms, and information provided by FSS administrators and case managers.

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 54

Chapter 3. Participant Engagement

Exhibit 3.13. Impacts on Use of Services During Years 1 to 3, by Program Emphasis on Education and Training

15

14.3***

12.0***

12

9

6.9*

7.2*

6.4**

5.9**

6

5.2

2.9

3

1.9

1.3

0.5

-1.3

0

-3

Used Any

Education

Earned

Earned

Services

or Training

Academic

Occupational

Credential†

Credential or

License

Low

Medium

High

NOTES: The Family Self-Sufficiency (FSS) 36-Month Survey respondent sample includes Housing Choice Voucher heads of household who were randomly assigned between October 18, 2013, and December 22, 2014, were age 18 to 61 at the time of random assignment and responded to the FSS 36-Month Survey. Estimates were regression-adjusted using ordinary least squares, controlling for differences in sample member characteristics recorded at the time of random assignment. No special weights were applied to responses to adjust for differences in sample size by housing authority. A two-tailed t-test was applied to differences between the FSS group and the control group. Statistical significance levels are indicated as: * = 10 percent; ** = 5 percent; and *** = 1 percent. The H-statistic test was used to test for statistically significant differences in impact estimates across subgroups. Statistical significance levels are indicated as: † = 10

percent; †† = 5 percent; and ††† = 1 percent.

SOURCES: MDRC calculations using responses to the FSS 36-Month Survey, data collected from Individual Training and Services Plan forms, and information provided by FSS administrators and case managers.

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 55

Chapter 3. Participant Engagement

Exhibit 3.14. Impacts on Use of Services During Years 1 to 3, by Program Emphasis on Financial Services

25

24.6***

24.2***

21.0***

20

15

12.7***

10

9.4***

7.3

5

4.1*

3.5

2.8

1.3

1.6

-0.3

0

-5

Used Any

Financial

Earned

Earned

Services

Counseling or

Academic

Occupational

Workshops

Credential

Credential or

License

Low

Medium

High

NOTES: The FSS 36-Month Survey respondent sample includes Housing Choice Voucher heads of household who were randomly assigned between October 18, 2013, and December 22, 2014, were age 18 to 61 at the time of random assignment and responded to the FSS 36-Month Survey. Estimates were regression-adjusted using ordinary least squares, controlling for differences in sample member characteristics recorded at the time of random assignment. No special weights were applied to responses to adjust for differences in sample size by housing authority. A two-tailed t-test was applied to differences between the FSS group and the control group. Statistical significance levels are indicated as: * = 10 percent; ** = 5 percent; and *** = 1 percent. The H-statistic test was used to test for statistically significant differences in impact estimates across subgroups. Statistical significance levels are indicated as: † = 10 percent; †† = 5 percent; and ††† = 1 percent.

SOURCES: MDRC calculations using responses to the FSS 36-Month Survey, data collected from Individual Training and Services Plan forms, and information provided by FSS administrators and case managers.

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 56

Chapter 3. Participant Engagement

Conclusions

FSS is a relatively ambitious and long-term voluntary program that requires ongoing commitment from participants to work toward self-sufficiency goals. FSS group members’ patterns of service use resemble patterns recorded in other voluntary employment-focused programs for recipients of housing assistance, consisting of a relatively high incidence of involvement, primarily in short-term job search or financial security activities in the first year or so after enrol ment, fol owed by a drop-off in engagement.79 As discussed above, certain types of case management practices can increase participants’ engagement with the program.

Nonetheless, sustaining participants’ involvement in FSS during future years will likely be a chal enge for administrators and case managers. According to the FSS program design, participants, as they continue their involvement with the program, are expected to experience tangible signs of progress, such as the attainment of educational or vocational credentials, increased credit scores and other improvements in household finances, increased earnings, and the accumulation of funds in an escrow account. In turn, it is hoped that reaching these milestones will motivate participants to maintain their engagement, eventually leading to their graduation and the disbursement of their escrow dol ars. The fol owing chapters begin to explore whether FSS

group members are experiencing potential improvements in self-sufficiency in the form of accrual of escrow credits (chapter 4), increases in employment and earnings compared with the control group (chapter 5), or increases in indicators of self-sufficiency and financial wel -being (chapter 6).

79

See, for example, Nuñez, Verma, and Yang (2015: 34–42) and Greenberg et al. (2015: 25–28) for analyses of trends in service use by recipients of housing assistance.

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 57

Chapter 4. Escrow Accruals and Graduation

Chapter 4. Escrow Accruals and

Graduation

The escrow account, a core feature of the Family Self-Sufficiency (FSS) program, is intended to encourage families to increase their earnings, reduce their reliance on public cash assistance programs, and build savings.

Under the Housing Choice Voucher (HCV) program, most families pay 30 percent of their monthly adjusted income, known as the total tenant payment (TTP), for their rent and utility expenses, and the public housing agencies (PHAs) subsidize the remaining portion owed. As a result, in most instances, when a household’s income increases, its TTP also increases. In the FSS program, when the tenant pays the increased TTP, the housing agency credits the family’s escrow account based on the increase in earned income (HUD, 2017a).80

Upon graduation from FSS, the escrow balance in the account, with accrued interest, is disbursed to the FSS

participant (the head of household), with no restrictions on the use of the money. Under special circumstances, FSS al ows participants to access their escrow funds earlier than graduation—that is, receive an interim disbursement—for approved purposes related to their self-sufficiency goals, such as paying for emergency car repairs to prevent job loss.

The first report produced as part of this evaluation, which focused on the first 18 months of fol owup, showed that 38 percent of the FSS group had received an escrow credit during that fol owup period, and 35 percent of the FSS group maintained a positive balance in Month 18. Chapter 4 extends that analysis and examines escrow credits and balances during the first 3 years of fol owup. It first presents findings on escrow credits and accruals for the whole FSS group. Then, it explores how the incidence of escrow accrual and the accumulation of credits over time varied for selected subgroups. Next, the chapter considers whether the patterns of escrow accrual varied by housing agency and, if so, whether these differences appear to be related to variation in how particular housing authorities implement FSS.

The findings show the fol owing:

• Over the 3-year fol owup period, nearly 52 percent of the FSS group had received at least one escrow credit.

• By the last month of Year 3, 40 percent of the FSS group members still maintained a positive balance.

Those with a positive balance in Month 36 had an average of nearly $3,700 in their account (about $1,450

among all FSS group members), and about one-fourth of these participants had accrued more than $5,000.

• On average, FSS group members who were underemployed at random assignment—that is, who were working part-time hours or receiving relatively low annual earnings—were more likely to accrue escrow credits compared with subgroups who were working ful -time hours or receiving relatively high earnings.

Their incidence of escrow accrual also exceeded the rate for FSS group members who were without employment at random assignment.

• A larger proportion of FSS group members with a positive balance at the end of Year 3 maintained contact with their FSS case manager and used an FSS service during Year 3, compared with FSS group members with no balance. Nonetheless, a relatively large proportion of FSS group members with a positive balance were working during Year 3 but did not participate in FSS services.

80

The amount of escrow credited to the account depends on the household’s income level; those with the lowest incomes are credited the amount equal to the rental increase, and those with higher incomes are credited with a percentage of the increase.

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 58

Chapter 4. Escrow Accruals and Graduation

• Housing agencies differed substantially—from about 27 to 58 percent—in the proportion of FSS group members with a positive balance at the end of Year 3. Housing agencies with the highest rates of service use did not rank highest in escrow accrual, either because participants had not yet begun employment or for other reasons.

How Family Self-Sufficiency Escrow Works

Escrow calculations can be complicated (HUD, 2016a). When the head of household enrol s in the FSS

program and completes a Contract of Participation (COP), the FSS case manager records on the COP the participant’s “baseline” earnings from the most recent HCV recertification meeting. The earnings noted on the COP serve as the standard for calculating escrow credits in future months. Throughout the FSS contract period, FSS participants continue to pay their TTP for rent and utilities according to the same rules as other HCV holders. The housing agency maintains a single, interest-bearing depository account and records the balance for each individual in a separate ledger. When the head of household reports an increase in earned income to the housing agency (and the increase is verified), the FSS participant’s out-of-pocket payment for rent and utilities increases, but under FSS program rules the housing agency issues an escrow credit for the amount of the increase attributable to an increase in earned income and deposits the money in the FSS participant’s escrow account.

The likelihood of accruing escrow can vary due to a number of factors. FSS participants’ employment status at program enrol ment, level of educational attainment, and other characteristics and life experiences can often affect their chances of increasing earnings over time, which, in turn, affects whether—and how quickly—their escrow balance increases. In particular, the relationship between a participant’s relative advantages in the labor market and escrow accrual may be complex. At the extremes, FSS participants who are not working at program entry could potentially benefit most from the FSS escrow because all their future earnings would be included in the calculation of escrow credits. Unemployed adults, however, may also face the most severe barriers to finding and maintaining employment, which is required for graduation and earning the escrow.

In contrast, FSS participants who enter the program while they are working full time or receiving relatively high earnings may have the best prospects of increasing their earnings by finding a better job, increasing their hours, or advancing with their current employer (Verma et al., 2017).81 They may also be most likely to maintain their employment after they start accruing credits. The increase in their earnings, however, may be relatively small compared with their current earnings and lead to only a small amount being credited to their escrow account each month.

Housing agency practices and messages can also affect the incidence of escrow credits and the rate of increase of escrow account balances. HUD FSS program rules require PHAs to issue an annual escrow account statement to program participants. In addition, at most sites, the possibility of accumulating escrow is used to motivate participants throughout the contract period to stay engaged in the program and meet their goals to graduate and thus receive the amount accrued in their escrow accounts. Staff members interviewed reported that they often use the escrow account as an incentive to participate because “money talks.” They noted that accruing escrow was usually an HCV holder’s main incentive for joining FSS. According to them, even FSS

group members who initially showed little or no interest in building escrow became more motivated to accrue 81

In the Work Rewards FSS study, those working at study entry were more likely to meet the graduation requirements than those who were not working. About 31 percent of the FSS-only households in the subgroup who was not working at the time of random assignment graduated from FSS, whereas 55 percent from the working subgroup graduated. The working subgroup also collected more escrow savings on average than the nonworking subgroup. Those in the FSS-only group (who received only the FSS program and made up one of three different programs groups in the study) who were working at random assignment received an average disbursement of more than $2,000; those who were not working at random assignment received an average disbursement of $1,000.

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 59

Chapter 4. Escrow Accruals and Graduation

escrow once they saw their balance increase and gained a more tangible understanding of how escrow accrual worked. According to staff members, interest in accruing escrow credits also gets stronger among participants with a positive balance as they get closer to graduation and the funds seem more attainable.

Interim Escrow Disbursements

As chapter 1 notes, most FSS programs in the national evaluation have a policy that permits participants to request one or more interim (pre-graduation) disbursements of a portion of the funds in their escrow account. Federal guidelines on issuing interim disbursements stress the importance of using these funds

“for purposes consistent with [the participant’s] goals (for example, to pay for education or training or to purchase or repair a vehicle to get to work).” (HUD, 2017b: 93). Housing agencies have discretion on how to interpret these guidelines—for example, whether to approve the use of interim disbursements to reduce debt or prevent eviction. Irrespective of policy, the data examined in chapter 4 show that in practice, few interim disbursements were granted. Per case managers, and consistent with the policy, participants most often used their interim disbursement to pay for car repairs, uniforms, training fees, or books.

Early Graduations and Escrow Disbursements

FSS participants can spend 5 years or more in the program before they qualify for graduation but may graduate in less time under special circumstances. For example, per HUD guidelines, an employed head of household in the FSS program may graduate early if the household’s adjusted monthly income exceeds the maximum level for eligibility to receive a housing subsidy. FSS case managers may also al ow a participant to graduate early if he or she was employed and had already met all or most of the other FSS goals, but could no longer participate in the program.82 By definition, early graduates have less time to accrue escrow deposits than graduates who remain in FSS throughout their entire contract period, especially if they experience earnings increases.

Patterns of Escrow Accrual

Fol owing HUD guidance, housing agencies compare current earnings (projected into an annual total) with baseline earnings recorded in the COP to determine the amount of escrow to credit to a participant’s account (HUD, 2017b: 92–93).83 For the study sample, exhibit 4.1 shows the pattern during the 36-month fol owup period for this report. The solid line displays the percentage of the FSS group members with a positive escrow balance in each month of fol owup, and the dashed line shows the percentage of the FSS group with an escrow credit deposited to their escrow account in each month of fol owup. FSS group members began opening accounts and accruing escrow credits at a rapid pace during the last half of Year 1 and the first half of Year 2. At the end of Year 1, nearly one-fourth of FSS group members had an escrow account with a positive balance. One year later, the proportion had increased to 39 percent. During Year 2, around one-fourth of FSS group members accrued an escrow credit in any given month, the highest rate of escrow accrual recorded during the fol owup. This rate of escrow accrual resulted from additional FSS group members opening new accounts and because once FSS group members began accruing credits, they often continued to do so during several months. During Year 3, the proportion of FSS group members with a positive balance in their account remained steady at around 40 percent. Relatively few additional FSS group members (about 8 percent, as 82

As seen in one FSS group member’s case file, a situation that can sometimes lead to an early graduation occurs when an FSS participant who has accrued escrow moves to a new locality whose housing agency has no openings in their FSS program or does not operate an FSS program.

83

In reality, and as HUD’s newly released FSS resource guide describes, the housing agencies are required to deposit all escrow funds for all FSS participants into a single, interest-bearing depository account and to account for these funds through a subsidiary ledger that records the balance of each FSS participant’s individual account value (within the single account).

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 60

Chapter 4. Escrow Accruals and Graduation

exhibit 4.2 shows) opened an escrow account during Year 3, and a similar proportion of FSS group members who had accrued escrow credits either graduated early (and received their escrow dol ars) or, more often, left the FSS program and forfeited their escrow credits.

Exhibit 4.1. Monthly Indicators of Positive Escrow Balance and Credit Accrual, Family Self-Sufficiency Impact Sample

60

50

40

30

Percentage (%) 20

10

0

2 4 6 8 10 12 14 16 18 20 22 24 26 28 30 32 34 36

Month Relative to Random Assignment

Balance Greater Than $0

Accrued Escrow Credit

NOTES: The FSS impact sample includes Housing Choice Voucher heads of household that were randomly assigned between October 18, 2013, and December 22, 2014, and were 18 to 61 at the time of random assignment.

SOURCE: MDRC calculations from housing authority administrative data.

Exhibit 4.2 provides some context for the patterns displayed in exhibit 4.1. The top panel reports the percentage of FSS group members with household earnings recorded at baseline. About 43 percent of FSS

group members reported no household earnings at program entry. Nearly 13 percent of the FSS group reported earnings of up to $10,000. This group included the largest proportion of part-time employees (64

percent), according to FSS group members’ responses to questions about their employment status, recorded in the Baseline Information Form (BIF) that they completed at the time of random assignment. About 16 percent of FSS group members received annual household earnings of between $10,001 and $20,000. This group includes a relatively even mix of part-time and ful -time employees, according to responses to the BIF. The last group, composing about 22 percent of FSS group members, reported annual earnings in excess of $20,000.

Most members of this group (72 percent) were working full time at random assignment. As discussed above, FSS group members with no reported earnings at baseline could accumulate hundreds of dol ars per month in escrow once they found employment. Those with some earnings recorded at baseline would need to increase their hours of work or find a higher paying job to accrue escrow savings.

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 61

Chapter 4. Escrow Accruals and Graduation

The rest of exhibit 4.2 displays several important indicators of escrow accrual. The first column (“FSS Group”) shows results for all members of the impact sample, including FSS group members with no escrow accrual in Years 1 to 3; the second column displays findings for FSS group members who had at least one escrow credit; and the third column displays findings for FSS group members with a positive balance in their escrow account during Month 36, the final month of fol owup covered in this report. About 52 percent of all FSS group members (the impact sample) accrued at least one credit.84 Most FSS group members with an escrow balance in the 36-month fol owup period started accruing escrow after Year 1, usually during Year 2.

The typical FSS group member with at least one escrow credit continued accruing credits during 12 to 13

additional months. During the first 3 years of fol owup, only about 4 percent of FSS group members in the impact sample graduated or received an interim escrow disbursement. Exits from the FSS program (unrelated to graduations) accounted for a fairly large drop-off in escrow accumulation, as about one in five FSS group members who had accrued credits to their escrow account no longer maintained a positive balance in Month 36.85 Balances for many escrow account holders who remained in the FSS program, however, increased steadily month by month. As of Month 36, FSS group members with a positive balance had an average of nearly $3,700 in their account (nearly $1,500 among all FSS group members), and over one-fourth of these participants had accrued more than $5,000.86

84

For this analysis, the FSS impact sample includes HCV heads of household who were randomly assigned to the FSS group between October 18, 2013, and December 22, 2014, and were age 18 to 61 at the time of random assignment. This amount increases to about 55 percent of FSS group members receiving an escrow credit when only those who enrolled in the program are included in the calculation (not shown in the exhibit).

85

The middle column of exhibit 4.2 shows that 77 percent of FSS group members who accrued at least $1 in escrow credits during Years 1

to 3 maintained a positive balance in Month 36. By inference, about 23 percent no longer had escrow credits at that time, including about 5

percent who graduated early from FSS.

86

Similar to savings and money market accounts, escrow accounts earned negligible amounts of interest in the short term. With few exceptions, FSS group members with a positive balance accumulated less than $30 of interest through Month 36.

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 62

Chapter 4. Escrow Accruals and Graduation

Exhibit 4.2. Accrual and Disbursement of Family Self-Sufficiency Escrow Credits in Years 1 to 3, Family Self-Sufficiency Impact Sample

FSS

1 or More Months

Current Balance

Outcome

Group

of Escrow Accrual

Greater Than $0

Years 1–3

Annual household earnings recorded at FSS enrol ment (%)

Never enrol ed

6.2

n/a

n/a

$0

43.0

42.6

43.5

$1–$10,000

12.8

16.2

15.9

$10,001–$20,000

16.2

22.0

23.1

More than $20,000

21.8

19.2

17.5

At least one month of credit accrual (%)

51.7

100.0

100.0

Year of initial credit accrual (%)

No credit accrued

48.3

n/a

n/a

Year 1

24.5

47.4

44.4

Year 2

19.1

37.1

37.2

Year 3

8.0

15.5

18.5

Average months of credit accrual

7.1

13.8

15.1

Months of credit accrual (%)

0

48.3

n/a

n/a

1–6

13.2

25.6

19.8

7–12

12.2

23.6

23.4

13–24

18.8

36.3

38.7

25–36

7.5

14.5

18.1

Average credit per month of accrual ($)

n/a

243

238

Month 36

Had balance in Month 36 (%)

40.0

77.4

100.0

Average balance in Month 36 ($)

1,462

2,830

3,656

Month 36 balance (%)

$0

60.0

22.6

0.0

$1–$500

7.5

14.5

18.7

$501–$2,000

11.1

21.5

27.8

$2,001–$5,000

10.8

20.9

27.0

$5,001 or more

10.6

20.5

26.5

(continued)

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 63

Chapter 4. Escrow Accruals and Graduation

Exhibit 4.2. Accrual and Disbursement of Family Self-Sufficiency Escrow Credits in Years 1 to 3, Family Self-Sufficiency Impact Sample continued

FSS

1 or More Months

Current Balance

Outcome

Group

of Escrow Accrual

Greater Than $0

Received escrow disbursement (%)

3.5

6.8

1.6

Received interim disbursement

0.8

1.5

1.6

Received graduate disbursement

2.7

5.3

0.0

Sample size

1,285

664

514

FSS = Family Self-Sufficiency.

NOTES: The FSS impact sample includes Housing Choice Voucher heads of household who were randomly assigned between October 18, 2013, and December 22, 2014, and were age 18 to 61 at the time of random assignment. Rounding may cause slight discrepancies in calculating sums.

SOURCE: MDRC calculations from housing authority administrative data.

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 64

Chapter 4. Escrow Accruals and Graduation

Pathways to Accruing Escrow Credits

The FSS COP states that one of the FSS participant’s primary responsibilities is to “seek and maintain suitable employment after completion of the job training programs listed in the individual training and services plan”

(HUD, 2017a). This requirement implies that FSS participants should sequence their self-sufficiency efforts and focus on completing their education and training or other services before attempting to increase their earnings and accrue escrow credits. In practice, there is a fair amount of flexibility in how the PHAs exercise this requirement. Analysis of FSS group members’ patterns of service use and program-recorded employment during Months 1 to 36, discussed in chapter 3, suggests that FSS group members engaged in service use, employment, and accrual of escrow credits in different ways. Nevertheless, from the standpoint of accruing escrow credits, maintaining employment during Year 3 mattered most of them. About two-thirds of FSS group members with program-recorded employment in Year 3 had a positive balance in their escrow account at the end of the year. This average was similar for groups of FSS group members who (1) combined employment and service use in Year 3, (2) were employed but no longer used FSS-related services in Year 3, or (3) were employed and never used FSS-related services at any time since program enrol ment.87

Variation in Escrow Accrual for Selected Subgroups

This section explores whether most FSS group members experienced a similar pattern of escrow accrual during Years 1 to 3, or whether the incidence of escrow accrual or the amount accrued varied by subgroup.

Evidence from the recently completed Work Rewards evaluation in New York City, which fol owed study participants for 6 years, found that FSS participants who were working at the time of random assignment were more likely to regularly earn escrow credits and successfully graduate from the program. The households in the nonworking subgroup appeared to have accrued escrow credits at higher rates earlier in the program but were less likely to graduate from FSS than those in the working subgroup (Verma et al., 2017). The national FSS evaluation also examines escrow accruals for the same subgroups.

Exhibit 4.3 displays important indicators of escrow accrual for selected subgroups, using data on baseline earnings col ected from COP forms, housing agency administrative records, and responses to the BIF. These results suggest that FSS group members who were working intermittently or were working part time at program enrol ment were relatively successful in accumulating escrow credits through Month 36. As exhibit 4.3 shows, about 57 percent of FSS group members with recorded household earnings of between $10,001

and $20,000 per year (consistent with intermittent or part-time employment) had a positive balance in their escrow account in Month 36, as did 49 percent of FSS group members with household earnings of between $1 and $10,000, and 47 percent of FSS group members who reported at random assignment that they were working part time. These proportions exceed the averages both for FSS group members without employment and for FSS group members with the highest level of earnings or with ful -time employment. These three underemployed subgroups also averaged the largest number of months of escrow accrual (10.2, 8.9, and 8.5

months, respectively) of any related subgroup. Finally, the subgroup with household earnings of between $1

and $10,000 had the largest average balance in their escrow account ($2,066).

87

By FSS rules, an FSS group member who began accruing escrow credits but then left or lost employment could still maintain a positive balance at the end of Year 3 and resume accruing credits upon finding a new job. In practice, this rarely happened. It may be hypothesized that the group who combined employment and FSS service use has a greater likelihood of graduating from the FSS program and receiving the money in their FSS account, because these FSS group members appear to be more engaged and have probably fulfilled more program goals than the group with only program-recorded employment.

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 65

Chapter 4. Escrow Accruals and Graduation

Exhibit 4.3. Indicators of Family Self-Sufficiency Escrow Account Accrual in Months 1 to 36

for Selected Subgroups

Month 36

Number

Escrow

of Months

Escrow Balance

Average

Balance of

Sample with Escrow

Greater Than

Escrow

More Than

Subgroup and Outcome

Size

Credits

$0 (%)

Balance ($)

$2,000 (%)

Annual Household Earnings at FSS Enrollment

No earnings

552

6.9

40.2

1,808

25.2

$1–$10,000

165

8.9

49.1

2,066

26.1

$10,001–$20,000

208

10.2

56.7

1,669

27.4

$20,001 or more

280

6.2

31.8

622

11.8

Employment Status at Random Assignment

Not employed

568

5.9

34.9

1,486

21.3

Employed 1–34 hours per week

336

8.5

46.7

1,601

22.9

Employed 35 hours per week or

more

372

7.8

41.4

1,298

19.9

Highest Level of Educational Attainment at Random Assignment

No degree or credential

259

6.7

40.5

1,399

22.4

High school diploma or

equivalency certificate

320

7.3

38.1

1,507

20.0

Some col ege

470

6.6

39.8

1,271

20.0

2-year col ege degree or higher

236

8.6

42.4

1,852

25.0

PHA Emphasis on Monitoring and Engagement

Low

472

7.8

46.8

1,645

24.6

Medium

551

7.4

39.9

1,379

19.8

High

262

5.5

27.9

1,307

19.1

FSS = Family Self-Sufficiency. PHA = public housing agency.

NOTES: The FSS impact sample includes Housing Choice Voucher heads of household who were randomly assigned between October 18, 2013, and December 22, 2014, and were age 18 to 61 at the time of random assignment. PHA emphasis on monitoring and engagement is defined by intensity of contact and case management practices, as divided into three levels.

SOURCES: MDRC calculations from baseline data, Contract of Participation forms, and housing authority administrative data.

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 66

Chapter 4. Escrow Accruals and Graduation

In contrast, smal er proportions of FSS group members who worked ful -time hours at program entry or who lived in households with relatively high earnings had a positive balance in Month 36. Moreover, members of these subgroups who accrued credits tended to add relatively small amounts to their balance month by month. For example, only 32 percent of the subgroup with estimated annual household earnings that exceeded $20,000 had a positive balance in their escrow account in Month 36. Moreover, in Month 36, members of this subgroup averaged only $622 in escrow credits, less than one-half of the average for all other subgroups with less than or equal to $20,000 per year in household earnings.

The findings also highlight chal enges in accruing escrow experienced by members of subgroups with greater barriers to employment. For example, FSS group members who reported on the BIF that they were not working at random assignment recorded a relatively low incidence (about 35 percent) of having a positive balance in their escrow account. Yet, members of this subgroup who subsequently found employment or increased their earnings often realized relatively large increases in their escrow account during each month in which they accrued credits. For example, among FSS group members with a positive escrow balance in Month 36, the subgroup with no self-reported employment at random assignment had the highest average escrow balance ($4,262) among the three employment-status subgroups. This pattern of escrow accumulation was similar for subgroups based on the amount of household earnings recorded on FSS group members’

COP forms.88

One exception to the finding that FSS group members with greater advantages in the labor market tended to accrue fewer escrow credits concerns the subgroups defined by the highest educational attainment. As exhibit 4.3 shows, in Month 36, FSS group members in all four educational-attainment subgroups had a similar likelihood (about 40 percent) of maintaining a positive balance in their escrow account, but FSS group members with at least a 2-year col ege degree had a higher average balance ($1,852) compared with FSS group members with less educational attainment.

Variation in Escrow Accrual by Housing Agency

Exhibits 4.3 (bottom panel) and 4.4 summarize the variation in measures of escrow accrual among the 18

housing agencies.89 The differences among housing authorities were substantial. For example, for the three housing authorities with the highest incidence of escrow accrual, an average of 58 percent of FSS group members had a positive balance in their escrow account in Month 36, whereas the three housing authorities with the lowest incidence averaged only 27 percent (see exhibit 4.4). Similarly, the typical FSS group member in the three housing authorities ranked first to third had a balance of $2,708 in escrow credits in Month 36, more than four times the average for housing authorities ranked 16th through 18th on this measure ($635). A similar difference prevails for the highest and lowest ranking housing authorities for the indicator of having an escrow balance of more than $2,000.90

88

The calculation for the not-employed subgroup is $1,486 / .349 = $4,262. The comparable average for FSS group members with no recorded household earnings on their COP form was $4,496.

89

For this analysis, escrow accrual results are presented for the FSS service-use sample ( n = 1,004). Housing authority averages for this sample closely resemble results for FSS group members in the larger impact sample ( n = 1,285).

90

See chapter 7, pages 125–127 in this report for additional analysis of variation in rates of escrow accrual among housing agencies.

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 67

Chapter 4. Escrow Accruals and Graduation

Exhibit 4.4. Indicators of Escrow Credits Accrual in Months 1 to 36 by Housing Authority, Family Self-Sufficiency Service-Use Sample

Escrow Balance

Average

Escrow

Greater Than

Escrow

Balance Over

Outcome

$0 (%)

Balance ($)

$2,000 (%)

Average value for the 3 highest-ranking housing

authorities

58.3

2,708

37.7

Median value for all housing authorities

40.0

1,443

22.0

Average value for the 3 lowest-ranking housing authorities

27.0

635

9.6

Correlation Coefficients (housing authority averages)

Average FSS caseload sizea

-0.17

0.00

-0.11

Number of expected contacts

-0.19

-0.01

-0.08

Proportion of FSS group with a Year 1 goal

-0.60

-0.32

-0.33

Proportion of FSS group with a job search or

post-employment goal

-0.03

0.00

0.11

Proportion of FSS group with an education or

training goal

0.40

0.56

0.52

Proportion of FSS group with a financial security goal

0.38

0.47

0.35

Emphasis on monitoring and engagement

(composite score)

-0.38

-0.13

-0.21

b

Number of housing authorities (total = 18)

FSS = Family Self-Sufficiency.

aAverage caseload sizes were multiplied by -1 to test whether small caseload sizes are positively correlated with greater incidence of escrow accrual and higher positive balances.

bThe FSS monitoring and engagement composite score incorporates three components: average caseload size, expected number of contacts per year, and the proportion of FSS group members with a Year 1 goal. A Z-score for each component was calculated using the site value and the mean of all 18 sites. The Z-scores were summed to create the composite value.

NOTES: The FSS service-use sample includes all or a randomly selected subsample of Housing Choice Voucher heads of household who were randomly assigned to the FSS group between October 18, 2013, and December 22, 2014, and were age 18 to 61 at the time of random assignment.

SOURCE: MDRC calculations from housing authority administrative data and information provided by FSS administrators and case managers.

Measures displayed in the bottom panel of exhibit 4.4 present results of simple tests of association (correlation coefficients) among the indicators of housing agency implementation features that are presented in chapter 3 and housing agency averages for escrow credit accrual. As is true for the analysis in chapter 3, correlation coefficients with values close to +1.000 suggest a strong positive association between the program implementation feature and the accumulation of escrow credits. For example, the coefficient at the bottom of the first column “Escrow Balance Greater Than $0,” shows the result of testing whether FSS group members in housing authorities that strongly emphasized monitoring and engagement also tended to have high rates of maintaining a balance of at least $1 in their escrow account during Month 36 and vice versa. Alternatively, values close to -1.000 suggest that housing agency program implementation features that were intended to increase service use were associated with low rates of escrow credit accrual. Finally, coefficients of between

-0.399 and +0.399 show little or no association between the implementation feature and escrow accrual Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 68

Chapter 4. Escrow Accruals and Graduation

outcome. As exhibit 4.4 shows, the correlations do not support the hypothesis that housing authorities that strongly emphasize monitoring and engagement are likely to have a relatively high incidence of escrow accrual or relatively high average balances after 3 years. In fact, the few measures with a moderately strong association with escrow outcomes show a negative correlation—probably (as discussed in chapter 3) because of the relatively high incidence of participants exiting from FSS in housing authorities with a stronger emphasis on monitoring and engagement. The bottom panel of exhibit 4.3 shows similar results. For all measures of escrow accrual, the housing agencies with the least emphasis on monitoring and engagement had the highest levels, on average, and the housing agencies with the strongest emphasis on monitoring and engagement had the lowest.

Who Appears to Be Benefiting Most From Enrol ment in Family Self-Sufficiency?

This section extends the analysis of subgroup variation in escrow accrual and explores the characteristics and experiences of participants who appear most likely to be benefiting from their access to FSS services and the escrow incentive.91 For this analysis, FSS group members are divided into five groups, based on their enrol ment status within the HCV and FSS programs in Month 36 and the balance in their escrow accounts, if any, at that time. FSS group members who maintained a balance of more than $2,000 in their escrow account in Month 36 are considered to be benefiting the most from their engagement in the program. Other groups who could also be benefiting include FSS group members who maintained a positive balance up to $2,000 in Month 36 and FSS group members who left the HCV program before Month 36.92 The remaining two groups—FSS group members who left the FSS program but continued to receive HCV subsidies and FSS

group members who remained enrol ed in FSS but have a $0 balance in their escrow account—are considered to be the least likely to have benefited from their enrol ment in the program through the end of Year 3.93

Exhibit 4.5 displays selected baseline characteristics of members of the five groups. As exhibit 4.5 shows, the two groups of FSS group members with a positive balance in their escrow account in Month 36 share some characteristics (associated with greater employability) that differ from those of FSS leavers who remained enrol ed in the HCV program and FSS enrol ees with no escrow balance. On average, FSS group members with an escrow balance are about 2 to 3 years younger, have slightly larger household sizes, are more likely to have children, and are more likely to have infants or preschool-age children than FSS group members in the two “less likely to benefit” outcome groups. FSS group members with a positive balance in Month 36 were also more likely to have been employed at random assignment. Overal , they were less likely to report experiencing a barrier to employment at random assignment, although the pattern of responses varies by type of barrier.

More specifically, FSS group members with a positive balance reported a higher incidence of problems with childcare, but a much lower incidence of problems with physical or mental health. FSS group members with a positive balance were also more likely to report receipt of Supplemental Nutrition Assistance Program benefits (food stamps) than FSS group members in the other outcome groups.

91

A later report will include findings of a more comprehensive analysis that will include additional followup on earning and disbursement of escrow credits and use formal predictive analytics methodology.

92

At least some HCV households leave assistance because their household income increases above the maximum level for receipt of a housing subsidy. This group also includes most families that graduate from the FSS program before 6 years. See Smith et al. (2015: 21). Using survey and administrative data, the authors compared the characteristics and experiences of participants in the Moving to Opportunity program evaluation who left housing assistance with those of participants who continued to receive assistance. The authors found that about 35 percent of study participants left housing assistance during the followup period. Among “leavers,” about 52 percent left for positive reasons, such as increased earnings or homeownership, whereas 48 percent left for negative reasons, such as eviction or violation of program rules. Leavers for positive reasons reported greater financial security and better housing conditions than respondents who remained on assistance, but those who left for negative reasons fared about the same as those still on assistance.

93

It is acknowledged that some FSS group members in these outcome groups could also have benefited from their receipt of FSS-related services—for example, by attaining an academic degree or occupational credential or by improving their credit score or ability to budget or save.

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 69

Chapter 4. Escrow Accruals and Graduation

Exhibit 4.5. Baseline Characteristics of the Family Self-Sufficiency Group, by Housing Choice Voucher and Family Self-Sufficiency Program and Escrow Accrual Statuses

Enrolled in Housing Choice Voucher Program

Enrolled in Enrolled in

Not

Enrolled in FSS with

FSS with

Enrolled

Not

FSS with Balance of Balance of

in HCV Enrolled in Balance of

$1–

$2,001 or

Characteristic

Program

FSS

$0

$2,000

More

Total

Household Characteristics

Average number of household

members

3.2

3.3

2.9

3.5

3.5

3.2***

a

Households with more than 1 adult

(%)

37.4

37.8

30.6

31.9

39.9

35.3

Average number of children in

household

1.7

1.8

1.5

2.1

1.9

1.8***

Number of children (%)

***

No children

22.2

24.7

34.6

10.9

18.5

22.8

1 child

27.6

22.6

21.6

27.2

23.3

24.2

2 children

24.4

23.8

18.7

29.3

25.8

24.1

3 children or more

25.8

28.9

25.1

32.6

32.4

28.9

For households with children, age of youngest child (%)

**

0–5 years

35.3

39.5

37.1

40.3

48.4

40.5

6–12 years

42.4

38.4

49.8

43.1

36.8

42.1

13–17 years

22.4

22.0

13.2

16.6

14.8

17.4

Primary language spoken at home

is English (%)

92.2

88.9

95.8

94.1

87.9

91.9***

Receives TANF (%)

10.5

15.4

17.6

18.2

18.8

16.3

Receives food stamps/SNAP (%)

62.4

68.5

69.2

78.7

76.4

71.2***

Length of time receiving Section 8 Housing Choice Voucher (%)

Less than 4 years

35.6

29.1

33.3

34.7

28.7

32.2

4–7 years

22.8

23.5

20.5

20.8

22.4

21.9

7 years or more

41.6

47.4

46.2

44.5

48.9

45.9

Total annual household income (%)

**

Less than $10,000

37.0

44.0

45.3

35.2

43.1

41.3

$10,000–$19,999

30.1

31.0

27.7

40.3

34.5

32.5

$20,000 or more

32.9

25.0

27.0

24.6

22.5

26.2

(continued)

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 70

Chapter 4. Escrow Accruals and Graduation

Exhibit 4.5. Baseline Characteristics of the Family Self-Sufficiency Group, by Housing Choice Voucher and Family Self-Sufficiency Program and Escrow Accrual Statuses continued Enrolled in Housing Choice Voucher Program

Enrolled in Enrolled in

Not

Enrolled in FSS with

FSS with

Enrolled

Not

FSS with Balance of Balance of

in HCV Enrolled in Balance of

$1–

$2,001 or

Characteristic

Program

FSS

$0

$2,000

More

Total

Payment for rent and utilities (%)

*

Less than $400

34.2

44.0

44.1

40.0

46.9

42.2

$400–$599

20.5

22.2

20.1

22.1

22.3

21.4

$600 or more

45.2

33.8

35.8

37.9

30.8

36.3

During the past 12 months,

household experienced at least

57.9

57.9

57.1

60.3

55.6

7.7

one financial hardship (%)

Not able to buy prescription

drug

11.9

14.5

14.4

13.1

11.0

13.0

Not able to buy food

24.7

26.4

26.2

30.0

23.8

26.2

Not able to pay telephone bil

26.0

25.5

26.5

32.5

32.6

28.7

Not able to pay rent

17.4

18.3

18.2

19.8

16.1

17.9

Not able to pay utility bil

42.5

41.3

43.8

43.5

46.2

43.5

Sample Member Characteristics

Female (%)

91.0

89.8

88.6

95.0

93.1

91.4*

Age (%)

***

19–34 years

38.5

31.1

33.0

38.9

37.8

35.7

35–44 years

38.0

35.7

29.2

44.8

37.5

36.6

45–61 years

23.5

33.2

37.8

16.3

24.7

27.7

Average age (years)

38.3

40.6

41.0

37.3

38.0

39.1***

Married, living with spouse or

cohabitating (%)

8.6

12.8

6.3

9.6

12.4

9.8*

Race/Ethnicity (%)

**

Black, non-Hispanic/Latino

75.6

72.3

75.9

74.1

63.6

72.2

Hispanic/Latino

14.0

13.2

12.7

16.3

21.5

15.6

Other

10.4

14.5

11.4

9.6

14.9

12.2

(continued)

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 71

Chapter 4. Escrow Accruals and Graduation

Exhibit 4.5. Baseline Characteristics of the Family Self-Sufficiency Group, by Housing Choice Voucher and Family Self-Sufficiency Program and Escrow Accrual Statuses continued Enrolled in Housing Choice Voucher Program

Enrolled in Enrolled in

Not

Enrolled in FSS with

FSS with

Enrolled

Not

FSS with Balance of Balance of

in HCV Enrolled in Balance of

$1–

$2,001 or

Characteristic

Program

FSS

$0

$2,000

More

Total

Education (%)

No high school diploma or

GED

12.3

15.0

14.0

13.0

13.5

13.6

High school diploma or GED

12.8

17.2

14.0

13.0

13.5

14.1

Some col ege or received

technical/trade license

56.2

51.5

54.0

56.9

51.5

53.9

2-year degree or higher

18.7

16.3

18.1

17.2

21.5

18.4

Has trade license or training

certificate

51.1

43.4

49.5

44.8

46.2

47.1

Employment Status

Currently employed (%)

60.0

51.9

46.2

67.5

55.7

55.6***

Current employment type (%)

***

Regular job

53.6

42.6

39.5

59.9

46.3

47.7

Self-employed

3.2

4.7

3.5

3.0

2.6

3.4

Temporary or seasonal job

3.2

4.7

3.2

4.6

6.6

4.5

Currently working 35 hours or

more per week (%)

37.3

22.6

26.4

33.8

27.1

29.1***

Average hours worked per week

20.9

16.0

15.4

21.5

16.8

17.9***

Average weekly earnings ($)

257

188

179

238

181

205***

Barriers to Employment (%)

Has any problem that limits work

34.8

47.7

47.6

36.8

38.9

41.6***

Physical, emotional, or mental

health

18.6

26.6

28.7

13.0

15.1

20.8***

Childcare access or cost

14.0

17.9

16.8

21.5

21.7

18.5

Other

11.4

18.2

16.6

11.8

12.4

14.2

Limited English-speaking ability

2.7

3.8

1.6

3.3

6.5

3.6**

No access to public

transportation

20.0

13.8

14.4

18.2

16.8

16.5

No access to an automobile

19.1

21.5

20.1

14.7

16.8

18.5

(continued)

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 72

Chapter 4. Escrow Accruals and Graduation

Exhibit 4.5. Baseline Characteristics of the Family Self-Sufficiency Group, by Housing Choice Voucher and Family Self-Sufficiency Program and Escrow Accrual Statuses continued Enrolled in Housing Choice Voucher Program

Enrolled in Enrolled in

Not

Enrolled in FSS with

FSS with

Enrolled

Not

FSS with Balance of Balance of

in HCV Enrolled in Balance of

$1–

$2,001 or

Characteristic

Program

FSS

$0

$2,000

More

Total

FSS Program (%)

Heard of escrow before random

assignment

45.5

45.5

44.6

39.7

44.2

43.9

Interest in FSS services related to

Job-related services

64.1

71.9

66.3

70.7

78.5

70.4***

Social services

25.0

38.3

32.7

27.6

36.0

32.2***

Financial services

98.2

95.3

97.1

95.0

91.6

95.4***

Sample Size

221

235

315

239

275

1,285

FSS = Family Self-Sufficiency. GED = General Educational Development. HCV = Housing Choice Voucher. SNAP = Supplemental Nutrition Assistance Program. TANF = Temporary Assistance for Needy Families.

aMaximum response option for number of adults in a household is four.

NOTES: The FSS impact sample includes HCV households that were randomly assigned between October 18, 2013, and December 22, 2014, and with a head of household age 18 to 61 at the time of random assignment. Sample sizes for specific measures may vary because of missing values. Rounding may cause slight discrepancies in calculating sums. Detail may sum to more than total for questions that allow more than one response.

SOURCES: MDRC calculations from Baseline Information Form data and housing agency administrative data.

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 73

Chapter 4. Escrow Accruals and Graduation

FSS group members with a balance of $2,001 or more in Month 36 are further distinguished by having relatively low income at random assignment (they had the smal est proportion of FSS group members with a household income of $20,000 or higher) and reporting the least out-of-pocket housing costs (they had the smal est proportion of FSS group members with housing-related expenses of $600 or more per month).

FSS group members with a balance of $1 to $2,000 in their escrow account reported the highest level of employment at random assignment, although about one-half of these FSS group members reported working part-time hours.

FSS group members who left HCV assistance after random assignment were similar in age to members with a positive escrow balance and in having a higher reported employment rate and lower incidence of reporting a barrier to employment compared with the two “less likely to benefit” groups.

As a group, FSS group members who left HCV assistance after random assignment appear to have entered the study in better financial condition than other FSS group members. HCV leavers are distinguished by having the highest incidence of ful -time employment at random assignment among all outcome groups, the lowest incidence of receipt of SNAP benefits, the highest incidence of reporting $20,000 or more in household income, and the highest incidence of paying $600 or more per month for housing-related expenses.

Exhibit 4.6 looks at indicators of service use and contacts with FSS case managers for the same five groups.

The exhibit shows a relatively clear relationship between the level of engagement with FSS and the amount of FSS group members’ escrow balance in Month 36. As would be expected, the two groups of FSS group members who exited from the program (HCV leavers and FSS-only leavers) had the lowest incidence of use of FSS-related services among the five outcome groups, especially during Year 3 of fol owup.94 In contrast, FSS group members with a balance of more than $2,000 in Month 36 had the largest proportion of FSS

group members who used FSS-related services for 6 months or more during Years 1 to 3 and also the highest incidence of service use (54 percent) and contacts (80 percent, with rounding) with FSS case managers during Year 3. FSS group members who continued their enrol ment in the program but had no balance in their escrow account at the end of fol owup had much lower levels of engagement in Year 3. A little more than one-half of the group had at least one contact with their FSS case manager, and a slightly smal er proportion (47

percent) used at least one FSS-related service. FSS group members with a balance of $1 to $2,000 in Month 36

recorded levels of service use and contacts in Year 3 that were between those for the groups with a zero balance or with a balance of $2,001 or more.

94

Nonetheless, most FSS group members in both groups (62 percent and 76 percent, respectively) participated in at least one FSS-related activity during Years 1 to 3.

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 74

Chapter 4. Escrow Accruals and Graduation

Exhibit 4.6. Selected Indicators of Contacts and Service Use During Years 1 to 3, by Housing Choice Voucher and Family Self-Sufficiency Enrollment Status and Escrow Balance Enrolled in Housing Choice

Voucher Program

Enrolled Enrolled in Enrolled in

Left

Not

in FSS

FSS with

FSS with

with

Balance of Balance of

HCV

Enrolled Balance

$1–

$2,001 or

Outcome

Program

in FSS

of $0

$2,000

More

Total

Years 1 to 3

Services and Employment (%)

Used FSS-related services

61.7

76.4

86.5

85.8

85.6

80.4***

Job search or post-employment

services

20.9

37.0

44.4

42.5

37.9

37.4***

Financial counseling

27.8

38.6

58.4

48.3

46.0

45.2***

Education or training

13.0

29.1

28.1

45.8

41.4

32.1***

Homeownership preparation

10.4

11.8

19.1

15.8

19.0

15.8

Health coverage or health

assistance

0.0

7.9

8.4

5.8

10.3

7.0**

Social services

27.0

30.7

39.9

27.5

44.8

35.3***

Supportive services

10.4

20.5

22.5

25.8

26.4

21.7**

Had program-recorded employment

60.0

65.4

64.0

93.3

92.0

75.4***

Used FSS-related services for 6

months or more

13.0

24.4

33.7

40.8

43.7

32.4***

Year 3

Contacts With Case Managers

Had 1 contact or more (%)

5.2

27.6

52.8

70.0

80.5

50.3***

Had contact during 3 months or

more (%)

1.7

8.7

21.9

25.0

36.2

20.3***

Average number of months with

contacts

0.1

0.6

1.5

2.0

2.5

1.5***

Service Use and Employment (%)

***

Service-use and program-recorded

employment

3.5

11.0

29.2

40.8

47.7

28.3

Service use only

0.0

3.1

17.4

9.2

6.3

8.0

Program-recorded employment only

6.1

17.3

24.7

41.7

37.9

26.5

Neither service use nor program-

recorded employment

90.4

68.5

28.7

8.3

8.0

37.3

Sample size (total = 714)

115

127

178

120

174

(continued)

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 75

Chapter 4. Escrow Accruals and Graduation

Exhibit 4.6. Selected Indicators of Contacts and Service Use During Years 1 to 3, by Housing Choice Voucher and Family Self-Sufficiency Enrollment Status and Escrow Balance continued FSS = Family Self-Sufficiency. HCV = Housing Choice Voucher.

NOTES: The FSS service-use sample includes all or a randomly selected subsample of HCV heads of household who were randomly assigned to the FSS group between October 18, 2013, and December 22, 2014 in 12 housing agencies, and were age 18 to 61 at the time of random assignment. Rounding may cause slight discrepancies in calculating sums and differences. A chi-square test was performed on subgroup differences in frequency distributions and an F-test was performed on subgroup differences in means. Statistical significance levels are indicated as: * = 10 percent; ** = 5 percent; *** = 1 percent.

SOURCES: MDRC calculations using housing authority administrative data.

It is also noteworthy that for both groups with a positive escrow balance in Month 36, a relatively large proportion of FSS group members (around 40 percent for each group) had program-recorded employment but no use of FSS-related services during Year 3. Many of these FSS group members participated in FSS-related activities before Year 3 and were maintaining at least minimal contact with their FSS case manager during Year 3.

Exhibit 4.7 shows the average change in employment and earnings over 3 years for each of the five groups, using data reported to HUD’s Inventory Management System/PIH (Office of Public and Indian Housing) Information Center (IMS/PIC) system. For these outcomes, FSS group members with a balance of $2,001

or more clearly stand out. On average, these group members increased their reported estimated annual earnings by more than $9,000, more than twice the increase of any other group. About two-thirds of the group members experienced an increase in earnings over 3 years, a rate that also exceeded the increase for al other groups. Only one other group, FSS group members with an escrow balance of between $1 and $2,000, had more than half of its members report an increase in earnings over time. An important reason for these positive results for the group with a balance that exceeded $2,000 was the relatively large proportion of group members (34 percent) who found employment during the fol owup period. Members of the other two groups considered to be benefiting as well from their involvement in the FSS program had the highest incidence of employment around the time of random assignment and at the end of fol owup, whereas members of the two groups deemed least likely to be benefiting from FSS recorded the highest incidence of joblessness at both points in time.

For the most part, these patterns of outcomes are similar when earnings from other household members are included in the totals. The main exception concerns households that left the FSS program but continued to receive HCV assistance. FSS-group households in this group averaged an increase in earnings over time of about $5,400, an amount that exceeded the average increase for FSS-group households with a balance of from $1 to $2,000.

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 76

Chapter 4. Escrow Accruals and Graduation

Exhibit 4.7. Change in Reported Annual Head-of-Household Earnings and Total Household Earnings During Years 1 to 3, by Housing Choice Voucher and Family Self-Sufficiency Enrollment Status and Escrow Balance in Month 36

Enrolled in Housing Choice

Voucher Program

Enrolled in Enrolled in

Not

Enrolled in FSS with

FSS with

FSS with Balance of Balance of

Left HCV Enrolled Balance of

$1–

$2,001 or

Outcome

Program in FSS

$0

$2,000

More

Total

Head-of-Household Earnings

Average earnings in Month 1 ($)

11,941

8,226

8,779

10,417

7,628

9,288***

Average increase in estimated annual

earnings ($)

4,163

3,857

1,226

3,300

9,199

4,335***

Change in earnings (%)

***

Decrease

19.1

17.7

22.2

27.6

16.7

20.7

No change

36.7

36.7

46.8

20.7

17.1

31.9

Increase

44.2

45.6

31.0

51.7

66.2

47.5

Head of Household Employment (%)

***

In Month 1 and in current or most

recent month

50.2

33.2

30.6

47.8

38.3

39.4

In Month 1 only

8.4

10.2

10.8

15.1

11.2

11.1

In current or most recent month only

13.5

19.9

12.5

16.4

33.5

19.3

No employment at either time

27.9

36.7

46.1

20.7

17.1

30.2

Total Household Earnings

Average earnings in Month 1 ($)

13,216

10,360

10,143

11,744

9,917

10,970**

Average increase in estimated annual

earnings ($)

6,238

5,355

2,558

4,809

12,274

6,223***

Change in earnings (%)

***

Decrease

17.1

20.4

22.9

26.7

17.3

20.9

No change

32.9

27.9

41.1

12.9

11.7

25.6

Increase

50.0

51.8

36.0

60.3

71.1

53.4

Household Employment (%)

***

In Month 1 and in current or most

recent month

52.8

38.9

35.7

54.3

46.6

45.1

In Month 1 only

7.9

11.1

10.1

13.8

10.5

10.7

In current or most recent month only

15.7

22.1

13.8

19.0

31.2

20.4

No employment at either time

23.6

27.9

40.4

12.9

11.7

23.8

Sample size

221

235

315

239

275

1,285

(continued)

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 77

Chapter 4. Escrow Accruals and Graduation

Exhibit 4.7. Change in Reported Annual Head-of-Household Earnings and Total Household Earnings During Years 1 to 3, by Housing Choice Voucher and Family Self-Sufficiency Enrollment Status and Escrow Balance in Month 36 continued

FSS = Family Self-Sufficiency. HCV = Housing Choice Voucher.

NOTES: The FSS impact sample includes HCV households that were randomly assigned between October 18, 2013, and December 22, 2014, and with a head of household age 18 to 61 at the time of random assignment. Calculations of change in annual earnings use the most recent available estimate reported to HUD, which could occur prior to or after Month 36 of follow-up, depending on each study participant’s date of random assignment, HCV program status, and, if applicable, date of exit from the HCV program. Rounding may cause slight discrepancies in calculating sums and differences. Sample sizes for specific outcomes may vary because of missing values. A chi-square test was performed on subgroup differences in frequency distributions and an F-test was performed on subgroup differences in means. Statistical significance levels are indicated as: * = 10 percent; ** = 5 percent; *** = 1 percent.

SOURCES: MDRC calculations using housing authority administrative data and HUD Inventory Management System/Office of Public and Indian Housing Information Center data.

Conclusions

As suggested by existing evidence and the year-by-year decline in the number of newly opened escrow accounts by FSS group members, it appears likely that few additional FSS group members will begin accruing escrow credits during the remaining 2 years of their FSS contract.95 (One exception could be FSS group members who complete education and training activities and then find jobs or increase their hours of employment.) FSS group members who entered the study without employment include participants with serious barriers to employment or who need to care for other family members. In addition, under current economic conditions, FSS group members who were working full time at program enrol ment may have difficulty finding better paying jobs and may have little chance of receiving a raise or promotion in the near term. Finally, FSS group members who worked at relatively wel -paying jobs face limits in how much more they can earn before losing their eligibility to receive a housing subsidy and to continue in the FSS program. Stil , it should be expected that perhaps one-fourth to one-third of FSS group members will graduate and receive a disbursement of at least $5,000 and possibly much higher. Over the final 2 years of the FSS contract period, the prospect of imminent graduation and escrow disbursement may provide a powerful incentive to these FSS participants to maintain their employment and to stay sufficiently engaged in the FSS program to qualify for graduation at the end of Year 5. Their continued employment may result in gains in average employment and earnings for the entire FSS group, compared with the control group—results that may not be seen until after Year 3.

Possibly, many FSS group members will also not graduate from the program after 5 years or will graduate without accruing escrow. As discussed in a previous report on FSS, some case managers assert that FSS

services can help participants find jobs and advance in their careers even when participants realize that receiving an escrow disbursement is unlikely. By facilitating participants’ attainment of an educational or occupational credential, higher credit scores, or even greater self-confidence, the FSS program may lead to better employment and earning outcomes for nongraduates. (Chapters 5 and 7 consider the effects of the FSS

program on employment and earnings during Years 1 to 3 of fol owup, for the full sample and for selected subgroups, respectively.) With nearly 40 percent of the FSS group already having left the program by the end of Year 3, however, the number of FSS group members who may derive future benefits from their access to services and case management is limited.

95

See Verma et al. (2017) on new earners of escrow credits in later years (pages 40, 42-43) and figure 3.2 (p. 53).

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 78

Chapter 5. Impacts on Employment and Earnings

Chapter 5. Impacts on Employment

and Earnings

The Family Self-Sufficiency program is meant to encourage participants to find a steady job or, if they are already working, to increase their earnings. As described in previous chapters, staff members provide referrals to a range of supportive services designed to help participants move into work or pursue training, including adult basic education courses. FSS participants who go to work or increase their earnings pay more rent, but the housing agency credits the household’s escrow account based on the increased rent attributable to in the earned income, which they can then get back after graduating from FSS. The escrow account serves as both an asset-building instrument and a work incentive. Stil , the escrow account represents a potentially distant work incentive, in that participants do not generally have access to their escrow funds for 5 years, or until they graduate from the program. This chapter focuses on the program’s effects on employment and earnings during the first 3 years (36 months) after program enrol ment, recognizing that participants could still be pursuing a wide range of goals during this period. It builds on the 2-year results in the earlier report on this evaluation.

The early data revealed high levels of employment for both study groups, but no notable differences in earnings or employment outcomes for the FSS and control groups. There was some evidence that participation in FSS

may have led to a shift to ful -time from part-time employment at the time of the 18-month survey. This longer term analysis uses data from administrative records and the 36-month survey, which provides information about job characteristics, to determine whether the FSS program group experienced a higher employment rate or earned more on average than members of the control group (chapter 7 examines variations in the effects by subgroup). Focusing on the full sample, chapter 5 considers whether, over a longer fol owup period, the FSS

group experienced a higher employment rate or earned more on average than members of the control group.

In brief, for the full impact sample:

• Analysis of National Directory of New Hires (NDNH) quarterly wage data reveal high levels of employment for both study groups and no statistically significant impacts on employment and earnings during the 3-year fol owup period.

• FSS and control group respondents to the FSS 36-month survey reported similar incidences of employment during the fol owup period. Among survey respondents, however, FSS group members reported working an additional 2–3 weeks in a ful -time job compared with the control group, during the 12 months before the survey.

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 79

Chapter 5. Impacts on Employment and Earnings

Data Sources and Methods

Program impacts in the FSS study are estimated using administrative records and survey responses. Earnings and employment impact estimates use NDNH quarterly wage data and responses to the 36-month survey.96

NDNH data provide quarterly earnings and are available for the impact sample for two quarters before and 13 quarters after the quarter of random assignment.97 NDNH records provide data on employment and earnings in all work covered by unemployment insurance, including across state lines (for those who commute into another state for work or who moved to a different state after random assignment) and on federal employment not captured in state unemployment insurance records. The records do not cover earnings from self-employment, some agricultural work, and informal jobs. Other research suggests that administrative data may miss relatively more employment for low-income populations than for higher income groups, given the former group’s greater prevalence of work in informal jobs (Abraham et al., 2009). NDNH records also do not provide information about the hours worked during a quarter or week or on the characteristics of jobs held, such as hourly wage rates, benefits, and schedule. For this reason, the evaluation draws on the 36-month survey to examine study participants’ job characteristics. Responses to the 36-month survey may also pick up employment in jobs not covered by NDNH records.

Exhibit 5.1 includes an explanation of how to read the impact tables in this report. The study design uses random assignment to create the FSS and control groups. Therefore, effects or impacts of the FSS program can be calculated as the difference in average outcomes between the research groups. Differences that are statistically significant (indicated by asterisks in the exhibits) are considered to be true program effects and not the result of chance.98 The effects of the program are presented for the FSS impact sample, which excludes voucher holders age 62 or older at the time of random assignment, and for the subgroups included in the analyses of service use in the previous chapters. The essential research question for subgroup analysis is whether the differences in impacts across subgroups are statistically significant. (Subgroup differences that are statistically significant are noted with daggers in the exhibits.)

96

For a description of the variables included in the presented models and for analyses of the sensitivity of results to outliers and to different data-weighting approaches, see appendix C.

97

Employment recorded during the quarter of random assignment may have occurred before the study participant’s date of random assignment. Accordingly, the analysis excludes this quarter from the followup period.

98

An exception to this statement concerns a situation in which only one comparison among a series of related comparisons shows a statistically significant difference between the research groups—for example, if FSS group members averaged higher earnings than control group members during only one quarter of followup. In this situation, less credence would be given to this single impact estimate, even if the difference were statistically significant.

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 80

Chapter 5. Impacts on Employment and Earnings

Exhibit 5.1. How to Read the Impact Tables in This Report

In the context of this evaluation, an “impact” is a measure of how much the intervention—Family Self-Sufficiency (FSS)—changed outcomes for program participants. The group outcome for the intervention is compared with that of the control group. The top row of the excerpted table below, for example, shows that 26 percent of the FSS group was working part time at the time of the 18-month survey, compared with 29

percent of the control group.

Because participants were assigned randomly to either the program group or the control group, the effects of the program can be estimated by the difference in outcomes between the two groups. The “Difference”

column in the table excerpt shows the differences between the two research groups’ outcomes—that is, the program’s estimated impacts on the outcomes. For example, the estimated program impact of the FSS

program on the number of individuals working part-time study can be calculated by subtracting 29 percent from 26 percent, yielding a decrease, or estimated impact, of 3 percentage points.

The p-value shows the probability that this difference, or impact, arose by chance. In the table excerpt below, the difference between the program and control groups in current part-time employment has a 16.6 percent probability of arising as a result of chance rather than as a result of the FSS-only program. In contrast, the difference on the measure current ful -time employment has a 4.5 percent probability of having arisen by chance. For this evaluation, only differences that have a 10 percent probability or less of arising by chance are considered “statistically significant” and therefore represent true program effects. The number of asterisks indicates whether the impact is statistically significant at the 1 percent (***), 5 percent (**), or 10 percent (*) level, meaning that there is only a 1,5, or 10 percent probability, respectively, that the impact arose by chance.

Impacts on Employment

Family Self-Sufficiency 18-Month Survey Respondent Sample

Outcome

FSS Group Control Group Difference (Impact)

P-Value

Currently employed (%)

Works part-time hours

26.0

29.0

-3.0

0.166

Works ful -time hours

41.5

37.1

4.3

0.045**

FSS = Family Self-Sufficiency.

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 81

Chapter 5. Impacts on Employment and Earnings

Program Impacts on Employment and Earnings

Employment and Earnings Trends for Control Group Members

Exhibits 5.2 and 5.3 and appendix exhibit C.1 display the quarterly employment and earnings trends for control group members, calculated with NDNH data. These averages, which reflect what would have happened in the absence of the program, provide the basis of comparison for estimating the impacts of the FSS program on employment and earnings. As shown by the solid line in exhibit 5.2 employment levels for control group members overall increased slightly over time, from about 58 percent in the second quarter before random assignment to around 63 percent in Quarter 13 fol owing random assignment. Control group members who reported on the BIF that they were not working (see the dotted line) experienced the biggest increase during the fol owup period, of more than 16 percentage points, whereas employment rates remained relatively static for other control group members.

As exhibit 5.3 shows, on average, control group members earned more over time—from $2,683 per quarter (including zeros for control group members without employment) in the second quarter before random assignment to $3,493 in Quarter 13 after random assignment. This increase was a result of some control group members entering employment and other control group members increasing their hours or weeks of employment or earning more on the job. The average quarterly earnings also increased for control group members in all three subgroups based on members’ self-reported employment at random assignment.

Once again, control group members in the subgroup that reported no employment at random assignment experienced the biggest increase during these quarters.

Impacts on Employment and Earnings

Though the FSS program is designed to provide services and referrals to address a variety of difficulties faced by participants, including lack of adequate childcare, mental and physical health issues, and transportation chal enges, this component is in service to the ultimate goal of the program: to move people to work and build economic self-sufficiency. The escrow incentive is designed with the same goal and is thus attached to increases in earnings. Therefore, an important test of the FSS model is an assessment of its ability to increase employment, employment quality, and earnings. Exhibit 5.5 presents some staff members’ reflections on the strategies they use to support both working and not-working FSS participants.

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 82

Chapter 5. Impacts on Employment and Earnings

Exhibit 5.2. Quarterly Employment Rate Among Control Group Members, by Self-Reported Employment Status at Random Assignment, Family Self-Sufficiency Impact Sample 100%

90%

80%

70%

60%

50%

40%

Employment Rate

30%

20%

10%

0

Pre-Q2 Pre-Q1

Q1

Q2

Q3

Q4

Q5

Q6

Q7

Q8

Q9

Q10

Q11

Q12

Q13

Quarter Relative to Random Assignment

Not employed

Part Time (1-34 hours)

Full Time (35+ hours)

Total

FSS = Family Self-Sufficiency.

NOTE: The FSS impact sample includes Housing Choice Voucher households that were randomly assigned between October 18, 2013, and December 22, 2014, and with a head of household age 18 to 61 at the time of random assignment.

SOURCE: MDRC calculations using baseline data and quarterly wage data from the National Directory of New Hires.

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 83

Chapter 5. Impacts on Employment and Earnings

Exhibit 5.3. Average Quarterly Earnings Among Control Group Members, by Self-Reported Employment Status at Random Assignment, Family Self-Sufficiency Impact Sample 7,000

6,000

5,000

4,000

3,000

Earnings ($)

2,000

1,000

0

Pre-Q2 Pre-Q1

Q1

Q2

Q3

Q4

Q5

Q6

Q7

Q8

Q9

Q10

Q11

Q12

Q13

Quarter Relative to Random Assignment

Not employed

Part Time (1-34 hours)

Full Time (35+ hours)

Total

NOTES: The Family Self-Sufficiency impact sample includes Housing Choice Voucher households that were randomly assigned between October 18, 2013, and December 22, 2014, and with a head of household age 18 to 61 at the time of random assignment.

SOURCE: MDRC calculations using quarterly wage data from the National Directory of New Hires.

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 84

Chapter 5. Impacts on Employment and Earnings

The bottom half of exhibit 5.4 shows program impacts on yearly earnings, as measured by the NDNH data.

The control group averaged $12,144 over the first year of fol owup. This exhibit rose to $15,300 in the third year of fol owup. Over this 3-year fol owup period, members of both groups averaged a bit more than $41,000

in earnings, which translates into yearly earnings of nearly $14,000.99

Overal , after 12 quarters, more than halfway through the 5-year FSS program duration, no evidence exists that FSS has led to changes in quarterly earnings or consistent impacts in employment covered by NDNH.

To determine whether the program affected job characteristics and outcomes that cannot be observed from NDNH data, the evaluation also considers self-reported data col ected from respondents to the FSS 36-month survey.

Exhibit 5.6 shows responses to a set of questions in the FSS 36-month survey that are focused on current employment and employment in the 12 months before the survey interview. Survey respondents were asked if they were currently employed, if they were looking for work, and what were the characteristics of their jobs in the prior 12 months. The top panel of the exhibit shows comparable recent work history for the two groups.

Roughly 78 percent of the FSS group worked during the prior 12 months, compared with about 76 percent of the control group. About one-half of those in the FSS group worked for 12 months in the prior year, compared with 48 percent of the control group. Both groups also reported working similar numbers of jobs. According to survey responses, however, FSS group members averaged slightly more months of ful -time employment in the prior 12 months. FSS group members reported working full time close to 6 months during this period, compared with 5 months for the control group, a difference that is statistically significant. FSS group members were more likely to report they were looking for ful -time work, compared with their control group counterparts (31.5 percent versus 27.0 percent).

Using the FSS 36-month survey, the research team also explored whether the program had impacts on current work in terms of the types of jobs, the hourly wages, and the hours worked. For those who were not employed at the time of the survey, their reasons for not working were explored. As these outcomes are not covered in NDNH records, the survey is the only source of this information. These results are displayed in exhibit 5.7.

The analysis provides some important insights into the types of jobs held by FSS participants and their control group counterparts.

99

Using quasi-experimental methodology and a matched comparison group, an analysis of the Compass Working Capital FSS program, administered by the nonprofit agency for the PHAs in Lynn and Cambridge, Massachusetts, shows that the program produced employment and earnings impacts for participants. The study, which used income data available to HUD, found that the Compass FSS program was associated with an average gain in annual household earnings of $6,305 between the fourth quarter of 2010 and the first quarter of 2016

(Geyer et al., 2017).

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 85

Chapter 5. Impacts on Employment and Earnings

Exhibit 5.4. Impacts on Employment and Earnings, Years 1 to 3, Family Self-Sufficiency Impact Sample

FSS

Control Difference

Outcomes

Group

Group

(Impact)

p-Value

Employment (%)

Ever employed

Year 1

73.4

72.5

0.9

0.496

Year 2

74.8

72.8

1.9

0.182

Year 3

74.2

72.5

1.7

0.257

Years 1 through 3

83.4

81.7

1.7

0.170

Average quarterly employment rate

Year 1

61.6

61.4

0.2

0.887

Year 2

64.0

63.7

0.3

0.847

Year 3

64.9

63.9

1.0

0.502

Years 1 through 3

63.5

63.0

0.5

0.679

Earnings ($)

Total earnings

Year 1

11,967

12,144

-177

0.596

Year 2

14,178

14,043

135

0.757

Year 3

15,505

15,300

204

0.685

Years 1 through 3

41,650

41,487

163

0.884

Average annual earnings (%)

0.697

$0

16.6

18.3

-1.7

$1–$10,000

31.6

31.2

0.3

$10,001–$25,000

31.6

30.4

1.2

Greater than $25,000

20.3

20.1

0.2

Sample size (total = 2,548)

1,282

1,266

FSS = Family Self-Sufficiency.

NOTES: The FSS impact sample includes Housing Choice Voucher households that were randomly assigned between October 18, 2013, and December 22, 2014, and with a head of household age 18 to 61 at the time of random assignment. Average quarterly employment rate is calculated as total quarters with employment divided by total quarters of follow-up, expressed as a percentage.

Estimates were regression-adjusted using ordinary least squares, controlling for pre-random assignment characteristics of sample members. Rounding may cause slight discrepancies in calculating sums and differences. A two-tailed t-test was applied to differences between research groups. The p-value indicates the likelihood that the difference between the FSS group and control group arose by chance. Statistical significance levels are indicated as: * = 10 percent; ** = 5 percent; and *** = 1 percent.

SOURCE: MDRC calculations using quarterly wage data from the National Directory of New Hires.

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 86

Chapter 5. Impacts on Employment and Earnings

Exhibit 5.5. Strategies to Support Participants’ Work Efforts: Family Self-Sufficiency Program Staff Members’ Reflections

Staff cited the need to conduct a “deeper dive” to better understand how to support participants with their employment needs. They cited several reasons why participants were not working: lack of relevant job skil s, education, and work experience relative to the local job market; unmet childcare and transportation needs; need for improvement in soft skil s in the workplace; and pressing family, mental health, and life skill issues.

They described a variety of strategies for working with unemployed clients who may need more support.

These include:

• Requiring unemployed participants to meet with their case managers more frequently, usually monthly and preferably by face-to-face meetings, either at the public housing agency office, the client’s home, or a venue in the community, such as at a café.

• Helping clients develop a sense of self-motivation and a strong work ethic and helping them assess the direction of their lives—where they are currently, where they want to go.

• Sending job leads and listings to their participants. Sometimes these leads are tailored for a specific client, but often these leads are not customized.

• Reviewing participants’ resumes and offering recommendations on how to improve them to better align with their job search objectives in the local labor market.

• Referring job seekers to community partners or job developers that specialize in providing job search assistance, as well as to special events such as job fairs.

• Requiring them to attend mandatory job search workshops.

• Providing a limited number of participant stipends, especially for transportation needs. These stipends took the form of bus tickets and gas cards.

While most sites indicated that some of their participants experience rapid job turnover, only staff at one site stated that this development was a big issue for their program; rapid job turnover was exacerbated at this site by the number of clients who obtained their jobs through temp agencies.

Supervisors and case managers in a number of sites reported that many of their employed participants were seeking better jobs, which staff defined as having higher pay (which would also help them build escrow at a faster rate), being ful -time positions, or being in their field of interest. Staff in most of the other sites said that the number of working clients who wanted better jobs varied by circumstance.

• Some participants were content with their current positions because they were employed in their field of choice or had a schedule that permitted them the time to work on other FSS goals, such as going to school or improving their credit rating, or to attend to their family or household responsibilities.

• For clients who wanted to find better jobs, case managers helped them work toward this goal in several ways. Staff at one site reviewed and critiqued participants’ resumes, while staff at another site helped participants assess their job search objectives and aspirations.

FSS = Family Self-Sufficiency.

SOURCE: 2018 field research interviews with program staff in study sites.

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 87

Chapter 5. Impacts on Employment and Earnings

Exhibit 5.6. Impacts on Employment in the 12 Months Prior to Interview, Family Self-Sufficiency 36-Month Survey Respondent Sample

FSS Control Difference

Outcome

Group Group

(Impact)

p-Value

In the 12 Months Prior to Interview

Ever employed (%)

78.0

76.4

1.6

0.342

Average number of jobs

1.1

1.1

0.0

0.860

Number of jobs (%)

0.717

Not employed

22.1

23.6

-1.5

1

53.2

51.5

1.7

2

18.0

17.4

0.6

3 or more

6.8

7.5

-0.8

Average number of months with employment

7.9

7.6

0.3

0.150

Number of months with employment (%)

0.865

Not employed

22.1

24.0

-1.9

1–6

13.6

14.0

-0.4

7–11

14.0

13.9

0.1

12

50.3

48.1

2.2

Average number of months with mostly ful -time

employment

5.6

5.0

0.6**

0.014

Number of months with mostly ful -time employment (%)

0.452

Not employed

47.4

52.4

-5.0

1–6

6.6

6.5

0.2

7–11

8.8

7.9

0.9

12

37.2

33.2

3.9

In the Month Prior to Interview (%)

Employed

66.5

65.6

1.0

0.629

Looked for a job

36.8

36.2

0.7

0.759

Looked for a ful -time job

30.5

26.8

3.7*

0.070

Employed and looked for a job

21.8

21.9

-0.1

0.947

Not employed and looked for a job

15.1

14.3

0.9

0.590

Sample size (total = 1,936)

988

948

(continued)

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 88

Chapter 5. Impacts on Employment and Earnings

Exhibit 5.6. Impacts on Employment in the 12 Months Prior to Interview, Family Self-Sufficiency 36-Month Survey Respondent Sample continued

FSS = Family Self-Sufficiency.

NOTES: The FSS 36-Month Survey respondent sample includes Housing Choice Voucher heads of household who were randomly assigned between October 18, 2013, and December 22, 2014, were age 18 to 61 at the time of random assignment, and responded to the FSS 36-Month Survey. Sample sizes for specific outcomes may vary because of missing values. Estimates were regression-adjusted using ordinary least squares, controlling for differences in sample member characteristics recorded at the time of random assignment.

No special weights were applied to responses to adjust for differences in sample size by housing authority. Rounding may cause slight discrepancies in calculating sums and differences. A two-tailed t-test was applied to differences between the FSS group and the control group for continuous variables and selected outcomes expressed as proportions. In addition, a chi-square test for categorical variables was run to determine whether there is a difference in the distribution of related outcomes by research group. The p-value indicates the likelihood that the difference between the FSS group and control group arose by chance. Statistical significance levels are indicated as:

* = 10 percent; ** = 5 percent; *** = 1 percent.

SOURCE: MDRC calculations using responses to the FSS 36-Month Survey.

For example, most respondents in each research group who were working at the time of their interview reported working in a regular job (self-employment or employment in temporary or seasonal jobs was rare). The largest proportion of respondents worked ful -time hours and worked the regular daytime shift.

Respondents tended to work at relatively low-paying jobs with few or no benefits. For example, only a little more than 10 percent of respondents in each research group reported working at a job with weekly earnings equivalent to 130 percent of the federal poverty threshold and with paid vacation and sick days and access to medical coverage.

Few notable differences are evident across the range of jobs characteristics presented in this exhibit. Members in both the FSS and control groups appear to be in jobs with similar characteristics. As exhibit 5.6 shows, FSS

group respondents reported working slightly more hours per week, an increase of about one hour (a statistically significant difference). FSS group respondents also reported earning nearly $20 (or 7 percent) more per week than their counterparts in the control group, although this difference is slightly above the 10 percent level of statistical significance (p-value = 0.113).

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 89

Chapter 5. Impacts on Employment and Earnings

Exhibit 5.7. Impacts on Current Employment and Job Characteristics, Family

Self-Sufficiency 36-Month Survey Respondent Sample

FSS

Control Difference

Outcome

Group

Group

(Impact)

p-Value

Current Employment

Employed (%)

65.5

63.5

2.0

0.331

Job type (%)

0.962

Employee

54.6

53.5

1.1

Self-employed

6.2

6.0

0.3

Temporary or seasonal job

4.1

4.0

0.1

Average hours of work per week

22.7

21.3

1.3*

0.083

Hours of work (%)

0.315

1–20

9.6

11.9

-2.3

21–34

14.2

14.2

0.0

35 or more

41.6

37.3

4.3

Average hourly wage ($)

8.51

8.05

0.46

0.155

Hourly wage (%)

0.934

$0.01–$8.99

12.4

12.3

0.1

$9.00–$11.99

16.9

15.9

1.1

$12.00–$14.99

12.3

11.5

0.8

$15.00–$19.99

14.9

14.6

0.3

$20 or higher

7.2

6.8

0.3

Average weekly earnings ($)

291

273

19

0.113

Weekly earnings (%)

0.718

$1–$199

7.7

8.9

-1.2

$200–$399

19.1

19.0

0.1

$400–$599

19.6

17.4

2.2

$600 or higher

17.2

16.0

1.2

Usual work schedule (%)

0.806

Regular daytime shift

41.2

39.1

2.1

Regular evening or night shift

8.2

8.0

0.3

Rotating or split shift

7.9

9.1

-1.1

Irregular shift

5.1

4.6

0.5

(continued)

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 90

Chapter 5. Impacts on Employment and Earnings

Exhibit 5.7. Impacts on Current Employment and Job Characteristics, Family

Self-Sufficiency 36-Month Survey Respondent Sample continued

FSS

Control Difference

Outcome

Group

Group

(Impact)

p-Value

Employer-provided benefits (%)

Paid sick days

32.4

33.2

-0.8

0.706

Paid vacation days

34.2

33.2

1.0

0.613

Paid overtime

37.5

35.1

2.4

0.249

A retirement plan

31.1

31.9

-0.8

0.697

A health or medical insurance plan offered

39.4

36.7

2.7

0.171

With paid vacation and sick days and medical

plan offered

11.8

13.2

-1.4

0.309

Earnings at 130 percent of federal poverty threshold

or higher (%)

23.0

22.2

0.8

0.665

If currently not employed, main reason

Respondent’s il ness or disability

36.0

42.2

-6.2

No jobs available

17.7

21.4

-3.7

In school or training

8.2

6.5

1.7

Il ness or disability, other household member

8.2

6.2

2.0

Temporarily laid off

6.4

6.8

-0.4

Taking care of household member

4.8

3.3

1.5

Unable to find childcare

4.0

4.2

-0.2

Other

14.6

9.4

5.2

Sample size (total = 1,936)

988

948

FSS = Family Self-Sufficiency.

NOTES: The FSS 36-Month Survey respondent sample includes Housing Choice Voucher heads of household who were randomly assigned between October 18, 2013, and December 22, 2014, were age 18 to 61 at the time of random assignment, and responded to the FSS 36-Month Survey. Sample sizes for specific outcomes may vary because of missing values. Estimates were regression-adjusted using ordinary least squares, controlling for differences in sample member characteristics recorded at the time of random assignment.

No special weights were applied to responses to adjust for differences in sample size by housing authority. Rounding may cause slight discrepancies in calculating sums and differences. A two-tailed t-test was applied to differences between the FSS group and the control group for continuous variables and selected outcomes expressed as proportions. In addition, a chi-square test for categorical variables was run to determine whether there is a difference in the distribution of related outcomes by research group. The p-value indicates the likelihood that the difference between the FSS group and control group arose by chance. Statistical significance levels are indicated as:

* = 10 percent; ** = 5 percent; *** = 1 percent.

Results displayed in italics are non-experimental. No tests of statistical significance were performed on differences between research groups in means or proportions.

SOURCE: MDRC calculations using responses to the FSS 36-Month Survey.

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 91

Chapter 5. Impacts on Employment and Earnings

Conclusions

Overal , the findings reviewed in chapter 5 continue to provide only slight evidence for the program’s impacts on earnings and employment. For the full sample, the NDNH data reveal no impacts on quarterly earnings or employment. Given that FSS is a 5-year program and that participants can be focused on a range of self-sufficiency goals during this time, it may be too early to conclude whether the program is effective at shifting outcomes in this domain. Chapter 7 examines the program’s effects on subgroups and whether study participants, defined by baseline characteristics such as their work and educational status or the type of FSS

program they were enrol ed in, respond differently to the services and escrow incentive offered by FSS.

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 92

Chapter 6. Impacts on Income, Subsidy Receipt, and Well-Being Chapter 6. Impacts on Income,

Subsidy Receipt, and Wel -Being

Chapter 6 analyzes whether the Family Self-Sufficiency (FSS) program led to positive effects on household income and on a series of self-sufficiency and financial security outcomes. The analysis uses responses to the FSS 36-month survey as the primary source for calculating financial and material wel -being outcomes and estimating program impacts, capturing circumstances and outcomes for the period before FSS escrow payments were made to program graduates.100 Additional measures were calculated using HUD administrative data on household income, participant housing expenditures, and rent subsidies.

The FSS program seeks to promote Housing Choice Voucher program households’ self-sufficiency and financial security in several ways. First and foremost, FSS programs offer participants access to services and an escrow account that promote new employment, employment stability, and earnings growth. As discussed in chapter 5, after 3 years, FSS heads of households experienced similar incidences of employment and earnings on average to their counterparts in the control group. Nonetheless, the program could still improve FSS group households’ financial situation in two other ways. Once a head of household enrol s in FSS, other adult household members covered by the housing voucher can receive FSS services. (Either way, their earnings are counted toward the issuance of escrow credits.) In this way, FSS could potentially increase participants’

household earnings and income, beyond what they would have received in the absence of the program, without directly affecting the head of household’s employment and earnings. In addition, as discussed in chapter 2, many FSS programs strongly encourage participants to attend workshops or meet with counselors to receive instruction in managing personal and household finances or qualifying to purchase a home. As advocates for financial empowerment services often attest, these activities can lead to tangible financial gains, even without increases in income.101 For example, participants in financial security activities can learn to (1) increase savings; (2) reduce debt; (3) increase credit scores; (4) forgo high-cost, nontraditional lending sources; and (5) avoid financial hardship. Participants in financial security or homeownership preparation activities could also benefit in less tangible ways—for example, by reducing stress and experiencing a greater sense of control over life decisions and more optimism for the future.102

The main findings from this analysis are as fol ows:

• As of the end of Year 3, the FSS program had few statistically significant effects on indicators of financial wel -being, such as average household income and incidence of material hardships and food insecurity. In addition, the differences between research groups were inconsistent. The FSS program increased the use of bank accounts and reported higher credit scores, compared with control group levels, but FSS group members, on average, incurred greater amounts of debt.

• Consistent with programs that emphasize the use of financial security and homeownership preparation counseling and workshops, FSS led to positive effects on attitudinal outcomes. FSS group members, when surveyed, were more likely than control group respondents to assess that they had improved their financial situation in the past year and were better able to plan for the future.

100 It is possible an exit survey will be conducted as part of the ongoing evaluation, which will provide information on the post-exit circumstances of participants, including those who received substantial escrow disbursements.

101 See, for example, Collins and Gjertson (2013); Abbi (2012); Lopez-Fernandini (2012); McKernan, Ratcliffe, and Vinopal (2009).

102 In this chapter, the terms material well-being, financial well-being, and financial security are used interchangeably.

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 93

Chapter 6. Impacts on Income, Subsidy Receipt, and Well-Being

• A slightly larger proportion of FSS group households than control group households remained enrol ed in the HCV program in Month 36. During Month 36, FSS group members also averaged slightly higher out-of-pocket expenses (of $19) for rent and utilities and lower amounts of housing subsidies. The differences in total housing subsidies over 3 years and in Year 3 are not statistically significant.

Impacts on Household Income

Exhibit 6.1 shows the income sources for respondents in each research group. It shows the typical control group household as having received a little more than $1,600 (equivalent to nearly $20,000 per year) in income from all sources during the month preceding their interview. About two-thirds of control group respondents reported working for pay in the prior month, and nearly 20 percent of control group households received earnings from an additional household member. A large majority (79 percent) of control group respondents reported that their household received income from at least one public benefit program, most often SNAP

benefits (food stamps), free and reduced-price school lunches, and disability benefits. As observed at random assignment, relatively few control group households (12 percent) indicated that they were receiving Temporary Assistance for Needy Families (TANF) cash welfare benefits. About one in five control group respondents reported receiving child support.

When surveyed, control group respondents often described themselves as encountering difficulties in maintaining a steady income. Nearly one-half of the control group related that their monthly income varied at least a little month by month, and most control group respondents reported that they had experienced at least 1 month with zero or unusually low income in the past 12 months.

FSS group respondents reported patterns of income receipt and average income levels that resembled those of the control group. As of the end of Year 3, the FSS program did not increase the incidence of having a second wage earner in the household, nor did the program increase household income above the control group average.103 According to survey responses, FSS led to a small decrease (of 2 percentage points) in the receipt of TANF cash welfare benefits. As discussed above, FSS households need to remain off TANF for at least 12

months to qualify for graduation from the program. Possibly, this decrease in the receipt of TANF benefits wil continue or become larger in later years, as FSS group members approach the end of their FSS contract.104

Exhibit 6.1 also shows that both study groups experienced relatively similar patterns of income volatility.

Survey respondents were asked whether their household’s total monthly income stayed about the same each month, varied a little month by month, or varied a lot month by month. A little more than one-half of the respondents in both groups said their household income stayed about the same. About 13 percent of the FSS

group and 11 percent of the control group reported that their income varied a lot from month to month.

These differences are not statistically significant across groups. About 40 percent of the respondents in each group reported experiencing 3 or more months in the prior year in which their household income was zero or unusually low.

103 The difference in monthly household income (of $53) between the FSS and control groups is not statistically significant.

104 The evaluation does not collect TANF administrative data from state or local human services agencies. For reasons that are not clear, FSS also led to small increases above control group levels in the receipt of Supplemental Nutrition Program for Women, Infants, and Children (WIC) benefits. Appendix exhibit D.1 displays measures of head of household and total household income for FSS and control group households, calculated with federal HUD IMS/PIC administrative data. In contrast to findings from NDNH quarterly wage data, the housing data show positive effects on earnings. Possibly, the additional incentive to the FSS group to declare income (to receive escrow credits) resulted in a higher average compared with the control group. In addition, the exhibit includes current enrollees in the HCV program. It is also another possible factor driving these results could be that additional control group members with relatively high earnings may have exited from FSS and were not included in the calculations.

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 94

Chapter 6. Impacts on Income, Subsidy Receipt, and Well-Being Exhibit 6.1. Impacts on Household Income and Income Sources, Family Self-Sufficiency 36-Month Survey Respondent Sample

FSS

Control

Difference

Outcome

Group

Group

(Impact)

p-Value

Household Income in Month Before Interview

Household employment (%)

72.6

72.0

0.7

0.723

Employment by household member (%)

0.857

Respondent only

52.2

52.3

-0.1

Other household member only

6.0

6.1

-0.1

Respondent and other household member

14.4

13.5

0.9

Receipt of publicly-funded benefits (%)

81.0

79.0

2.0

0.244

SNAP/food stamps

57.3

58.3

-1.0

0.622

TANF

9.6

11.9

-2.3*

0.081

SSI-SSDI

26.4

27.1

-0.6

0.700

Unemployment Insurance

3.8

3.2

0.6

0.450

WIC

12.5

8.5

4.0***

0.004

Home energy assistance

13.3

10.8

2.5*

0.066

Free or reduced-priced lunch

50.9

49.4

1.4

0.443

Other income (%)

Child support

20.0

21.7

-1.7

0.339

Alimony

0.4

0.2

0.2

0.427

Other

4.7

5.2

-0.5

0.620

Average total household income ($)

1,686

1,633

53

0.281

Total household income (%)

0.694

No income

2.8

2.6

0.2

$1–$499

7.5

8.3

-0.8

$500–$999

19.7

20.6

-0.9

$1,000–$1,999

36.7

35.9

0.8

$2,000–$2,999

18.7

20.3

-1.6

$3,000 or more

14.6

12.3

2.3

Trends in Monthly Household Income

Stability of monthly household income (%)

0.290

Income amount stays about the same

53.9

54.3

-0.4

Income amount varies a little

33.0

35.0

-2.0

Income amount varies a lot

13.2

10.8

2.4

(continued)

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 95

Chapter 6. Impacts on Income, Subsidy Receipt, and Well-Being Exhibit 6.1. Impacts on Household Income and Income Sources, Family Self-Sufficiency 36-Month Survey Respondent Sample continued

FSS

Control

Difference

Outcome

Group

Group

(Impact)

p-Value

Number of months with zero or unusually low income in year before interview (%)

0.828

0

44.8

45.9

-1.1

1–2

14.3

13.6

0.8

3 or more

40.8

40.5

0.3

Sample size (total = 1,936)

988

948

FSS = Family Self-Sufficiency. SNAP = Supplemental Nutrition Assistance Program. SSI-SSDI = Supplemental Security Income-Social Security Disability Insurance. TANF = Temporary Assistance for Needy Families. WIC = Special Supplemental Nutrition Program for Women, Infants, and Children.

NOTES: The FSS 36-Month Survey respondent sample includes Housing Choice Voucher heads of household who were randomly assigned between October 18, 2013, and December 22, 2014, were age 18 to 61 at the time of random assignment and responded to the FSS 36-Month Survey. Sample sizes for specific outcomes may vary because of missing values. Estimates were regression-adjusted using ordinary least squares, controlling for differences in sample member characteristics recorded at the time of random assignment.

No special weights were applied to responses to adjust for differences in sample size by housing authority. Rounding may cause slight discrepancies in calculating sums and differences. A two-tailed t-test was applied to differences between the FSS group and the control group for continuous variables and selected outcomes expressed as proportions. In addition, a chi-square test for categorical variables was run to determine whether there is a difference in the distribution of related outcomes by research group. The p-value indicates the likelihood that the difference between the FSS group and control group arose by chance. Statistical significance levels are indicated as:

* = 10 percent; ** = 5 percent; *** = 1 percent.

SOURCE: MDRC calculations using responses to the FSS 36-Month Survey.

Impacts on Material Hardship and Food Insecurity

Having described themselves as receiving relatively low and sometimes irregular monthly incomes, many control group respondents then related that they had encountered at least one material hardship in the past year. Specifically, about 6 in 10 control group respondents stated that they could not pay for necessities, such as rent and utilities, telephone bil s, food, medical care, or prescription drugs during at least 1 month in the previous year. (See exhibit 6.2.) The FSS group was somewhat more likely to report difficulty paying their utility bil s, a difference of 4 percentage points that is statistically significant. As exhibit 6.2 shows, about 40

percent of control group members reported experiencing two or more material hardships and one in five control group members experienced a material hardship during 4 months or more in the year before the interview. A similar proportion of control group respondents stated that they sometimes did not have enough to eat. (FSS respondents were slightly less likely than their control counterparts to say they often did not have enough to eat.) As with measures of household income, for the most part, the FSS program led to only smal and inconsistent differences between the FSS and control groups in the incidence of material hardship and food insecurity.

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 96

Chapter 6. Impacts on Income, Subsidy Receipt, and Well-Being Impacts on Connection to Traditional Financial Institutions and

Sources of Credit

Having a savings, money market, or checking account at a bank or credit union and using these institutions to cash checks or as a source of credit can often protect households from paying high fees and high interest rates. Moreover, accumulating savings can enable households to deal with sudden decreases in income or unexpected expenses without accumulating debt. Having savings of $2,000 or more can also help households who are interested in purchasing a home to qualify for a mortgage. Irrespective of whether they accrued escrow credits, FSS group members may also have responded to encouragement to save from their meetings with financial counselors or attendance at financial management workshops.

At their 36-month interview, as exhibit 6.3 shows, a majority (61 percent) of control group respondents reported that they currently maintained a bank account (checking or savings).105 Apparently, most of these respondents were keeping checking accounts and paying out month by month about as much as or more than they were depositing.

Only a little more than one in five control group respondents stated that they had any savings, and control group respondents, as a group, saved less than $200. These savings rates are extremely low and underscore the relatively precarious financial position that many control group members were in. Adding to their financial insecurity, the typical control group respondent reported having accumulated nearly $14,000 in non-housing-related debt, and nearly 80 percent of control group respondents answered a hypothetical question about how they would likely pay for an emergency expense of $400 by relating that they would increase their debt.106

Most control group members (63 percent) also reported having used a nontraditional financial service, such as overdrawing on their checking account. As with other measures of financial security, the FSS program led to few and inconsistent differences with the control group on measures of savings, debt, and the use of traditional and nontraditional financial services. As exhibit 6.3 shows, by 6 percentage points, a larger proportion of FSS

group respondents reported having a bank account. By a slightly larger margin, FSS group respondents were more likely than their counterparts in the control group to use traditional financial services, although most of these FSS group members also used nontraditional financial services. Yet, for reasons that are not clear, FSS

group respondents averaged an additional $1,600 (12 percent) in debt, compared with the control group.107 The FSS program had no effect on savings, beyond the accrual of escrow credits.

105 In New York City’s Work Rewards demonstration, the 42-month survey showed that under one-half (or 43 percent) of the control group respondents reported having a bank account. Banking and savings rates were generally lower for the Work Rewards sample. See Verma et al.

(2017).

106 See Azurdia and Freedman (2016); Pew Charitable Trusts (2015); Cramer, King, and Schreur (2015); Hannagan and Morduch (2015); Abbi (2012) for related research on how low- and moderate-income households cope with sudden loss of income or emergency expenses.

107 The evaluation does not collect information on sources of debt for this report.

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 97

Chapter 6. Impacts on Income, Subsidy Receipt, and Well-Being Exhibit 6.2. Impacts on Material Hardship and Food Insufficiency, Family Self-Sufficiency 36-Month Survey Respondent Sample

FSS

Control

Difference

Outcome (%)

Group

Group

(Impact)

p-Value

Any material hardship in the past 12 months

59.8

59.0

0.8

0.712

Hardship paying commonly recurring monthly bil s

50.7

47.0

3.7*

0.089

Did not pay full rent or mortgage

24.7

22.1

2.6

0.169

Did not pay utility bil

38.3

34.3

4.0*

0.055

Did not pay telephone bil

27.3

27.8

-0.5

0.814

Health and physical wel -being hardships

37.6

40.6

-3.0

0.162

Did not buy prescription drug

16.6

17.4

-0.8

0.657

Did not see a doctor or get medical assistance

17.0

16.9

0.2

0.931

Did not buy food

24.3

27.1

-2.8

0.156

Number of material hardships in the past 12 months

0.833

0 hardship

40.2

41.2

-0.9

1 hardship

18.2

18.4

-0.2

2 hardships

16.1

16.2

0.0

3 hardships

12.1

10.3

1.8

4 or more hardships

13.4

14.0

-0.6

Number of months experiencing hardships paying

commonly recurring bil s in the past 12 months

0.294

0 months

49.5

53.1

-3.7

1–3

32.3

28.0

4.3

4–6

12.4

13.4

-1.0

7–12

5.8

5.4

0.4

Any severe material hardship in the past 12 monthsa

23.4

24.5

-1.1

0.560

Did not pay full rent or mortgage

6.7

5.7

1.0

0.380

Did not pay utility bil

11.9

12.1

-0.2

0.888

Did not pay telephone bil

9.0

10.1

-1.0

0.444

Did not buy food

10.7

11.3

-0.6

0.685

Food insufficiency in prior month (%)

**

0.045

Had enough of food wanted to eat

50.1

48.5

1.6

Had enough food, but not the kinds wanted to eat

29.8

29.7

0.1

Sometimes did not have enough to eat

15.7

14.6

1.2

Often did not have enough to eat

4.4

7.2

-2.9

Sample size (total = 1,936)

988

948

(continued)

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 98

Chapter 6. Impacts on Income, Subsidy Receipt, and Well-Being Exhibit 6.2. Impacts on Material Hardship and Food Insufficiency, Family Self-Sufficiency 36-Month Survey Respondent Sample continued

FSS = Family Self-Sufficiency.

aA severe material hardship is defined here as a hardship lasting 4 or more months.

NOTES: The FSS 36-Month Survey respondent sample includes Housing Choice Voucher heads of household who were randomly assigned between October 18, 2013, and December 22, 2014, were age 18 to 61 at the time of random assignment, and responded to the FSS 36-Month Survey. Sample sizes for specific outcomes may vary because of missing values. Estimates were regression-adjusted using ordinary least squares, controlling for differences in sample member characteristics recorded at the time of random assignment.

No special weights were applied to responses to adjust for differences in sample size by housing authority. Rounding may cause slight discrepancies in calculating sums and differences. A two-tailed t-test was applied to differences between the FSS group and the control group for continuous variables and selected outcomes expressed as proportions. In addition, a chi-square test for categorical variables was run to determine whether there is a difference in the distribution of related outcomes by research group. The p-value indicates the likelihood that the difference between the FSS group and control group arose by chance. Statistical significance levels are indicated as:

* = 10 percent; ** = 5 percent; *** = 1 percent.

SOURCE: MDRC calculations using responses to the FSS 36-Month Survey.

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 99

Chapter 6. Impacts on Income, Subsidy Receipt, and Well-Being Impacts on Financial Wel -Being and Confidence in Handling

Family Finances

FSS case managers and financial counselors who work with FSS participants often try to help participants fix problems in their credit history and boost their credit score. Increasing credit scores can lead to multiple benefits, including paying lower interest rates on credit cards, qualifying for a mortgage, and even improving an applicant’s chances of being hired for a new job. As a first step, financial counselors and workshop leaders often teach participants how to access and interpret their credit reports, learn their credit scores, and identify problems that need immediate attention. As exhibit 6.4 shows, according to survey responses, control group members were experiencing considerable problems with their credit. About 80 percent of control group respondents reported that they did not have a credit score, did not know their score, or had a score below 600, which is generally considered to be a low and disadvantageous level. One-half of control group respondents also assessed their credit rating as “bad” or “very bad.”

Consistent with their greater use of financial security and homeownership preparation services, by a margin of 6 percentage points, a larger proportion of FSS group respondents knew their credit score sufficiently wel to place it within a range of scores. Moreover, by a similar margin, a larger proportion of FSS group members reported having a credit score of 600 or higher. Compared with the control group, a slightly smal er proportion of FSS group respondents assessed their credit rating as “bad” or “very bad,” although the difference of 2

percentage points is not statistically significant.

Typically, financial counselors and workshop leaders also teach strategies for improving the management of family finances, including budgeting income and expenditures in an organized and formal way, monitoring expenses more carefully, and spending less on non-necessities. Exposure to these financial empowerment strategies can have “spil over” effects by giving participants a sense of gaining better control over their financial situations and greater confidence in their ability to plan for the future. As exhibit 6.4 shows, the FSS

program did not affect respondents’ actual financial situation, as measured by their reported ability to have money left over at the end of the month but the FSS group realized gains above the control group in measures of their perception of financial wel -being and goal orientation and attainment. Specifically, by a margin of 7 percentage points, a larger proportion of FSS group respondents reported that their financial situation had improved over the past year.108 The FSS program also led to increases over control group levels of similar magnitude in the proportion of respondents who characterized themselves as meeting goals and having the ability to “get out of a jam.”

108 The financial well-being scale comprises the four financial hardship measures shown in the table (financial situation better than the last year, not worrying about having enough money in the future, generally affording buying needed things, and sometimes having enough money to buy something or go somewhere just for fun).

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 100

Chapter 6. Impacts on Income, Subsidy Receipt, and Well-Being Exhibit 6.3. Impacts on Use of Financial Services, Savings, and Debt, Family

Self-Sufficiency 36-Month Survey Respondent Sample

FSS

Control Difference

Outcome

Group

Group

(Impact)

p-Value

Use of Banking/Financial Services

Currently has bank account (%)

67.5

61.3

6.2***

0.001

Currently has savings (%)

20.3

22.5

-2.2

0.236

Average savings ($)

145

170

-26

0.344

Savings (%)

0.404

No savings

82.9

79.8

3.0

$1–$499

8.7

10.7

-2.0

$500–$1,999

5.7

5.7

0.0

$2,000 or more

2.8

3.8

-0.9

Use of Non-Traditional Financial Services in the Past 12 Months (%)

Used any non-traditional financial service

63.3

62.6

0.7

0.751

Cashed check at check casher

27.5

27.5

0.0

0.985

Paid bill at check casher

30.4

29.1

1.3

0.528

Took cash advance on credit card

7.8

6.0

1.8

0.128

Bounced check or overdrew checking account

28.5

26.9

1.6

0.426

Got payday loan

13.0

13.6

-0.6

0.693

Used any non-traditional financial services at least once

per month

36.7

37.3

-0.6

0.795

Cashed check at check casher

17.3

16.2

1.2

0.492

Paid bill at check casher

21.2

20.4

0.9

0.635

Took cash advance on credit card

2.9

2.6

0.3

0.680

Bounced check or overdrew checking account

8.5

9.9

-1.5

0.257

Got payday loan

3.0

2.7

0.4

0.653

Used two non-traditional financial services or more

30.7

28.3

2.5

0.238

Type of financial services used in past 12 months

**

0.040

Only non-traditional financial services used

20.6

24.8

-4.2

Only traditional financial services used

25.0

22.9

2.2

Traditional and non-traditional financial services

used

42.7

38.3

4.4

No financial services used

11.6

14.0

-2.4

(continued)

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 101

Chapter 6. Impacts on Income, Subsidy Receipt, and Well-Being Exhibit 6.3. Impacts on Use of Financial Services, Savings, and Debt, Family

Self-Sufficiency 36-Month Survey Respondent Sample continued

FSS

Control Difference

Outcome

Group

Group

(Impact)

p-Value

Debt

Currently has debt (%)

76.2

76.0

0.2

0.893

Average current debt ($)

15,403

13,778

1,625**

0.045

Debt (%)

0.879

No debt

24.6

24.7

-0.1

$1–$1,999

9.7

11.1

-1.4

$2,000–$9,999

22.1

21.4

0.7

$10,000–$19,999

16.7

16.4

0.3

More than $20,000

26.9

26.4

0.5

How would pay for an emergency expense of $400 (%)

0.177

With money in savings, checking, or cash on hand

20.6

18.8

1.8

Increase debt

76.6

79.2

-2.7

Sell something

2.8

2.0

0.9

Was late on payment of debt or bill in past 12 months (%)

38.1

37.8

0.3

0.882

Sample size (total = 1,936)

988

948

FSS = Family Self-Sufficiency.

NOTES: The FSS 36-Month Survey respondent sample includes Housing Choice Voucher heads of household who were randomly assigned between October 18, 2013, and December 22, 2014, were age 18 to 61 at the time of random assignment and responded to the FSS 36-Month Survey. Sample sizes for specific outcomes may vary because of missing values. Estimates were regression-adjusted using ordinary least squares, controlling for differences in sample member characteristics recorded at the time of random assignment.

No special weights were applied to responses to adjust for differences in sample size by housing authority. Rounding may cause slight discrepancies in calculating sums and differences. A two-tailed t-test was applied to differences between the FSS group and the control group for continuous variables and selected outcomes expressed as proportions. In addition, a chi-square test for categorical variables was run to determine whether there is a difference in the distribution of related outcomes by research group. The p-value indicates the likelihood that the difference between the FSS group and control group arose by chance. Statistical significance levels are indicated as:

* = 10 percent; ** = 5 percent; *** = 1 percent.

SOURCE: MDRC calculations using responses to the FSS 36-Month Survey.

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 102

Chapter 6. Impacts on Income, Subsidy Receipt, and Well-Being Exhibit 6.4. Impacts on Financial Strain, Family Self-Sufficiency 36-Month Survey Respondent Sample

FSS

Control Difference

Outcome

Group

Group

(Impact)

p-Value

Financial Well-Being

Financial wel -being score (4-low; 16-high)

11.0

10.7

0.3**

0.048

Strongly or somewhat agree with the fol owing (%)

Financial situation is better than last year

54.3

47.4

6.9***

0.003

Doesn’t worry about having enough money in

the future

21.8

20.3

1.5

0.433

Can generally afford to buy needed things

63.0

62.2

0.8

0.720

Sometimes have enough money to buy something or

just go somewhere for fun

23.1

23.4

-0.2

0.899

Financial situation at the end of the month (%)

0.344

Has money left over

13.4

13.2

0.2

Has just enough money to make ends meet

52.7

50.9

1.8

Does not have enough money to make ends meet

33.9

35.9

-2.0

Credit score (%)

***

0.005

Does not know or have score

47.5

53.4

-5.9

Less than 600

26.9

26.9

-0.1

600–649

12.6

8.6

4.0

650 or higher

13.1

12.0

1.9

Self-assessed credit rating (%)

0.532

Bad or very bad

50.0

52.2

-2.2

Average

34.2

31.8

2.4

Good or very good

15.8

16.0

-0.1

(continued)

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 103

Chapter 6. Impacts on Income, Subsidy Receipt, and Well-Being Exhibit 6.4. Impacts on Financial Strain, Family Self-Sufficiency 36-Month Survey Respondent Sample continued

FSS

Control Difference

Outcome

Group

Group

(Impact)

p-Value

Goal-oriented thinking

Average score on “State of Hope” scalea

21.2

20.7

0.5**

0.031

Agree with the fol owing statements: (%)

At this time, I am meeting the goals that I set for

myself

55.5

50.7

4.8**

0.035

I can think of many ways to reach my current goals

81.4

78.9

2.5

0.175

Right now, I see myself as pretty successful

39.9

37.5

2.4

0.294

There are lots of ways around the problems I am

facing now

66.1

63.5

2.6

0.252

At the present time, I am energetically pursuing my

goals

66.8

60.6

6.2***

0.005

If I should find myself in a jam, I could think of

many ways to get out of it.

74.1

68.9

5.3**

0.011

Sample size (total = 1,936)

988

948

FSS = Family Self-Sufficiency.

aThe State of Hope scale was used to measure goal-oriented thinking (6 = low; 30 = high).

NOTES: The FSS 36-Month Survey respondent sample includes Housing Choice Voucher heads of household who were randomly assigned between October 18, 2013, and December 22, 2014, were age 18 to 61 at the time of random assignment and responded to the FSS 36-Month Survey. Sample sizes for specific outcomes may vary because of missing values. Estimates were regression-adjusted using ordinary least squares, controlling for differences in sample member characteristics recorded at the time of random assignment.

No special weights were applied to responses to adjust for differences in sample size by housing authority. Rounding may cause slight discrepancies in calculating sums and differences. A two-tailed t-test was applied to differences between the FSS group and the control group for continuous variables and selected outcomes expressed as proportions. In addition, a chi-square test for categorical variables was run to determine whether there is a difference in the distribution of related outcomes by research group. The p-value indicates the likelihood that the difference between the FSS group and control group arose by chance. Statistical significance levels are indicated as:

* = 10 percent; ** = 5 percent; *** = 1 percent.

SOURCE: MDRC calculations using responses to the FSS 36-Month Survey.

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 104

Chapter 6. Impacts on Income, Subsidy Receipt, and Well-Being Impacts on Housing Status, Housing Expenditures, and

Rent Subsidies

The FSS program implements a 5-year strategy to increase voucher holders’ household income through better employment and, possibly, additional household members’ employment, leading to graduation and the disbursement of escrow dol ars. In the longer term (beyond graduation), FSS is intended to facilitate voucher holders’ transition to homeownership or unsubsidized housing. During the 5-year contract period, the effects of FSS on housing status, expenditures, and subsidies are more difficult to predict. It should be expected that FSS

group members would have a greater incentive than control group members to remain enrol ed in the HCV

program to accrue escrow credits and qualify for escrow disbursement, and possibly to maintain their access to FSS-related services and case management. Among study participants who remained in the HCV program, FSS group members would also be expected to pay more out-of-pocket for rent and utilities as their earnings increase. Nevertheless, other factors, such the lack of availability of low- and moderate-cost rental housing in each locality, household composition, and household income levels likely affect study participants’ housing choices, expenditures, and subsidies.

Exhibit 6.5 displays important summary measures of both groups’ cumulative expenditures and rent subsidies during Years 1 to 3 and in Month 36, calculated for the impact sample from HUD administrative data.

As exhibit 6.5 shows, a slightly larger proportion of FSS group households than control group households remained enrol ed in the HCV program in Month 36. During Month 36, FSS group members also averaged slightly higher out-of-pocket expenses (of $19) for rent and utilities. Concomitantly, FSS group members averaged lower amounts of housing subsidies, but the differences over 3 years and in Year 3 are not statistically significant—exhibit 6.6 displays findings on study participants’ current and recent housing situations, calculated from responses to the FSS 36-month survey. According to survey data, around 80 percent of respondents in both research groups reported that they were currently residing in subsidized or public housing.109 Most other respondents lived in unsubsidized rental housing at the time of their interview. The FSS

program did not affect the incidence of living in any particular type of housing, nor did the program lead to increases or decreases in the amount of rent that respondents paid. Among survey respondents, about one-third of each research group moved to a new home during the fol owup period, and 10 percent moved two times or more. Respondents from both research groups assessed their current living situations positively. A little more than one-half of each group expressed satisfaction with both their current home and neighborhood, whereas only a little over 10 percent expressed dissatisfaction with both. The remaining respondents indicated mixed assessments of their current home and neighborhood.

109 It is possible that some HCV holders transitioned to public housing after random assignment. Nevertheless, it may be assumed that some respondents who reported living in public housing were actually residing in subsidized private housing.

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 105

Chapter 6. Impacts on Income, Subsidy Receipt, and Well-Being Exhibit 6.5. Impacts on Shelter Costs and Housing Subsidies, Years 1 to 3, Family Self-Sufficiency Impact Sample

FSS

Control Difference

Outcome

Group

Group

(Impact)

p-Value

Enrol ed in HCV program in Month 36 (%)

81.4

79.0

2.4

0.120

Gross Rent (Contract Rent + Utilities Allowance)

Average gross rent in Month 36, if received HCV ($)

1,395

1,406

-11

Gross rent in Month 36, if received HCV (%)

Less than $1,000

23.8

24.1

-0.3

$1,000–$1,499

39.1

38.7

0.4

$1,500 or above

37.1

37.2

-0.1

Family Share of Rent and Utilities

Family share in Month 36 (%)

**

0.048

Exited HCV

18.6

21.2

-2.6

$0

0.7

0.5

0.3

$1–$100

10.0

10.8

-0.8

$101–$300

18.0

18.5

-0.5

$301–$700

31.1

29.9

1.1

$701 or above

21.5

19.1

2.4

Average family share in months received HCV ($)

500

489

11

Average family share in Month 36, if received HCV ($)

513

494

19

Housing Subsidy

Total housing subsidy ($)

Year 1

9,800

9,774

26

0.833

Year 2

8,982

8,890

92

0.577

Year 3

8,363

8,566

-203

0.300

Years 1–3

27,146

27,231

-86

0.839

Average housing subsidy in Month 36, if received HCV ($)

877

907

-31

Sample size (total = 2,548)

1,281

1,267

(continued)

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 106

Chapter 6. Impacts on Income, Subsidy Receipt, and Well-Being Exhibit 6.5. Impacts on Shelter Costs and Housing Subsidies, Years 1 to 3, Family Self-Sufficiency Impact Sample continued

FSS = Family Self-Sufficiency. HCV = Housing Choice Voucher.

NOTES: The FSS impact sample includes HCV households that were randomly assigned between October 18, 2013, and December 22, 2014, and with a head of household age 18 to 61 at the time of random assignment. Rent and subsidy calculations used data from each household’s HCV annual and interim eligibility re-examinations. Recorded amounts were copied to successive months until a new eligibility re-examination took place, or the household left housing assistance. Cumulative totals for former HCV households cover their months of eligibility following random assignment. Estimates were regression-adjusted using ordinary least squares, controlling for pre-random assignment characteristics of sample members. For each dollar amount outcome, values above the 99th percentile were considered as outliers and dropped from the calculations. As a result of this procedure, adjusted mean values for total family share and subsidies detail do not sum to total rent plus utility allowance. A two-tailed t-test was applied to differences between the FSS group and the control group for continuous variables and selected outcomes expressed as proportions. In addition, a chi-square test for categorical variables was run to determine whether there is a difference in the distribution of related outcomes by research group. The p-value indicates the likelihood that the difference between the FSS group and control group arose by chance. Statistical significance levels are indicated as: * = 10 percent; ** = 5 percent; and *** = 1 percent. Results displayed in italics are non-experimental. No tests of statistical significance were performed on differences between research groups in means or proportions. Sample sizes for specific outcomes may vary because of missing values. Rounding may cause slight discrepancies in calculating sums and differences.

SOURCE: HUD Inventory Management System/Office of Public and Indian Housing Information Center data.

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 107

Chapter 6. Impacts on Income, Subsidy Receipt, and Well-Being Exhibit 6.6. Impacts on Housing, Family Self-Sufficiency 36-Month Survey Respondent Sample

FSS

Control Difference

Outcome

Group

Group

(Impact)

p-Value

Current Housing Status

Housing type (%)

0.772

Owns home or apartment

2.7

2.3

0.5

Rents home or apartment

93.6

94.0

-0.4

Lives in public housing

5.9

7.7

-1.8

Lives in subsidized rental housing

76.8

73.5

3.2

Lives in unsubsidized private housing

10.9

12.8

-2.0

Does not pay rent (at-risk housing)

3.7

3.7

0.0

Average rent paid ($)

497

474

23

0.204

Rent (%)

0.285

Does not pay rent

13.8

13.9

-0.1

$1–$299

26.6

26.6

0.0

$300–$499

16.5

18.0

-1.6

$500–$799

18.6

20.8

-2.2

$800 or higher

24.6

20.7

3.9

Moves

Number of moves (%)

0.834

Did not move

67.6

67.7

-0.1

1 time

22.0

22.9

-1.0

2 times

7.5

7.1

0.4

3 times or more

2.9

2.3

0.6

Primary reason for most recent move (%)

0.520

Did not move

67.7

67.8

-0.1

Moved for negative reasons (compel ed to move by

problems in previous home or locality)

18.8

20.4

-1.6

Moved for positive reasons (expected better housing or

personal or family wel -being)

13.5

11.9

1.7

Satisfaction with current living conditions (%)

0.923

Satisfied with housing and neighborhood

56.7

56.6

0.1

Mixed assessment of satisfaction

31.5

31.4

0.1

Dissatisfied with housing and neighborhood

11.8

12.0

-0.2

Sample size (total = 1,936)

988

948

(continued)

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 108

Chapter 6. Impacts on Income, Subsidy Receipt, and Well-Being Exhibit 6.6. Impacts on Housing, Family Self-Sufficiency 36-Month Survey Respondent Sample continued

FSS = Family Self-Sufficiency.

NOTES: The FSS 36-Month Survey respondent sample includes Housing Choice Voucher heads of household who were randomly assigned between October 18, 2013, and December 22, 2014, were age 18 to 61 at the time of random assignment and responded to the FSS 36-Month Survey. Sample sizes for specific outcomes may vary because of missing values. Estimates were regression-adjusted using ordinary least squares, controlling for differences in sample member characteristics recorded at the time of random assignment.

No special weights were applied to responses to adjust for differences in sample size by housing authority. Rounding may cause slight discrepancies in calculating sums and differences. A two-tailed t-test was applied to differences between the FSS group and the control group for continuous variables and selected outcomes expressed as proportions. In addition, a chi-square test for categorical variables was run to determine whether there is a difference in the distribution of related outcomes by research group. The p-value indicates the likelihood that the difference between the FSS group and control group arose by chance. Statistical significance levels are indicated as:

* = 10 percent; ** = 5 percent; *** = 1 percent.

SOURCE: MDRC calculations using responses to the FSS 36-Month Survey.

Conclusions

As discussed in chapters 2 and 3, FSS group members’ involvement in employment-related and financial security services decreased over time, as did the rate at which FSS group members opened escrow accounts.

Most likely, in the future years FSS group members’ ability to achieve greater financial wel -being, compared with the control group, will largely depend on how many FSS group members increase their earnings, add new credits to their escrow account, and graduate from the program and receive their escrow dol ars. A one-time infusion of $5,000 or more can make a big difference to a household’s finances, irrespective of whether the household uses the money to make a large purchase, fund the education of a family member, or reduce debt. Therefore, future research on the implementation of the program needs to focus on how successful the FSS programs will be in keeping participants at least minimally engaged and helping them graduate from the program.

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 109

Chapter 7. Variation in Program Impacts

Chapter 7. Variation in

Program Impacts

Findings on program impacts for the entire impact sample may mask positive or negative effects for certain groups who may have had different exposure to the Family Self-Sufficiency program or may have responded differently to FSS services and financial incentives.110 Chapter 7 first examines the FSS interim impacts for subgroups defined by characteristics of study participants, recorded at baseline. Next, the chapter considers whether programs with similar program implementation orientations and practices led to more positive (or negative) effects on employment, earnings, and indicators of financial security, compared with other programs.

In addition, exploratory analyses were conducted to examine variations in subgroup effects for FSS programs clustered according to implementation outcomes recorded after random assignment.

These analyses draw on housing agency program records, quarterly wage data from NDNH, HUD

administrative rent and subsidy data, and responses to the FSS 36-month survey. Like in chapters 5 and 6, for each subgroup, the analysis considers whether differences in average outcomes between the FSS group and the control group are statistically significant at the 10 percent level or less. In addition, exhibits in chapter 7 present the results of an H-statistic, which shows the likelihood that the observed variation in program effects among related subgroups is statistically significant and did not occur by chance.111

For subgroups defined by baseline characteristics, the main findings include the fol owing:

• There are isolated differences in impacts by subgroup on measures of employment and earnings and financial wel -being, but there is no consistent pattern of variation to suggest that the program is more effective for selected subgroups of respondents.

• The FSS program appears to have positive effects on self-reported measures of employment and financial wel -being among more disadvantaged subgroups. This pattern was not seen for employment and earnings outcomes measured with NDNH wage data, however.

For FSS program clusters with similar implementation features, the main findings include the fol owing:

• With one exception, there are isolated differences in impacts when study participants are grouped together into “site clusters” based on similar implementation features for FSS programs—but no consistent pattern of variation.

• Over 3 years of fol owup, FSS programs with a strong emphasis on monitoring and engagement was associated with earnings decreases relative to the control group of about $8,000 (or 20 percent) per study participant.

110 Throughout this chapter, the terms housing agency, public housing agency (PHA), site, and FSS program will be used interchangeably.

111 Findings for subgroups or groups of PHAs with similar post-random assignment outcomes should be considered as less reliable than findings for subgroups defined based on similar baseline characteristics or for groups of PHAs with similar implementation approaches or practices in effect at the time the study was launched. Outcomes may change over time, which could affect the membership of post-random assignment subgroups or groupings of PHAs. For now, these findings may provide insights as to why impacts occurred (or did not), but they are not suitable for hypothesis testing or predicting impacts in future years. More generally, the larger number of comparisons between research groups increases the risk that some positive impact estimates for one or more subgroups will occur due to natural sampling variation and are not true effects of the FSS program.

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 110

Chapter 7. Variation in Program Impacts

• An exploratory and tentative analysis suggests that FSS programs can facilitate escrow accrual in several ways, including al owing some employed FSS group members to accumulate escrow while delaying or forgoing participation in FSS activities. Fol owing this strategy appears to boost average total earnings above control group levels by about $3,750 per study participant over 3 years.

Program Impacts by Subgroups

Subgroups Based on Participant Baseline Characteristics

Exhibits 7.1A to 7.1E and appendix exhibits E.1 to E.3 show FSS program impacts on selected outcomes for subgroups based on baseline characteristics of the impact sample around the time of random assignment: employment status, highest educational attainment, estimated annual household income, receipt of Supplemental Security Income or Social Security Disability Insurance (SSI/SSDI) benefits, reported having a barrier to employment, and percentage of rent and utilities expenses paid out of pocket during the month of random assignment.112 Of particular importance for the FSS evaluation is examining whether the FSS

program’s combination of services and financial incentives leads to increases above the control group more consistently among subgroups with greater disadvantages in the labor market or subgroups with fewer disadvantages.113 Each exhibit il ustrates the impacts for a range of selected outcomes for a particular subgroup.

For measures of employment and earnings recorded from NDNH data, mean values for control group members (and for FSS group members) varied substantially by subgroup. However, the FSS program led to only small and not statistically significant differences (or impacts) on these measures when average values for each research group are compared. For example, in any given quarter, about 84 percent of control group members who reported working ful -time hours at baseline were employed, according to NDNH quarterly wage data—over twice the rate for control group members with no reported employment at baseline (see appendix exhibit E.1). Control group members’ average quarterly employment rate exceeded the rate for FSS group members by about 2 percentage points, but the difference is not statistically significant. Similarly, control group members with ful -time employment at baseline averaged $71,000 in total earnings over 3 years, more than three times the average for control group members with no employment at baseline. Moreover, the proportion of control group members in the ful -time employment subgroup who averaged more than $25,000

per year in earnings (46 percent) was nearly 10 times as high as the proportion for control group members with no employment at baseline. For both these measures, averages for the FSS group are slightly smal er, but 112 Employment status, highest educational attainment, having a barrier to employment, and SSI/SSDI benefits receipt are self-reported; estimated household income and out-of-pocket rent and utilities expenses were collected from HUD administrative data. Appendix exhibits E.1 through E.3 include regression-adjusted mean values for FSS and control group members, as well as FSS–control group differences (impacts) in outcomes. These tables also include results for two additional subgroups: having or not having at least one barrier to employment (self-reported) and percentage of household income used to pay for rent and utilities (family share, from HUD administrative data). This subgroup analysis should be considered as exploratory. Future reports will present results of a more formal analysis that will include outcomes data covering additional years of followup and a machine learning framework which includes cross validation to help reduce the number of tests and help validate any patterns we see.

113 The most disadvantaged subgroups include those who are not employed at random assignment, have no educational degree or credential, have no annual household income or income up to $10,000, reported one or more barriers to employment, are receiving SSI or SSDI benefits, and are paying up to 25 percent of their housing expenses out of pocket (the last measure is related to the amount of household income). The moderately disadvantaged subgroups include those who are employed part-time hours, have a high school diploma or equivalent or some college, have an annual household income of $10,001 to $20,000, and are paying between 25.01 and 50.00 percent of their housing expenses out of pocket. The least disadvantaged subgroups include those who are employed with full-time hours, have a 2-year college degree or higher, have an annual household income of more than $20,000, and are paying more than 50 percent of their household expenses out-of-pocket.

The remaining subgroups may be considered as “mixed,” meaning that they include study participants with a broad range of characteristics that could affect their future employment or earnings. An example of a mixed subgroup would be “not receiving SSI/SSDI” and “not having a barrier to employment.”

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 111

Chapter 7. Variation in Program Impacts

once again, the differences are not statistically significant. With few exceptions, these patterns of impacts are repeated for the other subgroups.114

As discussed in chapter 6 (for the full impact sample), respondents to the FSS 36-month survey reported a more complex pattern of employment outcomes and impacts. For most subgroups, the FSS group reported a higher incidence of working ful -time hours at the time of the interview compared with control group members. Statistically significant increases above the control group ranged from 4 percentage points to 12 percentage points. Impacts were recorded for four subgroups with greater disadvantages and for three additional subgroups.115 Although the impacts are not statistically significant across subgroups, these findings suggest that the FSS program may have facilitated more disadvantaged study participants to realize slightly better employment outcomes (more hours of work but not higher earnings) compared with their control group counterparts.

As appendix exhibit E.2 shows, control group members reported relatively low household incomes, ranging from $1,268 per month (or about $15,000 per year) for control group respondents with no household income at baseline to $2,246 (or about $27,000 per year) for control group respondents with more than $20,000 in annual household income at baseline. About one-half to three-fourths of control group members reported having a bank account, although average savings were extremely low—averaging less than $250 for all subgroups.

Average debt levels greatly exceeded savings for all subgroups, with control group members with a 2-year col ege degree or higher reporting the largest amount of debt by far of any subgroup (an average of nearly $27,000). Control group respondents in all subgroups had a mixed reaction when asked whether their current financial situation was better than their situation a year ago. Among all subgroups, more than 40 percent of control group respondents assessed their financial situation as better than a year ago, but only in subgroups of less disadvantaged respondents (those who are employed with ful -time hours, whose household income is above $20,000, or who are paying more than 50 percent of their housing expenses out of pocket) did a majority of control group members report that they were better off. Most control group members reported that they were in good, very good, or excel ent health, although control group members in the more disadvantaged subgroups reported the lowest incidence of having good health compared with other subgroups.

For three outcomes—average monthly household income, having a checking or savings account, and assessing one’s financial situation as better than a year ago—FSS group respondents in most subgroups responded more positively than control group respondents. For two of these measures (not monthly household income) most of the positive impacts for the FSS group were either statistically significant or were slightly above the 10 percent level of statistical significance. In addition, FSS group members in nearly all subgroups reported higher debt levels than control group members, but most differences are not statistically significant.

114 As appendix exhibit E.1 shows, the FSS program increased the average quarterly employment rate by 3 percentage points above the control group level among study participants who paid more than 50 percent of their household income for rent and utilities. The FSS program also led to an impact of nearly 5 percentage points above the control group in the measure of having average annual earnings of more than $25,000 among study participants whose highest educational credential was a high school diploma or equivalent.

115 The more disadvantaged subgroups are those who are not employed at random assignment, have no educational degree or credential, have an annual household income of up to $10,000, and are paying up to 25 percent of their housing expenses out of pocket.

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 112

Chapter 7. Variation in Program Impacts

Exhibit 7.1A. Impacts on Selected Outcomes, by Employment Status at Random Assignment $3,000

11.6**

12.0

10.0

9.2***

$2,000

7.4**

8.0

$1,492

6.1**

6.4**

5.6

5.3

6.0

$1,000

4.0

$814

1.9

1.6

2.0

2.0

0.8

1.0

0

0.0

-0.2

-1.1

-2.0

-1.8

Has Bank

Reports

Enrolled

Percentage Points

-$1,000

Account

Better

in HCV

-4.0

in

Financial

Program

Month 36

Situation

in

in

Month 36

-6.0

Average

Employed

Quarterly

in

Month 36

-$2,000

Employment

Month 36

Than a

-8.0

Rate in

with

Year Ago

Years

Full-Time

-$2,350

-10.0

1 to 3

Hours

-$3,000

Total

-12.0

Earnings

in Years

1 to 3

Not Employed

Employed Part Time

Employed Full Time

FSS = Family Self-Sufficiency. GED = General Educational Development. HCV = Housing Choice Voucher. SSI/SSDI = Supplemental Security Income/Social Security Disability.

NOTES: The FSS impact sample includes HCV households that were randomly assigned between October 18, 2013, and December 22, 2014, and with a head of household age 18 to 61 at the time of random assignment. Estimates were regression-adjusted using ordinary least squares, controlling for differences in sample member characteristics recorded at the time of random assignment. No special weights were applied to responses to adjust for differences in sample size by housing authority. Average quarterly employment rate is calculated as total quarters with employment divided by total quarters of followup, expressed as a percentage. Statistical significance levels are indicated as: * = 10 percent; ** = 5 percent; *** = 1 percent. The H-statistic test was used to test for statistically significant differences in impact estimates across different subgroups. Statistical significance levels are indicated as: † = 10 percent; †† = 5 percent; and ††† = 1

percent.

SOURCES: MDRC calculations from baseline data, HUD Inventory Management System/Office of Public and Indian Housing Information Center data, quarterly wage data from the National Directory of New Hires, and responses to the FSS 36-Month Survey.

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 113

Chapter 7. Variation in Program Impacts

Exhibit 7.1B. Impacts on Selected Outcomes, by Educational Attainment at

Random Assignment

$1,200

14.0

$986

$1,029

12.1**

12.0

$965

10.6** 10.9*

10.0

$800

8.7

8.0

6.8**

5.6*

5.3

5.6

6.0

$400

4.5

3.9

4.0

4.0

2.7

3.1

1.3

1.7 1.9

1.3

2.0

0.2

0

0.0

-0.1 -0.1

-2.0

Average

Employed

Has Bank

Reports

Enrolled

-$266

Quarterly

in

Account

Better

in HCV

Percentage Points

-4.0

-$400

Employment

Month 36

in

Financial

Program

Rate in

with

Month 36

Situation

in

Total

-6.0

Years

Full-Time

in

Month 36

Earnings

1 to 3

Hours

Month 36

in Years

-8.0

Than a

-$800

1 to 3

Year Ago

-10.0

-12.0

-$1,200

-14.0

No Degree

High School Diploma

Some College

2-Year College Degree

or Credential

or GED

or Higher

FSS = Family Self-Sufficiency. GED = General Educational Development. HCV = Housing Choice Voucher. SSI/SSDI = Supplemental Security Income/Social Security Disability.

NOTES: The FSS impact sample includes HCV households that were randomly assigned between October 18, 2013, and December 22, 2014, and with a head of household age 18 to 61 at the time of random assignment. Estimates were regression-adjusted using ordinary least squares, controlling for differences in sample member characteristics recorded at the time of random assignment. No special weights were applied to responses to adjust for differences in sample size by housing authority. Average quarterly employment rate is calculated as total quarters with employment divided by total quarters of followup, expressed as a percentage. Statistical significance levels are indicated as: * = 10 percent; ** = 5 percent; *** = 1 percent. The H-statistic test was used to test for statistically significant differences in impact estimates across different subgroups. Statistical significance levels are indicated as: † = 10 percent; †† = 5 percent; and ††† = 1

percent.

SOURCES: MDRC calculations from baseline data, HUD Inventory Management System/Office of Public and Indian Housing Information Center data, quarterly wage data from the National Directory of New Hires, and responses to the FSS 36-Month Survey.

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 114

Chapter 7. Variation in Program Impacts

Exhibit 7.1C. Impacts on Selected Outcomes, by Total Household Income at

Random Assignment

$3,000

$2,824

20.0

16.7**

15.0

$2,000

11.4

10.1***

10.0***

10.0

7.2

$1,000

4.6

5.05.5

5.0

3.5 3.0

2.6

2.0 2.1

1.9

2.1 0.7

0

0.0

-$132

-0.1

-2.4

Has Bank

Reports

Enrolled

Account

Better

in HCV

-5.0 Percentage Points

-$1,000

-6.0

in

Financial

Program

Month 36

Situation

in

in

Month 36

-10.0

Average

Employed

Month 36

Quarterly

in

Than a

-$2,000 -$1,856

Employment

Month 36

Year Ago

Rate in

with

-15.0

Years

Full-Time

Total

1 to 3

Hours

Earnings

-$3,000

in Years

-20.0

1 to 3

No Income

$1–$10,000

$10,001–$20,000

More Than $20,000

FSS = Family Self-Sufficiency. GED = General Educational Development. HCV = Housing Choice Voucher. SSI/SSDI = Supplemental Security Income/Social Security Disability.

NOTES: The FSS impact sample includes HCV households that were randomly assigned between October 18, 2013, and December 22, 2014, and with a head of household age 18 to 61 at the time of random assignment. Estimates were regression-adjusted using ordinary least squares, controlling for differences in sample member characteristics recorded at the time of random assignment. No special weights were applied to responses to adjust for differences in sample size by housing authority. Average quarterly employment rate is calculated as total quarters with employment divided by total quarters of followup, expressed as a percentage. Statistical significance levels are indicated as: * = 10 percent; ** = 5 percent; *** = 1 percent. The H-statistic test was used to test for statistically significant differences in impact estimates across different subgroups. Statistical significance levels are indicated as: † = 10 percent; †† = 5 percent; and ††† = 1

percent.

SOURCES: MDRC calculations from baseline data, HUD Inventory Management System/Office of Public and Indian Housing Information Center data, quarterly wage data from the National Directory of New Hires, and responses to the FSS 36-Month Survey.

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 115

Chapter 7. Variation in Program Impacts

Exhibit 7.1D. Impacts on Selected Outcomes, by Disability Status at Random Assignment $2,500

12.0

$2,198

10.4

10.0

$2,000

8.3

7.3***

8.0

$1,500

5.3

5.7***

6.0

$1,000

4.3*

3.8

3.5**

4.0

$500

2.0

0

0.0

-0.3

Has Bank

Reports

Account

Better

-2.0

-$500

-$482

in

Financial

Percentage Points

Month 36

Situation

-2.9

-4.0

-$1,000

in

Month 36

Than a

-6.0

Total

Average

Employed

-$1,500

Earnings

Quarterly

in

Year Ago

Enrolled

in HCV

in Years

Employment

Month 36

Program

-8.0

1 to 3

Rate in

with

in

-$2,000

Years

Full-Time

Month 36

-10.0

1 to 3

Hours

-$2,500

-12.0

Receive SSI/SSDI

Did Not Receive SSI/SSDI

FSS = Family Self-Sufficiency. GED = General Educational Development. HCV = Housing Choice Voucher. SSI/SSDI = Supplemental Security Income/Social Security Disability.

NOTES: The FSS impact sample includes HCV households that were randomly assigned between October 18, 2013, and December 22, 2014, and with a head of household age 18 to 61 at the time of random assignment. Estimates were regression-adjusted using ordinary least squares, controlling for differences in sample member characteristics recorded at the time of random assignment. No special weights were applied to responses to adjust for differences in sample size by housing authority. Average quarterly employment rate is calculated as total quarters with employment divided by total quarters of followup, expressed as a percentage. Statistical significance levels are indicated as: * = 10 percent; ** = 5 percent; *** = 1 percent. The H-statistic test was used to test for statistically significant differences in impact estimates across different subgroups. Statistical significance levels are indicated as: † = 10 percent; †† = 5 percent; and ††† = 1

percent.

SOURCES: MDRC calculations from baseline data, HUD Inventory Management System/Office of Public and Indian Housing Information Center data, quarterly wage data from the National Directory of New Hires, and responses to the FSS 36-Month Survey.

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 116

Chapter 7. Variation in Program Impacts

Exhibit 7.1E. Impacts on Selected Outcomes, by Percentage of Rent and Utilities Expenses Paid by Household at Random Assignment

$3,000

12.0

$2,782

10.1**

10.0

8.1**

$2,000

7.6**

7.7**

8.0

5.9

6.0

5.3

4.0

$1,000

3.4*

3.9

3.8

4.0

3.3

2.7

1.9

2.0

0

0.0

-$96

-1.3

-2.0

-1.6

Employed

Has Bank

Reports

in

Account

Better

Percentage Points

-$1,000

Month 36

in

Financial

-4.0

with

Month 36

Situation

-$1,156

Average

Full-Time

in

Enrolled

Quarterly

-6.0

Hours

Month 36

in HCV

Employment

Than a

Program

-$2,000

Rate in

Year Ago

in

-8.0

Total

Years

Month 36

Earnings

1 to 3

-10.0

in Years

1 to 3

-$3,000

-12.0

0-25

25.01–50

More than 50

FSS = Family Self-Sufficiency. GED = General Educational Development. HCV = Housing Choice Voucher. SSI/SSDI = Supplemental Income/Social Security Disability Insurance.

NOTES: The FSS impact sample includes HCV households that were randomly assigned between October 18, 2013, and December 22, 2014, and with a head of household age 18 to 61 at the time of random assignment. Estimates were regression-adjusted using ordinary least squares, controlling for differences in sample member characteristics recorded at the time of random assignment. No special weights were applied to responses to adjust for differences in sample size by housing authority. Average quarterly employment rate is calculated as total quarters with employment divided by total quarters of followup, expressed as a percentage. Statistical significance levels are indicated as: * = 10 percent; ** = 5 percent; *** = 1 percent. The H-statistic test was used to test for statistically significant differences in impact estimates across different subgroups. Statistical significance levels are indicated as: † = 10 percent; †† = 5 percent; and ††† = 1

percent.

SOURCES: MDRC calculations from baseline data, HUD Inventory Management System/Office of Public and Indian Housing Information Center data, quarterly wage data from the National Directory of New Hires, and responses to the FSS 36-Month Survey.

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 117

Chapter 7. Variation in Program Impacts

Variation in Housing Outcomes

Appendix exhibit E.3 also displays research group means and differences (impacts) for rent and housing subsidy outcomes for subgroups defined with baseline data. Most outcomes presented in the table are nonexperimental because they exclude FSS and control group members who exited the HCV program before the end of Year 3. Among control group members, enrol ment levels in HCV (experimental comparison) show a fair amount of variation, ranging from just over two-thirds (for control group members who paid more than one-half of their housing expenses out of pocket) to 85 percent (for SSI/SSDI recipients and control group members who paid between one-fourth and one-half of their housing expenses out of pocket). Among most subgroups, a slightly larger proportion of FSS group members continued their enrol ment in the HCV

program, but the differences are not statistically significant (see exhibits 7.1A to 7.1E). In contrast, FSS led to an impact of 6 percentage points above the control group level among study participants who entered the evaluation with ful -time employment.116

A relatively consistent pattern appears when comparing rent and subsidy outcomes for ongoing HCV enrol ees (nonexperimental). On average, at the end of Year 3, FSS group members in most subgroups are paying a larger share of their housing costs out of pocket and are receiving smal er subsidies than their counterparts in the control group.

Subgroups Based on Program Implementation Features

This section considers whether the impacts of FSS on employment, earnings, and other outcomes varied by implementation features adopted by different PHAs. As a first step, the analysis tests whether impacts on important outcomes differed by PHA for any reason. To address this issue, statistical tests were run for 3-year employment and earnings outcomes, using quarterly wage data from NDNH, to determine whether any differences in impacts by PHA could be found and whether this variation was unlikely to have occurred by chance. Test results showed that impacts varied by PHA overal , although sample sizes for most PHAs are too small to reliably estimate the magnitude of site-specific impacts. Nonetheless, results from these initial statistical tests and results of additional testing of impacts by PHA that control for sample member characteristics strongly suggest that in the first 3 years of fol owup, some PHAs have positive impacts on employment and earnings, some PHAs have effects close to zero, and some PHAs have negative impacts.117 Future reports wil draw on longer-term data and continue to explore this pattern and attempt to examine the factors driving the cross-site variation in impacts.

Next, the analysis considers whether PHAs with similar implementation orientation and practices have impacts of a similar magnitude and direction. (See exhibit 7.2 for a description of the program clusters examined in chapter 7.) PHAs may vary in impacts for reasons unrelated to differences in program implementation. For example, variation in impacts by PHA could result from differences in the characteristics of the research samples—as when certain PHAs have an unusually large proportion of subgroups who experienced positive or negative impacts on crucial outcomes. In theory, this caveat could apply to FSS, although, as discussed above, the analysis based on 3 years of fol owup has not identified subgroups with consistently positive or negative 116 At this point, it is not clear why the impact would be largest for FSS group members with full-time employment at random assignment.

Possibly, earnings levels for FSS group members will increase more quickly as more FSS group members finish their participation in FSS

activities, graduate from the program, and receive their disbursement.

117 See Bloom et al. (2017) for a description of the statistical test. Test results showed that variation in the overall pattern of impacts by PHA is statistically significant at the 1 percent level, meaning that it was unlikely that all PHAs had the same impacts—negative, positive, or close to zero.

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 118

Chapter 7. Variation in Program Impacts

impacts.118 PHAs may also vary in impacts because of differences in local labor markets or local housing markets that may affect employment opportunities or employment choices for FSS and control group members in unique ways. (This issue will be explored in later reports and with additional fol owup.) Grouping together PHAs based on their implementation features, however, helps al eviate this measurement issue by combining participants from different regions of the United States; from smal -, medium-, and large-sized cities and from suburban areas; and from high-growth and slow-growth labor and housing markets.

Program Clusters Based on Program Focus

PHAs could vary in impacts because of differences in how FSS administrators and case managers work with participants to set their individual goals and training plans and whether these individual self-sufficiency road maps emphasize work, education, and training, or focus on other aspects of financial security and management. For this analysis, the PHAs are grouped into site clusters based on information recorded by case managers in FSS group members’ Individual Training and Services Plan (ITSP) soon after random assignment (for most, recorded the same day as or shortly after enrol ment), interviews with FSS program administrators and case managers, and documents col ected onsite. Exhibits 7.3A to 7.3C and appendix exhibits E.4 and E.5 display the impacts on employment, earnings, and other outcomes for site clusters based on the relative emphasis of each FSS program’s (1) job search and post-employment services focus, (2) education and training focus, and (3) financial services focus.

118 Results from additional tests using conditional impacts and with additional covariates that record the interaction between subgroup characteristics and membership in the FSS group suggest that variation in impacts by PHA is only partly explained by variation in the baseline characteristics of each PHA. An additional (informal) test involving the creation of site clusters of PHAs suggested that impacts could be positive for one or more subgroups in one cluster of PHAs and negative for subgroups in a different cluster.

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 119

Chapter 7. Variation in Program Impacts

Exhibit 7.2. Program Clusters: Data and Definitions

Site Cluster Measure

Source Data

Component Measure(s) per PHA

Program Emphasis and Orientation

Emphasis on job search and Individual Training and

Proportion of initial ITSPs that listed

post-employment services

Services Plans (ITSPs)

participation in job search, self-employment

preparation, or post-employment services as

a goal or service.

Emphasis on education and Individual Training and

Proportion of initial ITSPs that listed

training

Services Plans (ITSPs)

participation in education or training as a

goal or service.

Emphasis on financial

Individual Training and

Proportion of initial ITSPs that listed

services

Services Plans (ITSPs)

participation in financial counseling or

workshops as a goal or service.

Emphasis on monitoring

Interviews with FSS

1. Average FSS caseload size.

and engagement

administrators and case

2. Expected number of communications

managers; Individual

with an FSS case manager per year.

Training and Services Plans

(ITSPs)

3. Proportion of ITSPs that included at

least one goal to be completed in Year 1.

Participant Engagement Outcomes (exploratory analysis)

FSS service use in Year 3

PHA FSS service use data

1. Proportion of FSS group members who

participated in FSS services in Year 3.

2. Proportion of FSS group members who

participated in FSS activities during 3

months or more in Year 3.

FSS = Family Self-Sufficiency. PHA = public housing agency.

Exhibits 7.3A, 7.3B, and 7.3C show the impacts of having a low, medium, or high emphasis on each type of program service. For each test, the FSS program’s impacts were estimated on average quarterly employment and average total earnings over 3 years, as measured with NDNH quarterly wage data. As these exhibits show, none of the clusters of PHAs had a statistically significant effect on quarterly employment or earnings. There was some variation of impacts among site clusters on ful -time employment, measured with survey responses, as exhibits 7.3A and 7.3B show. PHAs with medium levels of emphasis on job search and post-employment services and with low levels of emphasis on education and training increased the incidence of ful -time employment among survey respondents by relatively large margins. In contrast, FSS group survey respondents, irrespective of their PHA’s program emphasis, more often reported having a bank account and that their financial situation had improved in the past 12 months, compared with their counterparts in the control group. (See exhibits 7.3A, 7.3B, and 7.3C.)

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 120

Chapter 7. Variation in Program Impacts

Exhibit 7.3A. Impacts on Selected Outcomes, by Program Emphasis on Job Search and Post-Employment Services

$4,000

16.0

$3,352

13.8***

$3,000

12.0

9.3**

9.2* 9.8**

$2,000

8.0**

7.4**

8.0

4.8

4.1*

$1,000

3.8

4.0

1.7 0.4 0.1

0.5 1.1

0

0.0

-0.3

-$1,000 -$639

-4.0 Percentage Points

-$2,000

-$1,638

-8.0

Total

Average

Employed

Has Bank

Reports

Enrolled

-$3,000

Earnings

Quarterly

in

Account

Better

in HCV

-12.0

in Years

Employment

Month 36

in

Financial

Program

1 to 3

Rate in

with

Month 36

Situation

in

-$4,000

Years

Full-Time

in

Month 36

-16.0

1 to 3

Hours

Month 36

Than a

Year Ago

Low Emphasis

Medium Emphasis

High Emphasis

FSS = Family Self-Sufficiency. HCV = Housing Choice Voucher.

NOTES: The FSS 36-Month Survey respondent sample includes HCV heads of household who were randomly assigned between October 18, 2013, and December 22, 2014, were age 18 to 61 at the time of random assignment and responded to the FSS 36-Month Survey. Estimates were regression-adjusted using ordinary least squares, controlling for differences in sample member characteristics recorded at the time of random assignment. No special weights were applied to responses to adjust for differences in sample size by housing authority. A two-tailed t-test was applied to differences between the FSS group and the control group for continuous variables and selected outcomes expressed as proportions. In addition, a chi-square test for categorical variables was run to determine whether there is a difference in the distribution of related outcomes by research group. Statistical significance levels are indicated as: * = 10

percent; ** = 5 percent; *** = 1 percent. The H-statistic test was used to test for statistically significant differences in impact estimates across different subgroups. Statistical significance levels are indicated as: † = 10 percent; †† = 5 percent; and ††† = 1 percent.

SOURCES: Baseline Information Form data, HUD Inventory Management System/Office of Public and Indian Housing Information Center data, and responses to the FSS 36-Month Survey.

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 121

Chapter 7. Variation in Program Impacts

Exhibit 7.3B. Impacts on Selected Outcomes, by Program Emphasis on Education

and Training

$2,500

20.0

18.7***

$2,000 $1,826

15.0

$1,500

10.0

$1,071

8.3**

$1,000

7.6**

6.9

6.3

6.2**

5.0

$500

3.5

3.4

3.4

3.2

2.2

2.5

0

0.0

-0.1

Has Bank

Reports

-1.6

-$500

-2.5

Account

Better

in

Financial

-5.0 Percentage Points

Average

Employed

Month 36

Situation

Enrolled

-$1,000

Quarterly

in

in

in HCV

Employment

Month 36

Month 36

Program

Than a

-10.0

Rate in

with

in

-$1,500

Years

Full-Time

Year Ago

Month 36

1 to 3†

Hours

-15.0

-$2,000

-$1,968

-$2,500

Total

-20.0

Earnings

in Years

1 to 3

Low Emphasis

Medium Emphasis

High Emphasis

FSS = Family Self-Sufficiency. HCV = Housing Choice Voucher.

NOTES: The FSS 36-Month Survey respondent sample includes HCV heads of household who were randomly assigned between October 18, 2013, and December 22, 2014, were age 18 to 61 at the time of random assignment and responded to the FSS 36-Month Survey. Estimates were regression-adjusted using ordinary least squares, controlling for differences in sample member characteristics recorded at the time of random assignment. No special weights were applied to responses to adjust for differences in sample size by housing authority. A two-tailed t-test was applied to differences between the FSS group and the control group for continuous variables and selected outcomes expressed as proportions. In addition, a chi-square test for categorical variables was run to determine whether there is a difference in the distribution of related outcomes by research group. Statistical significance levels are indicated as: * = 10

percent; ** = 5 percent; *** = 1 percent. The H-statistic test was used to test for statistically significant differences in impact estimates across different subgroups. Statistical significance levels are indicated as: † = 10 percent; †† = 5 percent; and ††† = 1 percent.

SOURCES: Baseline Information Form data, HUD Inventory Management System/Office of Public and Indian Housing Information Center data, and responses to the FSS 36-Month Survey.

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 122

Chapter 7. Variation in Program Impacts

Exhibit 7.3C. Impacts on Selected Outcomes, by Program Emphasis on Financial Services $2,000

$1,889

16.0

14.8***

$1,500

12.0

8.3

$1,000

7.2***

8.0

6.4

4.1

4.4

$500

3.0

3.2**

4.0

$212

2.2

2.5

1.5

1.7 1.6

0

0.0

-0.6 -1.2

-$500

-4.0

Average

Employed

Has Bank

Reports

Enrolled

Percentage Points

Quarterly

in

Account

a Better

in HCV

-$1,000

Employment

Month 36

in

Financial

Program

-8.0

Rate in

with

Month 36

Situation

in

Years

Full-Time

in

Month 36

-$1,500

1 to 3

Hours

Month 36

-12.0

Than a

Year Ago

-$2,000

-16.0

-$2,287

-$2,500

-20.0

Total

Earnings

in Years

1 to 3

Low Emphasis

Medium Emphasis

High Emphasis

FSS = Family Self-Sufficiency. HCV = Housing Choice Voucher.

NOTES: The FSS 36-Month Survey respondent sample includes HCV heads of household who were randomly assigned between October 18, 2013, and December 22, 2014, were age 18 to 61 at the time of random assignment and responded to the FSS 36-Month Survey. Estimates were regression-adjusted using ordinary least squares, controlling for differences in sample member characteristics recorded at the time of random assignment. No special weights were applied to responses to adjust for differences in sample size by housing authority. A two-tailed t-test was applied to differences between the FSS group and the control group for continuous variables and selected outcomes expressed as proportions. In addition, a chi-square test for categorical variables was run to determine whether there is a difference in the distribution of related outcomes by research group. Statistical significance levels are indicated as: * = 10

percent; ** = 5 percent; *** = 1 percent. The H-statistic test was used to test for statistically significant differences in impact estimates across different subgroups. Statistical significance levels are indicated as: † = 10 percent; †† = 5 percent; and ††† = 1 percent.

SOURCES: Baseline Information Form data, HUD Inventory Management System/Office of Public and Indian Housing Information Center data, and responses to the FSS 36-Month Survey.

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 123

Chapter 7. Variation in Program Impacts

Exhibit 7.3D. Impacts on Selected Outcomes, by Program Emphasis on Monitoring and Engagement

$10,000

12.0

9.9**

10.0

$8,000

8.9**

7.5**

7.7

8.0

$6,000

6.6**

6.3* 5.4*

5.5

6.0

$4,000

4.0

$1,821

3.2

2.7

$2,000 $1,740

1.5

2.0

1.3

0.8

0

0.0

Has Bank

Reports

Enrolled

Account

Better

in HCV

-2.0

-$2,000

in

Financial

Program

Percentage Points

Month 36

Situation

in

-4.0

-$4,000

-3.8

in

Month 36

Month 36

Average

-5.7

Than a

-6.0

-$6,000

Quarterly

Year Ago

Employment

Employed

-8.0

Rate in

in

-$8,000

Years

Month 36

1 to 3

-10.0

-$8,006***

with

Full-Time

-$10,000

Total

Hours†

-12.0

Earnings

in Years

1 to 3†††

Low Emphasis

Medium Emphasis

High Emphasis

FSS = Family Self-Sufficiency. HCV = Housing Choice Voucher.

NOTES: The FSS 36-Month Survey respondent sample includes HCV heads of household who were randomly assigned between October 18, 2013, and December 22, 2014, were age 18 to 61 at the time of random assignment and responded to the FSS 36-Month Survey. Estimates were regression-adjusted using ordinary least squares, controlling for differences in sample member characteristics recorded at the time of random assignment. No special weights were applied to responses to adjust for differences in sample size by housing authority. A two-tailed t-test was applied to differences between the FSS group and the control group for continuous variables and selected outcomes expressed as proportions. In addition, a chi-square test for categorical variables was run to determine whether there is a difference in the distribution of related outcomes by research group. Statistical significance levels are indicated as: * = 10

percent; ** = 5 percent; *** = 1 percent. The H-statistic test was used to test for statistically significant differences in impact estimates across different subgroups. Statistical significance levels are indicated as: † = 10 percent; †† = 5 percent; and ††† = 1 percent.

SOURCES: Baseline Information Form data, HUD Inventory Management System/Office of Public and Indian Housing Information Center data, and responses to the FSS 36-Month Survey.

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 124

Chapter 7. Variation in Program Impacts

Program Clusters Based on Site Monitoring and Engagement Practices

Another test of variation in program implementation features, also presented in Verma et al. (2019), concerns the measure of how strongly administrators and case managers in PHAs emphasizes monitoring and engagement with FSS group members. As discussed in previous chapters, the data used to group PHAs into low-emphasis, medium-emphasis, and high-emphasis site clusters are based on the PHAs’ FSS program implementation features and practices at study launch—caseload sizes, expectations about the frequency of contacts, and focus on establishing short-term goals. Sites classified as having a high or strong emphasis on monitoring and engagement have smal er caseload sizes, expect FSS group members and case managers to have more frequent contacts and focus on establishing short-term goals. As discussed in chapter 3, PHAs that ranked high on this measure tended to have relatively high participation rates for the FSS group, although differences in participation with the control group were not especially large. In addition, some previous studies that analyzed links between program implementation practices and impacts on employment and earnings found that programs that ranked high on the studies’ version of a monitoring and engagement indicator did not lead to statistically significant increases in employment and earnings.119

Appendix exhibit E.4 shows the PHAs in the monitoring and engagement clusters. From 61 to 65 percent of control group members were employed in any given quarter during Years 1 to 3. Average total earnings for control group members ranged from about $39,000 to $49,000, with the highest average earned by control group members in housing agencies with the strongest emphasis on monitoring and engagement.

This variation in earnings is also reflected in the differences among the clusters of housing agencies in the proportion of control group members who averaged more than $25,000 per year in earnings.

For outcomes calculated with NDNH quarterly wage data, the FSS program did not affect employment or earnings levels in housing agencies with a low or medium emphasis on monitoring and engagement. In addition, on average, FSS group members in all three types of programs earned about the same amount (about $40,000) during Years 1 to 3. FSS programs with the strongest emphasis on monitoring and engagement, however, led to a relatively large and statistically significant decrease in total earnings (of about $8,000) compared with the control group. Similarly, by a margin of nearly 7 percentage points, a larger proportion of control group members than FSS group members earned more than $25,000 per year. Not shown in any exhibit, the FSS programs with a high emphasis on monitoring and engagement led to an increase of similar magnitude in the proportion of study participants with low annual earnings (up to $10,000). This pattern of earnings differences suggests that some FSS group members were working part time or working intermittently, whereas control group members were more likely to work ful -time hours or in jobs with more weeks or months of employment. This conjecture is supported by findings from survey data. As exhibit 7.3D shows, the FSS

programs with low or medium emphasis on monitoring and engagement led to increases in employment with ful -time hours at 36 months, but programs with a high emphasis on monitoring and engagement did not.

Additional findings based on survey data show that FSS programs with a strong emphasis on monitoring and engagement did not lead to better outcomes in other measures of financial or personal wel -being compared with other types of programs. As exhibit 7.3D shows, FSS programs with a high emphasis on monitoring and engagement led to relatively large and positive impacts on having a savings or checking account, but other types of FSS programs also led to increases in this outcome that were nearly as large. Likewise, all three types of programs increased the proportion of study participants who assessed their financial situation as better than the previous year, but only the largest increase, recorded for programs with a low emphasis on monitoring and engagement, is statistically significant.

119 See Bloom, Hill, and Riccio (2001: 40–42).

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 125

Chapter 7. Variation in Program Impacts

Supplemental Exploratory Analyses

So far, results from cross-site variation analysis suggest that some FSS programs led to effects on total earnings in Years 1 to 3 that vary from negative to positive. In addition, the analyses of the variation of impacts by program implementation clusters suggest that FSS programs with a strong emphasis on monitoring and engagement led to negative impacts on total earnings over 3 years. So far, the analyses reviewed above did not suggest reasons why any type of FSS program would lead to positive impacts on employment and earnings.

Additional supplemental analyses of program effects have been conducted to see whether they may offer additional clues for exploring this issue. Appendix exhibit E.7 shows the impact results for two additional site clusters that are created with data col ected after random assignment. The first measure uses PHA averages on two indicators of FSS service use during Year 3, and the second measure uses PHA averages on two indicators of escrow accrual during Year 3. For each measure, PHA averages are combined into a single score, which is then used to group PHAs into “low,” “medium,” and “high” categories. Thus, an FSS program (PHA) with a high score for the composite measure of service use would have recorded a greater incidence of service use in Year 3 compared with the average for all PHAs. Similarly, an FSS program (PHA) with a high score for the composite measure of escrow accrual would have recorded a greater incidence of having a positive balance in Month 36 compared with the average for all PHAs.120 These measures should be seen as exploratory because they are based on outcomes that occurred after random assignment. They are also tentative because data for future reports will cover Years 4 through 6 of fol owup and could change the rankings of the PHAs.

As discussed in chapter 3, FSS programs that strongly emphasized monitoring and engagement had relatively high rates of service use. As a result, the pattern of impacts for PHAs clustered on the service-use measure resembles the pattern for PHAs clustered by their emphasis on monitoring and engagement. For example, during Years 1 to 3, the PHAs with high scores on service use led to an average decrease in total earnings of about $4,000 compared with the control group. (See appendix exhibit E.7. This decrease could potentially be reversed after FSS group members complete their participation in education and training or other services and start employment.)

Appendix exhibit E.7 shows a much different pattern of impacts for the site cluster based on escrow accrual.

During Years 1 to 3, FSS programs (PHAs) with lower-than-average levels of escrow accrual led to a decrease in total earnings of about $3,200 compared with the control group, whereas FSS programs with higher-than-average levels of escrow accrual led to an increase in total earnings of about $3,750. (The variation in impacts on total earnings is statistically significant.)

Thus, a tentative explanation for site-level variation in impacts in total earnings could point to FSS programs with a strong emphasis on monitoring and engagement leading to negative impacts through the end of Year 3 and FSS programs with a high incidence of escrow accrual leading to positive impacts. There are several outstanding questions concerning this finding:

• Can PHAs have a high emphasis on monitoring and engagement and a high incidence of indicators of escrow accrual?

Among the 18 PHAs in the FSS evaluation, different PHAs ranked high in their emphasis on monitoring and engagement or in participants’ escrow accrual—there is no overlap. This variation is not inevitable, although possibly the tendency of programs with a high emphasis on monitoring and engagement to boost service use could (at least temporarily) decrease employment levels and therefore escrow accrual.

120 It is reasonable to assert that FSS programs with a higher-than-average incidence of escrow accrual have a greater likelihood of eventually attaining a higher-than-average graduation rate compared with other FSS programs. These programs also have a greater likelihood of having FSS group members who increased their earnings over time to a greater extent than other FSS group members.

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 126

Chapter 7. Variation in Program Impacts

• Do PHAs have high rates of escrow accrual because they enroll larger proportions of subgroups (such as HCV heads of household with part-time employment) who are most likely to accrue escrow?

• Does a high rate of escrow accrual simply reflect the results of other program implementation features that led to greater job finding and earnings advancement?

• Alternatively, are FSS programs with higher-than-average escrow accrual taking steps to increase the incidence or the amount of escrow accrued?

There is some evidence that variation in escrow accrual at least partly reflects differences in implementation practices among PHAs in the FSS evaluation. First, a conditional test of escrow accrual rates showed that al but one of the PHAs in the “high escrow accrual” group would remain in the high group after control ing for differences in sample member characteristics. Second, it appears that PHAs vary in the likelihood of accumulating escrow, even when control ing for differences in the incidence of employment and earnings increases. To test this hypothesis, an approximate “escrow coverage” rate for each PHA was calculated by dividing the number of FSS group members with a positive balance in Month 36 by the number with an increase in household earnings since random assignment, as recorded from federal HUD administrative data.121 As of the end of Year 3, PHAs in the high escrow accrual cluster had an escrow coverage rate of a little over 60 percent, whereas PHAs in the low escrow accrual cluster had an escrow coverage rate of a little under 40 percent (not shown in the exhibit). Correspondingly, among FSS group members with increased earnings, PHAs in the low escrow accrual cluster had a higher incidence of exiting from FSS and a higher incidence of continued enrol ment in the FSS program, but with a zero balance in their escrow account in Month 36 (not shown in the exhibit). Third, PHAs in the high escrow cluster appear to be using a case management approach that features moderate levels of monitoring and low levels of engagement for employed FSS group members.

During each year of fol owup, the PHAs in the high escrow cluster averaged the highest incidence of program-recorded employment but no participation in FSS activities. Yet, during Year 3, FSS group members in PHAs in the high escrow accrual group had a higher incidence of having at least one contact with an FSS case manager than their counterparts in PHAs in the low escrow accrual group (not shown in the exhibit). It may therefore be inferred that FSS case managers in PHAs in the high escrow cluster often knew about FSS group members’ employment status and were wil ing to tailor their program participant goals.

Conclusion

As discussed in chapter 3, for this evaluation of 18 FSS programs, housing agencies with a relatively strong emphasis on monitoring and engagement did not have the highest rates of escrow accrual. That finding is not inevitable and could conceivably change in later years as PHAs continue helping participants graduate from the program. Nonetheless, it is reasonable to hypothesize that some aspects of program implementation in the high monitoring and engagement housing agencies led to lower levels of earnings (at least temporarily) compared with the control group and more negative impacts compared with housing agencies with high rates of escrow accrual. Possibly, the FSS programs with high levels of monitoring and engagement are encouraging some participants to invest more of their time to participate in FSS-related activities, including longer term education and training, whereas other FSS programs are more likely to let employed participants continue working without an expectation of additional participation in education and training, financial counseling, or other FSS activities. In time, FSS group members who cut back on their hours or weeks of employment to attend education and training courses or receive other services may complete their attendance and move 121 This measure is approximate because some FSS group members could be ineligible to accrue escrow, even with an earnings increase—for example, because of unpaid rent.

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 127

Chapter 7. Variation in Program Impacts

into ful -time employment, possibly with higher earnings commensurate with their newly earned academic or occupational credentials. FSS group members who complete attendance at financial security workshops or counseling sessions may also realize other benefits, for example, from higher credit scores. These questions wil be explored further in later reports.

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 128

Chapter 8. Summary and Looking Forward

Chapter 8. Summary and

Looking Forward

This report updates and expands upon MDRC’s first report examining the implementation and early effects of the FSS program for housing voucher (HCV) recipients. These reports are part of the national evaluation of the FSS program sponsored by HUD. FSS programs offer participants referrals to a broad array of services, plus access to a long-term escrow account, designed to work together to promote earnings growth and financial self-sufficiency over the 5-year program period. At program enrol ment, participants work with case managers to set goals and action steps to achieve them. Most FSS participants commit to at least one employment-related goal, along with other goals related to financial security, education, or homeownership. The evaluation results, covering 3 years of the 5-year program, provide a glimpse of the outcomes and effects at about the midway point. So far, through the end of Year 3, the evidence appears mixed: while FSS appears to encourage higher rates of participation in self-sufficiency activities, as yet it has had limited effects on increasing employment or earnings or improving other aspects of financial wel -being compared with outcomes for a control group without access to FSS services or financial incentives. Given these are interim results, it is too soon to make a firm assessment about the program’s eventual effectiveness. Participants have another 2 years (and longer for those receiving extensions) to attain their FSS goals, and new patterns of outcomes and effects may emerge over the remaining fol owup period. Nevertheless, the interim results reveal some important questions and patterns. These insights and observations from the interim report, and how they will be further explored over the remainder of the evaluation, are the focus of this chapter.

What the Interim Findings Suggest

Program engagement is beginning to taper off. FSS is a voluntary program that requires long-term commitment from participants. Across all housing authorities, FSS group members averaged fewer than two interactions with the coordinator or case manager, equivalent to communications occurring every 8 or 9

months—or once every 6 months among FSS group members who are still enrol ed in the program (this is lower than the quarterly or monthly expectations set by most sites, and less frequent than during the first year of fol owup). The declining program contacts and service-use patterns described in this report for the FSS

group look similar to those observed in other voluntary employment-focused programs, consisting of a higher involvement, primarily in short-term job search or financial security activities, in the first year or so after enrol ment, fol owed by a drop-off in engagement.

By the end of Year 3, nearly 40 percent of FSS group members had exited the program. Voluntary withdrawals from FSS, moves to other PHAs, and terminations from the HCV program accounted for most exits. By the end of Year 3, only 4 percent of FSS group members had graduated from the program. Further, among study participants who were still officially enrol ed in FSS, about one-third did not appear to be engaged in some type of FSS-related activity or employed, per records maintained by the program.

When interviewed, most FSS administrators and case managers cited two factors that strongly influenced participants’ engagement and how much progress they make toward achieving their goals. The first is participants’ internal motivation to succeed. One case manager noted that a PHA could have the best resources and the best coordinators, but unless clients are motivated to improve their own situation, they wil not succeed. The second factor staff members cited was the case managers’ skil s in motivating clients. Staff members recognized that participants deal with things that “many people have never had to go through,” but Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 129

Chapter 8. Summary and Looking Forward

coordinators are able to work with them and keep them focused on moving toward their goals. Staff placing escrow as a central element of the FSS self-sufficiency strategy cited that the opportunity to make a down payment on a home, to get out of debt, or to support children should be motivating in and of itself.

The ongoing evaluation will continue to examine patterns of engagement for those who remain enrol ed in FSS and how the levels and types of engagement observed so far could result in the types of outcomes intended for the FSS program. How strongly will participants engage with the program in future years? Will they maintain meaningful contact with their case managers, build credentials and work history, and accrue escrow?

Also, what strategies will FSS administrators and coordinators use to keep participants engaged and working toward graduation? Hypothetically, as participants continue with the program, they may require less intense support from case managers and may experience tangible signs of progress, such as attainment of educational or vocational credentials, improved credit scores and other improvements in household finances, increased earnings, and accumulation of funds in an escrow account. In turn, it is hoped that such progress will motivate participants to stay connected with the program, eventually leading to graduation and escrow receipt.

Escrow credits appears to have plateaued, but families who have accumulated escrow have sizeable balances. At the end of Year 1, nearly one-fourth of the FSS group had accrued some escrow. Over the next year, additional FSS group members began accruing escrow, and by the end of Year 2, the proportion with escrow dol ars in their account had increased to 39 percent. By the end of Year 3, the proportion of FSS

group members with any escrow accrual remained at around 40 percent. Relatively few additional FSS group members started accruing escrow during Year 3, and about the same number of FSS group members who had previously accrued escrow left the program and forfeited their escrow. Those with a positive balance at the end of Year 3 had an average of nearly $3,700 in their account (about $1,450 among all FSS group members), and about one-fourth of these participants had accrued more than $5,000. On average, FSS group members who were underemployed at random assignment—that is, who were working part-time or receiving relatively low annual earnings—appear more likely to have accrued escrow compared with subgroups who were working ful -time or receiving relatively high earnings. As of the end of Year 3, about 20 percent of the FSS group stil enrol ed had no escrow balance. Questions remain about the escrow accrual over the remainder of the FSS

contract. Will those who have yet to start accruing escrow begin doing so during the remaining years of their FSS contract? Will the prospect of graduation and escrow disbursement encourage FSS participants with a positive balance to maintain their employment, continue accruing escrow credits, and qualify for graduation at the end of Year 5? Who is most likely to earn an escrow disbursement?

Participants may pursue different pathways toward economic self-sufficiency, but program effects on employment and earnings are yet to emerge midway through the program. The report shows high levels of employment for both study groups. In the first year of fol owup, soon after participants enrol ed in the study, the FSS group’s employment rate was around 73 percent and the control group’s rate was around 72 percent.

These percentages are not statistically significantly different from each other, suggesting that the program did not have an impact on employment in the early stages. This pattern is evident for each year of fol owup, with program and control group members showing fairly high levels of attachment to the labor force (quarterly employment rates are somewhat lower, reflecting employment churning). Participation in FSS may have led to more months of ful -time employment in the prior year, as measured by the 36-month survey. The subgroup analysis, focusing on baseline employment and education statuses, also does not point to clear differences in impacts for these groups. There is some suggestive evidence that FSS may have facilitated more disadvantaged study participants to realize slightly better employment outcomes (more hours of work but not higher earnings) compared with the control group. These impacts, however, are not statistically significant across groups.

As participants near graduation, will their employment trajectories diverge from those of their control Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 130

Chapter 8. Summary and Looking Forward

group counterparts? Will the incentive to claim their escrow accruals influence their work patterns during the remainder of their enrol ment in FSS? These and related questions will be monitored as the evaluation proceeds.

The absence of consistent program effects on employment and earnings may leave little room for spil over effects on other domains of wel -being. An intervention designed to help families make progress toward self-sufficiency through increased earnings and reduced reliance on benefits should eventually lead to changes in receipt of government benefits, such as Temporary Assistance for Needy Families program benefits, Supplemental Nutrition Assistance Program benefits, and housing assistance. For the most part, the FSS group respondents reported patterns of income receipt and average income levels that resemble those of the control group. By the end of Year 3, according to survey responses, FSS led to a small and statistically significant decrease (of 2 percentage points) in the receipt of TANF cash welfare benefits. FSS households are required to go off TANF for at least 12 months leading up to graduation. It is possible that this pattern may hold as FSS group members approach the end of their FSS contract. The evaluation is not col ecting TANF agency administrative records, so this pattern will not be verified as part of this study.

A self-sufficiency program should also demonstrate improvements in other aspects of families’ wel -being. At the 3-year mark (a period that precedes the receipt of escrow for most of those who will graduate and earn disbursements), the program is showing a few notable improvements in financial wel -being or reductions in the incidence of material hardships or food insecurity. In addition, some of the observed differences between research groups appear to be inconsistent. The FSS program was effective in increasing participants’

connections to mainstream financial institutions, and participants were more likely to be aware of their credit and reported higher credit scores, compared with the control group. FSS group members, on average, however, incurred greater debt (it is unclear what is driving the higher debt levels). Consistent with programs that emphasize the use of financial security and homeownership preparation counseling and workshops, FSS

led to positive effects on attitudinal and perceived wel -being outcomes. FSS group members, when surveyed, were more likely than control group respondents to assess that they had improved their financial situation in the past year and were better able to plan for the future.

Impacts on some financial security outcomes may occur without increases in earnings or in household income. The FSS program led to a large increase in attendance at financial security workshops and counseling. So far, the FSS program has not increased nonretirement savings above control group levels (not counting the escrow account as potential savings); nor has it reduced average debt levels. Nonetheless, FSS group members’ greater access to financial security–related services (and to homeownership preparation services) could lead to other positive effects, especially if FSS group members continue to use traditional financial instruments more often than control group members. Most important, FSS could lead to increases in average credit scores above control group levels, which can positively affect a range of financial outcomes, including access to credit and, in some instances, likelihood of employment. Future reports and papers wil address this issue in more detail.

Housing subsidy receipt rates and subsidy levels are almost indistinguishable between the two study groups. FSS participants are subject to the same rent policies as non-FSS voucher recipients. However, the housing authority credits an interest-bearing escrow account for FSS participants based on increases in household earned income during the term of the FSS contract. In this way, escrow builds “forced savings,” a potentially sizeable reward once FSS goals are achieved. Three years out, a slightly larger proportion of FSS

group households than control group households remained enrol ed in the HCV program. During Month 36, FSS group members also averaged slightly higher out-of-pocket expenses (of $19) for rent and utilities and lower amounts of housing subsidies. However, the differences in total housing subsidies over 3 years and in Year 3

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 131

Chapter 8. Summary and Looking Forward

are not statistically significant. The evaluation will continue to examine the long-term interaction between employment and housing outcomes. But what will happen to FSS group members who graduate from the program and receive a full escrow disbursement? Possibly, some FSS graduates will leave housing assistance, either to move to private rental housing without a subsidy or to use their escrow disbursement to purchase a home. If so, the public housing agency will be able to assist new households. Thus, FSS may lead to long-term effects on housing. Later analysis of HUD administrative data and responses to an exit survey, slated for 2020, will address these questions.

There is variation across housing agencies. A central goal of the evaluation is to examine whether variation in program implementation approaches affects program outcomes, and whether some implementation approaches are associated with positive outcomes. Site clusters were used to explore these patterns of variation by program approach. The analyses show some intriguing—and puzzling—findings.

PHAs with stronger monitoring and engagement practices (smal er caseload sizes, an expectation of more frequent contacts, and a focus on establishing short-term goals) tended to have higher rates of exit from the FSS program, compared with programs without these attributes, but they also worked more intensively with FSS group members who remained in the program. The housing agencies with a relatively strong emphasis on monitoring and engagement, however, did not have the highest rates of escrow accrual. Further, based on the employment and earnings patterns seen across these clusters of housing agencies, it is reasonable to hypothesize that some aspects of program implementation in the high monitoring and engagement housing agencies lead to lower levels of earnings (at least temporarily) compared with the control group and more negative impacts compared with other housing agencies. Possibly, the FSS programs with high levels of monitoring and engagement are encouraging some participants to invest more of their time to participate in FSS-related activities, including longer term education and training, whereas other FSS programs are more likely to let employed participants continue working without the expectation of additional participation in education and training, financial counseling, or other FSS activities. These questions and other exploratory analyses related to variation will be examined in future reports.

The Evaluation Going Forward

The FSS evaluation is slated to end in 2021, providing 6 to 7 years of fol owup for the study sample. This length of fol owup al ows the evaluation to track participants through the end of their 5-year FSS contracts and examine post-FSS circumstances for those who do not receive extensions (programs can extend participation by 2 additional years, for a total of up to 7 years, for participants who are deemed on track to graduate and earn escrow). Two additional reports will be published during this time. In addition to updating many of the administrative data-based results (for example, employment, earning, escrow, and housing subsidy) included in this report, future deliverables will also be informed by: an additional round of field research with program staff and a small number of FSS participants; a formal fol owup survey, conducted with both FSS and control group members, to help estimate the program’s longer-term effects on a range of economic and material wel -being outcomes after the end of the FSS program for most participants; and an analysis of financial behaviors using credit data. The longer timeframe and more comprehensive data will support a more complete assessment of the FSS program’s effects. It will also provide the information needed to assess whether—or the ways in which—the program may need to be strengthened to help support families in making progress toward self-sufficiency.

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 132

Appendix A

Appendix A

Appendix Exhibit A.1. Baseline Characteristics of Households in the Family Self-Sufficiency Impact Sample by Research Group

FSS

Control

Characteristic

Group

Group

Total

Average number of household membersa

3.2

3.2

3.2

Average number of adults in householda

1.5

1.4

1.5

Households with more than one adult (%)

35.3

32.0

33.7*

Average number of children in household

0

22.8

24.8

23.8

1

24.2

21.1

22.7

2

24.1

25.3

24.7

3 or more

28.9

28.8

28.8

For households with children, age of youngest child (%)

0–2 years

21.0

20.6

20.8

3–5 years

19.5

21.3

20.4

6–12 years

42.1

40.5

41.3

13–17 years

17.4

17.5

17.5

Primary language spoken at home is English (%)

91.9

92.4

92.2

Receives TANF (%)

16.3

15.3

15.8

Receives food stamps/SNAP (%)

71.2

67.9

69.6*

Length of time receiving Section 8 Housing Choice Voucher (%)

Less than 1 year

5.0

4.9

5.0

1–3 years

27.2

28.1

27.6

4–6 years

21.9

21.3

21.6

7–9 years

15.2

15.3

15.2

10 years or more

30.7

30.4

30.6

Total household income (%)

$0

4.4

4.7

4.5

$1–$4,999

18.0

16.1

17.0

$5,000–$9,999

18.9

18.5

18.7

$10,000–$14,999

18.4

16.9

17.6

$15,000–$19,999

14.1

14.4

14.3

$20,000–$24,999

10.3

12.0

11.2

(continued)

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 133

Appendix A

Appendix Exhibit A.1. Baseline Characteristics of Households in the Family

Self-Sufficiency Impact Sample by Research Group continued

FSS

Control

Characteristic

Group

Group

Total

$25,000–$29,999

7.8

8.5

8.2

$30,000 or more

8.1

9.0

8.5

Payment for rent and utilities (%)

$0

2.1

1.7

1.9

$1–$99

5.6

5.4

5.5

$100–$199

10.4

8.7

9.5

$200–$299

11.1

11.4

11.2

$300–$399

13.1

12.9

13.0

$400–$499

11.1

10.0

10.5

$500–$599

10.4

11.2

10.8

$600–$699

7.6

8.1

7.8

$700–$799

7.0

7.6

7.3

$800–$899

5.2

5.9

5.5

$900–$999

2.9

3.6

3.3

$1,000 or more

13.7

13.5

13.6

During the past 12 months, household experienced at least one

financial hardship (%)

57.7

60.4

59.0

Not able to buy prescription drug

13.0

13.6

13.3

Not able to buy food

26.2

31.6

28.9***

Not able to pay telephone bil

28.7

27.8

28.2

Not able to pay rent

17.9

19.1

18.5

Not able to pay utility bil

43.5

43.3

43.4

Sample size

1,285

1,271

2,556

FSS = Family Self-Sufficiency. SNAP = Supplemental Nutrition Assistance Program. TANF = Temporary Assistance for Needy Families.

aMaximum response option for number of adults in a household is four.

NOTES: The FSS impact sample includes Housing Choice Voucher households that were randomly assigned between October 18, 2013, and December 22, 2014, and with a head of household age 18 to 61 at the time of random assignment. Sample sizes for specific measures may vary because of missing values. A chi-square test for categorical variables and a t-test for continuous variables were run to determine whether there is a difference in the distribution of the characteristics by research group. Statistical significance levels are indicated as: * = 10 percent; ** = 5 percent; and *** = 1 percent. Rounding may cause slight discrepancies in calculating sums and differences. Detail may sum to more than total for questions that allow more than one response.

SOURCE: MDRC calculations from Baseline Information Form data.

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 134

Appendix A

Appendix Exhibit A.2. Baseline Characteristics of Heads of Households in the Family Self-Sufficiency Impact Sample by Research Group

FSS

Control

Characteristic

Group

Group

Total

Sample member characteristics

Female (%)

91.4

89.8

90.6

Age (%)

19–24 years

1.7

2.7

2.2

25–34 years

34.0

33.8

33.9

35–44 years

36.6

34.5

35.6

45–59 years

26.7

27.9

27.3

60–61 years

1.0

1.2

1.1

Average age (years)

39

39

39

Marital status (%)

Married, living with spouse

8.3

7.2

7.7

Married, not living with spouse

7.1

6.6

6.8

Cohabitating

1.5

1.3

1.4

Single, widowed, or divorced

83.1

84.9

84.0

Citizenship status (%)

U.S.-born

87.6

87.8

87.7

Naturalized

8.0

8.1

8.1

Non-citizen

4.4

4.0

4.2

Race/ethnicity (%)a

Black, non-Hispanic/Latino

72.2

74.4

73.3

Hispanic/Latino

15.6

16.1

15.8

White, non-Hispanic/Latino

7.1

6.2

6.7

Native American

0.3

0.3

0.3

Asian

2.7

1.3

2.0

Other

0.4

0.3

0.4

Multi-racial

1.7

1.4

1.6

(continued)

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 135

Appendix A

Appendix Exhibit A.2. Baseline Characteristics of Heads of Households in the Family Self-Sufficiency Impact Sample by Research Group continued

FSS

Control

Characteristic

Group

Group

Total

Education

Highest degree or diploma earned (%)

GED certificate

3.5

2.5

3.0

High school diploma

10.5

10.8

10.6

Some col ege or received technical/trade license

53.9

56.2

55.0

Associate’s or 2-year col ege degree

11.3

10.2

10.8

4-year col ege or graduate degree

7.1

5.9

6.5

None of the above

13.6

14.4

14.0

Has trade license or training certificate (%)

47.1

47.0

47.0

Employment status

Currently employed (%)

55.6

56.9

56.2

Regular job

47.7

49.2

48.4***

Self-employed

3.4

5.1

4.2***

Temporary or seasonal job

4.5

2.5

3.5***

Currently working 35 hours or more per week (%)

29.1

32.0

30.5

Average hours worked per week

17.9

18.8

18.3

Average weekly earnings ($)

205

221

213

Barriers to employment

Has any problem that limits work (%)

41.6

40.8

41.2

Physical health

18.4

19.3

18.8

Emotional or mental health

7.6

7.7

7.6

Childcare access or cost

18.5

17.2

17.8

Need to care for disabled household member

7.9

6.7

7.3

Previously convicted of a felony

6.7

6.0

6.3

Limited English-speaking ability (%)

3.6

4.1

3.8

Does not have access to transportation for employment (%)

No public transportation access

16.5

19.1

17.8*

No automobile access

18.5

17.9

18.2

(continued)

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 136

Appendix A

Appendix Exhibit A.2. Baseline Characteristics of Heads of Households in the Family Self-Sufficiency Impact Sample by Research Group continued

FSS

Control

Characteristic

Group

Group

Total

FSS program

Heard of escrow before random assignment (%)

43.9

44.2

44.0

Interest in FSS services related to (%)

Job-related services

70.4

70.6

70.5

Social services

32.2

32.7

32.4

Financial services

95.4

95.5

95.5

Sample size

1,285

1,271

2,556

FSS = Family Self-Sufficiency. GED = General Educational Development.

aChi-square test may not be valid due to small cell sizes.

NOTES: The FSS impact sample includes Housing Choice Voucher households that were randomly assigned between October 18, 2013, and December 22, 2014, and with a head of household age 18 to 61 at the time of random assignment. Sample sizes for specific measures may vary because of missing values. A chi-square test for categorical variables and a t-test for continuous variables were run to determine whether there is a difference in the distribution of the characteristics by research group. Statistical significance levels are indicated as: * = 10 percent; ** = 5 percent; and *** = 1 percent. Rounding may cause slight discrepancies in calculating sums and differences. Detail may sum to more than 100 percent for questions that allow more than one response.

SOURCE: MDRC calculations from Baseline Information Form data.

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 137

Appendix A

Appendix Exhibit A.3. Staffing, Escrow, and Graduation Policies

Case Management Staffing

Escrow

Graduation Requirementsa

At Least

Home-

One Annual

Interim

Number

HCV

ownership

In-Person

Escrow

Limitations

Housing

with FSS

Responsi-

Responsi-

Meeting

Disbursement

Withdrawal

Employment

Employment

Employment on Revising

Authority Caseload

bilities

bilities

Expected

Permitted

Limits

Hours

Earnings

Stability

Goals

Housing

5 or more

1 year

Authority

(including

No

Yes

Yes

Yes

Up to 50% of the

30 hours per

(final goal);

#1

supervisor; if

balance

week

--

6 months

6 months

excluding, 4)

(interim goals)

Housing

Authority

1–4

Yes

No

Yes

Yes

Up to 25% of the

#2

balance

--

--

--

6 months

Housing

Up to 50% of the

Authority

1–4

No

No

No

Yes

balance (up to 2

--

--

--

1 year

#3

disbursements)

Housing

Authority

1–4

Yes

Yes

Yes

No

N/A

--

At local minimum

#4

wage or more

--

6 months

Housing

Wages must make

Authority

5 or more

Yes

No

No

Yes

None

--

clients self-sufficient

--

--

#5

from HCV (zero

HAP)

Housing

Earnings must be

Authority

5 or more

Yes

Yes

Yes

Yes

Less than 100% of

sufficient for clients

--

--

#6

the balance

--

to sustain themselves

Earnings must

Housing

be deemed

Authority

1–4

No

Yes

Yes

Yes

Up to 50% of the

32 hours per

“reasonable” by

--

--

#7

balance

week

Housing Authority

staff

Housing

Authority

1–4

Yes

No

Yes

No

N/A

32 hours per

#8

week

--

--

2 years

(continued)

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 138

Appendix A

Appendix Exhibit A.3. Staffing, Escrow, and Graduation Policies continued Case Management Staffing

Escrow

Graduation Requirementsa

At Least

Home-

One Annual

Interim

Number

HCV

ownership

In-Person

Escrow

Limitations

Housing

with FSS

Responsi-

Responsi-

Meeting

Disbursement

Withdrawal

Employment

Employment

Employment on Revising

Authority Caseload

bilities

bilities

Expected

Permitted

Limits

Hours

Earnings

Stability

Goals

Cannot withdraw

more than $1,000

in a calendar year.

Housing

For enrol ees after

Authority

1–4

No

Yes

Yes

Yes

June 1, 2018, there is

32 hours per

#9

a maximum $5,000

week

--

--

6 months

limit during the

entire term of FSS

participation

Housing

Authority

5 or more

Yes

No

Yes

Yes

None

--

--

12 months

--

#10

Cannot withdraw

Housing

until after 12 months

Authority

1–4

Yes (1 CM)

Yes

Yes

Yes

of accruing escrow

Ful -time

--

12 months

2.5 years

#11

and maintaining

employment

Housing

Authority

1–4

Yes

Yes

Yes

Yes

Less than 100% of

#12

the balance

--

More than

minimum wage

--

--

Housing

Authority

1–4

No

No

Yes

Yes

Up to 30% of the

30 hours per

#13

balance (one time)

week

--

--

3 months

Enrol ed for at least

one year; Up to

Housing

25% of the balance

Authority

1–4

No

Yes

Yes

Yes

(annual y); car

30 hours per

#14

repairs require 20%

week

--

6 months

--

contribution by

participant

(continued)

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 139

Appendix A

Appendix Exhibit A.3. Staffing, Escrow, and Graduation Policies continued Case Management Staffing

Escrow

Graduation Requirementsa

At Least

Home-

One Annual

Interim

Number

HCV

ownership

In-Person

Escrow

Limitations

Housing

with FSS

Responsi-

Responsi-

Meeting

Disbursement

Withdrawal

Employment

Employment

Employment on Revising

Authority Caseload

bilities

bilities

Expected

Permitted

Limits

Hours

Earnings

Stability

Goals

Housing

Authority

1–4

Yes

Yes

Yes

Yes

Less than 100% of

30 hours per

#15

the balance

week

--

--

--

Suitable

employment is

defined as earned

Housing

income exceeding

Authority

5 or more

Yes

Yes

Yes

Yes

Up to 50% of the

the 50th percentile

12 months

--

#16

balance

Full time

of the area income

limit. (Can be

waived for good

cause.)

Housing

Authority

1–4

Yes

Yes

No

Yes

None

Full time

--

12 months

6 months

#17

Housing

If employed at

Authority

1–4

Yes (2 CM)

Yes

Yes

No

N/A

Full time

start, must increase

--

1 year

#18

income

FSS = Family Self-Sufficiency. HCV = Housing Choice Vouchers. TANF = Temporary Assistance for Needy Families.

aUnless otherwise noted, all requirements are as of graduation. No receipt of TANF benefits for 12 months is a requirement at all sites. Further, some sites have different employment requirements for disabled and/or elderly clients.

These are not included in the exhibit.

SOURCE: Information collected during MDRC interviews with FSS administrators and case managers in quarter 4, 2015, and in quarter 2, 2018.

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 140

Appendix B

Appendix B

Appendix Exhibit B.1. Use of Family Self-Sufficiency Services (Detailed) and Recorded Employment During Years 1 to 3, Family Self-Sufficiency Service-Use Sample

Outcome (%)

FSS Group

Used FSS services

80.4

Job search

36.1

Job readiness or job search workshop

11.1

Individual job search, job search counseling, and job referrals

35.6

Work experience or volunteer jobs

3.5

Self-employment assistance

1.5

Post-employment services

2.4

Education or training

31.4

Basic education

6.9

Post-secondary education

12.6

Vocational training or occupational credential

15.4

Financial security or homeownership preparation

48.0

Household finances, money management, budgeting, credit repair, or debt reduction 45.2

Workshops

22.8

Individual counseling

33.2

Homeownership preparation workshops or counseling

15.8

Work supports

21.7

Transportation

9.1

Childcare

3.8

Equipment and other

3.1

Financial aid

9.7

Social services

35.3

Life skil s or self-esteem-building workshops

23.3

Housing or utilities assistance

5.9

Commodities assistance (food, clothing, or holiday gifts)

11.5

Legal assistance, crisis intervention, or mentoring

5.5

Health services

7.0

Program-recorded employment or self-employment

75.4

Sample size

714

(continued)

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 141

Appendix B

Appendix Exhibit B.1. Use of Family Self-Sufficiency Services (Detailed) and Recorded Employment During Years 1 to 3, Family Self-Sufficiency Service-Use Sample continued FSS = Family Self-Sufficiency.

NOTES: The FSS service-use sample includes all or a randomly selected subsample of Housing Choice Voucher heads of household who were randomly assigned to the FSS group between October 18, 2013, and December 22, 2014, in 12 public housing agencies and were age 18 to 61 at the time of random assignment. Rounding may cause slight discrepancies in calculating sums. Detail may sum to more than total because FSS group members could participate in more than one service.

SOURCE: MDRC calculations using housing authority administrative data.

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 142

Appendix B

Appendix Exhibit B.2. Selected Indicators of Contacts and Service Use During Years 1 to 3, by Self-Reported Employment Status at Random Assignment

Outcome

Not Employed

Employed

Years 1 to 3

Services and employment

Used FSS-related service

83.7

77.6**

Job search or post-employment services

43.9

32.0***

Financial counseling

42.6

47.4

Education or training

32.5

31.7

Homeownership preparation

14.4

17.0

Health coverage or health assistance

6.1

7.7

Social services

39.3

32.0**

Supportive services

24.5

19.3*

Had program-recorded employment

55.5

92.0***

Year 3

Contacts with case managers

Had 1 contact or more (%)

49.4

51.0

Had contact during 3 months or more (%)

18.4

21.9

Average number of months with contacts

1.3

1.6*

Service use and employment (%)

***

Service use and program-recorded employment

24.5

31.4

Service use only

12.9

3.9

Program-recorded employment only

16.9

34.5

Neither service use nor program-recorded employment

45.7

30.2

Still enrol ed in FSS program in Month 36

66.3

64.4

Sample size (total = 714)

326

388

FSS = Family Self-Sufficiency.

NOTES: The FSS service-use sample includes all or a randomly selected subsample of Housing Choice Voucher heads of household who were randomly assigned to the FSS group between October 18, 2013, and December 22, 2014 in 12 housing agencies, and were age 18 to 61 at the time of random assignment. Rounding may cause slight discrepancies in calculating sums and differences. Detail may sum more than total. A chi-square test was performed on subgroup differences in frequency distributions and an F- test was performed on subgroup differences in means. Statistical significance levels are indicated as: * = 10 percent; ** = 5 percent; *** = 1 percent.

SOURCES: MDRC calculations using baseline data and housing authority administrative data.

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 143

Appendix B

Appendix Exhibit B.3. Composite Score Component Values, by Site, Family Self-Sufficiency Service-Use Sample Standardized

Expected Frequency

Had Year 1

Standardized Expected Frequency Standardized Had

Composite

Caseload Size

of Contact

Goal (%)

Caseload Size

of Contact

Year 1 Goal

Score

Housing authority #1

52

7.3

36.1

0.73

0.73

-0.43

1.02

Housing authority #2

245

1.0

0.0

-2.23

-1.31

-1.57

-5.11

Housing authority #3

54

4.0

53.6

0.70

-0.34

0.12

0.48

Housing authority #4

36

6.0

85.7

0.97

0.30

1.14

2.42

Housing authority #5

80

4.0

53.8

0.30

-0.34

0.13

0.09

Housing authority #6

101

1.9

79.7

-0.02

-1.02

0.95

-0.09

Housing authority #7

69

4.0

85.0

0.47

-0.34

1.12

1.25

Housing authority #8

140

2.0

11.4

-0.62

-0.99

-1.21

-2.82

Housing authority #9

69

4.0

60.0

0.47

-0.34

0.33

0.45

Housing authority #10

40

12.0

82.8

0.91

2.24

1.05

4.20

Housing authority #11

178

4.0

76.5

-1.20

-0.34

0.85

-0.70

Housing authority #12

68

8.8

64.6

0.48

1.22

0.47

2.17

Housing authority #13

32

12.0

100.0

1.04

2.24

1.59

4.86

Housing authority #14

143

4.0

30.8

-0.66

-0.34

-0.60

-1.60

Housing authority #15

43

4.0

8.7

0.87

-0.34

-1.30

-0.77

Housing authority #16

183

4.0

13.0

-1.28

-0.34

-1.16

-2.78

Housing authority #17

60

4.0

15.2

0.61

-0.34

-1.09

-0.83

Housing authority #18

200

4.0

38.0

-1.54

-0.34

-0.37

-2.25

FSS = Family Self-Sufficiency.

NOTES: For each standardized variable the unit of measure is a z-score, indicating the number of standard deviations the site-level mean is either above or below the cross-site mean. Caseload size indicates the average caseload size for a case manager at a particular housing authority. The expected frequency of contact for some sites is conditional upon the circumstances of the participant and is often based on employment status. As a result, the number of expected contacts in a year varies across individuals. The z-scores for average caseload sizes were multiplied by -1 to test whether small caseload sizes are positively correlated with greater expected frequency of contacts and greater incidence of having a Year 1 goal.

SOURCES: MDRC calculations from FSS group members’ Individual Training and Services Plan (ITSP) forms completed at program enrollment and from information provided by FSS administrators and case managers.

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 144

Appendix B

Appendix Exhibit B.4. Impacts on Use of Services in the 12 Months Prior to Interview by Self-Reported Employment Status at the Time of Random Assignment, Family

Self-Sufficiency 36-Month Survey Respondent Sample

Control Difference

Service Use (%)

FSS Group

Group

(Impact)

p-Value

Not employed

Used any services

78.4

70.9

7.5**

0.012

Job search or post-employment services

49.9

32.3

17.6***

0.000

Financial counseling

40.4

19.7

20.7***

0.000

Education or training

31.7

24.4

7.3**

0.018

Homeownership preparation

16.4

6.8

9.6***

0.000

Health coverage or health assistance

26.2

30.0

-3.8

0.223

Social services

21.6

22.6

-1.1

0.708

Supportive services

20.8

17.7

3.1

0.254

Sample size (total = 867)

447

420

Employed Part-Time

Used any services

81.6

69.6

12.1***

0.003

Job search or post-employment services

49.7

34.1

15.6***

0.001

Financial counseling

43.3

15.9

27.4***

0.000

Education or training

27.9

24.1

3.8

0.353

Homeownership preparation

15.9

6.1

9.8***

0.001

Health coverage or health assistance

27.3

26.3

1.0

0.816

Social services

19.9

16.9

3.1

0.412

Supportive services

19.9

17.6

2.4

0.516

Sample size (total = 495)

255

240

Employed Full-Time

Used any services

78.3

66.7

11.7***

0.003

Job search or post-employment services

38.7

29.3

9.4**

0.021

Financial counseling

38.4

16.9

21.5***

0.000

Education or training

31.3

26.5

4.8

0.218

Homeownership preparation

20.1

5.8

14.3***

0.000

Health coverage or health assistance

23.2

26.5

-3.3

0.372

Social services

14.8

16.3

-1.5

0.649

Supportive services

18.1

17.2

0.9

0.792

Sample size (total = 574)

286

288

(continued)

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 145

Appendix B

Appendix Exhibit B.4. Impacts on Use of Services in the 12 Months Prior to Interview by Self-Reported Employment Status at the Time of Random Assignment, Family

Self-Sufficiency 36-Month Survey Respondent Sample continued

FSS = Family Self-Sufficiency.

NOTES: The FSS 36-Month Survey respondent sample includes Housing Choice Voucher heads of household who were randomly assigned between October 18, 2013, and December 22, 2014, were age 18 to 61 at the time of random assignment, and responded to the FSS 36-Month Survey. Sample sizes for specific outcomes may vary because of missing values. Estimates were regression-adjusted using ordinary least squares, controlling for differences in sample member characteristics recorded at the time of random assignment.

No special weights were applied to responses to adjust for differences in sample size by housing authority. Rounding may cause slight discrepancies in calculating sums and differences. A two-tailed t-test was applied to differences between the FSS group and the control group. The p-value indicates the likelihood that the difference between the FSS group and control group arose by chance. Statistical significance levels are indicated as: * = 10 percent; ** = 5 percent; and *** = 1 percent. The H-statistic test was used to test for statistically significant differences in impact estimates across subgroups. Statistical significance levels are indicated as: † = 10 percent; †† = 5 percent; and ††† = 1 percent.

SOURCES: MDRC calculations using baseline data and responses to the FSS 36-Month Survey.

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 146

Appendix B

Appendix Exhibit B.5. Impacts on Education and Training, by Self-Reported Employment Status at the Time of Random Assignment, Family Self-Sufficiency 36-Month Survey Respondent Sample

FSS

Control Difference

Outcomes (%)

Group

Group

(Impact)

p-Value

Not employed

Earned academic credential

10.4

5.9

4.5**

0.018†

High school diploma or equivalency certificate

2.9

1.5

1.5

0.155

Post-secondary degree

7.5

4.5

3.0*

0.059††

Earned occupational credential or license

17.7

13.8

4.0

0.122

Sample size (total = 867)

447

420

Employed part-time

Earned academic credential

7.8

11.1

-3.3

0.234†

High school diploma or equivalency certificate

2.7

1.8

0.9

0.513

Post-secondary degree

5.1

9.3

-4.2*

0.078††

Earned occupational credential or license

13.3

13.1

0.2

0.951

Sample size (total = 495)

255

240

Employed full-time

Earned academic credential

8.2

5.8

2.4

0.280†

High school diploma or equivalency certificate

1.8

1.4

0.4

0.715

Post-secondary degree

6.4

4.5

2.0

0.301††

Earned occupational credential or license

12.2

10.7

1.5

0.595

Sample size (total = 574)

286

288

FSS = Family Self-Sufficiency.

NOTES: The FSS 36-Month Survey respondent sample includes Housing Choice Voucher heads-of-household who were randomly assigned between October 18, 2013, and December 22, 2014, were age 18 to 61 at the time of random assignment, and responded to the FSS 36-Month Survey. Sample sizes for specific outcomes may vary because of missing values. Estimates were regression-adjusted using ordinary least squares, controlling for differences in sample member characteristics recorded at the time of random assignment.

No special weights were applied to responses to adjust for differences in sample size by housing authority. Rounding may cause slight discrepancies in calculating sums and differences. A two-tailed t-test was applied to differences between the FSS group and the control group. The p-value indicates the likelihood that the difference between the FSS group and control group arose by chance. Statistical significance levels are indicated as: * = 10 percent; ** = 5 percent; and *** = 1 percent. The H-statistic test was used to test for statistically significant differences in impact estimates across subgroups. Statistical significance levels are indicated as: † = 10 percent; †† = 5 percent; and ††† = 1 percent.

SOURCES: MDRC calculations using baseline data and responses to the FSS 36-Month Survey.

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 147

Appendix B

Appendix Exhibit B.6. Impacts on Use of Services in the 12 Months Prior to Interview by Self-Reported Level of Educational Attainment at the Time of Random Assignment, Family Self-Sufficiency 36-Month Survey Respondent Sample

FSS

Control Difference

Service Use (%)

Group

Group

(Impact)

p-Value

No degree or credential

Used any services

70.5

64.2

6.3

0.235

Job search or post-employment services

45.6

26.8

18.8***

0.000

Financial counseling

29.6

18.6

11.1**

0.019††

Education or training

24.4

21.5

2.9

0.548

Homeownership preparation

9.0

2.7

6.3**

0.020††

Health coverage or health assistance

18.5

24.1

-5.5

0.234

Social services

16.4

19.1

-2.8

0.523

Supportive services

18.2

13.1

5.2

0.223†

Sample size (total = 358)

179

179

High school degree or GED

Used any services

77.6

61.6

16.0***

0.000

Job search or post-employment services

45.0

31.4

13.6***

0.003

Financial counseling

35.5

13.7

21.8***

0.000††

Education or training

21.9

15.8

6.1

0.107

Homeownership preparation

20.1

4.4

15.8***

0.000††

Health coverage or health assistance

28.2

24.3

3.9

0.370

Social services

21.1

16.6

4.5

0.232

Supportive services

20.5

15.6

4.9

0.176†

Sample size (total = 476)

243

233

Some college

Used any services

81.9

73.8

8.2**

0.011

Job search or post-employment services

46.3

33.1

13.2***

0.000

Financial counseling

43.5

20.2

23.2***

0.000††

Education or training

33.3

30.5

2.8

0.424

Homeownership preparation

15.4

6.7

8.7***

0.000††

Health coverage or health assistance

28.5

31.0

-2.5

0.485

Social services

20.9

19.9

1.0

0.748

Supportive services

22.8

17.4

5.4*

0.080†

Sample size (total = 738)

362

376

(continued)

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 148

Appendix B

Appendix Exhibit B.6. Impacts on Use of Services in the 12 Months Prior to Interview by Self-Reported Level of Educational Attainment at the Time of Random Assignment, Family Self-Sufficiency 36-Month Survey Respondent Sample continued

FSS

Control Difference

Service Use (%)

Group

Group

(Impact)

p-Value

2-year college degree or higher

Used any services

83.8

75.7

8.1*

0.069

Job search or post-employment services

50.0

35.2

14.8***

0.009

Financial counseling

50.2

18.9

31.3***

0.000††

Education or training

42.7

28.0

14.7***

0.007

Homeownership preparation

26.8

9.6

17.2***

0.000††

Health coverage or health assistance

24.6

29.5

-4.8

0.344

Social services

16.8

21.4

-4.6

0.293

Supportive services

16.1

24.5

-8.4*

0.059†

Sample size (total = 364)

204

160

FSS = Family Self-Sufficiency. GED = General Educational Development.

NOTES: The FSS 36-Month Survey respondent sample includes Housing Choice Voucher heads of household who were randomly assigned between October 18, 2013, and December 22, 2014, were age 18 to 61 at the time of random assignment, and responded to the FSS 36-Month Survey. Sample sizes for specific outcomes may vary because of missing values. Estimates were regression-adjusted using ordinary least squares, controlling for differences in sample member characteristics recorded at the time of random assignment.

No special weights were applied to responses to adjust for differences in sample size by housing authority. Rounding may cause slight discrepancies in calculating sums and differences. A two-tailed t-test was applied to differences between the FSS group and the control group. The p-value indicates the likelihood that the difference between the FSS group and control group arose by chance. Statistical significance levels are indicated as: * = 10 percent; ** = 5 percent; and *** = 1 percent. The H-statistic test was used to test for statistically significant differences in impact estimates across subgroups. Statistical significance levels are indicated as: † = 10 percent; †† = 5 percent; and ††† = 1 percent.

SOURCES: MDRC calculations using baseline data and responses to the FSS 36-Month Survey.

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 149

Appendix B

Appendix Exhibit B.7. Impacts on Education and Training, by Self-Reported Level of Education Attainment at the Time of Random Assignment, Family Self-Sufficiency 36-Month Survey Respondent Sample

FSS

Control

Difference

Outcomes (%)

Group

Group

(Impact)

p-Value

No degree or credential

Earned academic credential

6.9

2.9

4.1*

0.098

High school diploma or equivalency certificate

6.8

2.5

4.3*

0.070

Post-secondary degree

0.2

0.4

-0.2

0.699

Earned occupational credential or license

11.1

3.1

8.0***

0.008

Sample size (total = 358)

179

179

High school degree or GED

Earned academic credential

6.0

4.1

2.0

0.363

High school diploma or equivalency certificate

4.6

2.5

2.1

0.258

Post-secondary degree

1.4

1.5

-0.1

0.930

Earned occupational credential or license

11.9

11.4

0.5

0.880

Sample size (total = 476)

243

233

Some college

Earned academic credential

9.2

6.7

2.5

0.218

High school diploma or equivalency certificate

0.7

0.9

-0.2

0.783

Post-secondary degree

8.5

5.8

2.7

0.164

Earned occupational credential or license

18.7

15.1

3.6

0.216

Sample size (total = 738)

362

376

2-year college degree or higher

Earned academic credential

17.5

14.2

3.3

0.416

High school diploma or equivalency certificate

0.0

0.0

0.0

0.000

Post-secondary degree

17.5

14.2

3.3

0.416

Earned occupational credential or license

16.0

18.5

-2.5

0.565

Sample size (total = 364)

204

160

(continued)

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 150

Appendix B

Appendix Exhibit B.7. Impacts on Education and Training, by Self-Reported Level of Education Attainment at the Time of Random Assignment, Family Self-Sufficiency 36-Month Survey Respondent Sample continued

FSS = Family Self-Sufficiency.

NOTES: The FSS 36-Month Survey respondent sample includes Housing Choice Voucher heads of household who were randomly assigned between October 18, 2013, and December 22, 2014, were age 18 to 61 at the time of random assignment and responded to the FSS 36-Month Survey. Sample sizes for specific outcomes may vary because of missing values. Estimates were regression-adjusted using ordinary least squares, controlling for differences in sample member characteristics recorded at the time of random assignment.

No special weights were applied to responses to adjust for differences in sample size by housing authority. Rounding may cause slight discrepancies in calculating sums and differences. A two-tailed t-test was applied to differences between the FSS group and the control group. The p-value indicates the likelihood that the difference between the FSS group and control group arose by chance. Statistical significance levels are indicated as: * = 10 percent; ** = 5 percent; and *** = 1 percent. The H-statistic test was used to test for statistically significant differences in impact estimates across subgroups. Statistical significance levels are indicated as: † = 10 percent; †† = 5 percent; and ††† = 1 percent.

SOURCES: MDRC calculations using responses to the FSS 36-Month Survey, data collected from Individual Training and Services Plan forms, and information provided by FSS administrators and case managers

.

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 151

Appendix B

Appendix Exhibit B.8. Impacts on Use of Services in the 12 Months Prior to Interview for Housing Authorities with Similar Levels of Emphasis on Monitoring and Engagement, Family Self-Sufficiency 36-Month Survey Respondent Sample

FSS

Control Difference

Service Use (%)

Group

Group

(Impact)

p-Value

Low emphasis on monitoring and engagement

Used any services

75.6

65.0

10.7***

0.003

Job search or post-employment services

44.0

28.1

16.0***

0.000

Financial counseling

31.6

13.8

17.8***

0.000

Education or training

28.7

19.9

8.8***

0.008

Homeownership preparation

12.6

3.8

8.8***

0.000

Health coverage or health assistance

25.1

23.0

2.0

0.538

Social services

18.5

16.0

2.5

0.402

Supportive services

19.1

15.3

3.8

0.188

Sample size (total = 703)

358

345

Medium emphasis on monitoring and engagement

Used any services

81.1

69.4

11.7***

0.000

Job search or post-employment services

46.6

31.2

15.5***

0.000

Financial counseling

43.5

19.0

24.5***

0.000

Education or training

31.1

24.9

6.2**

0.044

Homeownership preparation

19.2

9.2

10.0***

0.000

Health coverage or health assistance

28.3

29.2

-1.0

0.760

Social services

17.9

20.9

-3.0

0.282

Supportive services

20.5

15.9

4.6*

0.089

Sample size (total = 839)

427

412

High emphasis on monitoring and engagement

Used any services

81.8

76.4

5.3

0.225

Job search or post-employment services

50.6

40.4

10.2*

0.061

Financial counseling

49.3

24.0

25.3***

0.000

Education or training

33.3

33.9

-0.6

0.910

Homeownership preparation

20.4

6.0

14.4***

0.000

Health coverage or health assistance

23.6

31.3

-7.7

0.113

Social services

25.0

19.7

5.3

0.252

Supportive services

20.4

24.0

-3.7

0.420

Sample size (total = 394)

203

191

(continued)

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 152

Appendix B

Appendix Exhibit B.8. Impacts on Use of Services in the 12 Months Prior to Interview for Housing Authorities with Similar Levels of Emphasis on Monitoring and Engagement, Family Self-Sufficiency 36-Month Survey Respondent Sample continued

FSS = Family Self-Sufficiency.

NOTES: The FSS 36-Month Survey respondent sample includes Housing Choice Voucher heads of household who were randomly assigned between October 18, 2013, and December 22, 2014, were age 18 to 61 at the time of random assignment and responded to the FSS 36-Month Survey. Sample sizes for specific outcomes may vary because of missing values. Estimates were regression-adjusted using ordinary least squares, controlling for differences in sample member characteristics recorded at the time of random assignment.

No special weights were applied to responses to adjust for differences in sample size by housing authority. Rounding may cause slight discrepancies in calculating sums and differences. A two-tailed t-test was applied to differences between the FSS group and the control group. The p-value indicates the likelihood that the difference between the FSS group and control group arose by chance. Statistical significance levels are indicated as: * = 10 percent; ** = 5 percent; and *** = 1 percent. The H-statistic test was used to test for statistically significant differences in impact estimates across subgroups. Statistical significance levels are indicated as: † = 10 percent; †† = 5 percent; and ††† = 1 percent.

SOURCES: MDRC calculations using responses to the FSS 36-Month Survey, data collected from Individual Training and Services Plan forms, and information provided by FSS administrators and case managers.

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 153

Appendix B

Appendix Exhibit B.9. Impacts on Education and Training for Housing Authorities with Similar Levels of Emphasis on Monitoring and Engagement, Family Self-Sufficiency 36-Month Survey Respondent Sample

FSS

Control Difference

Outcomes (%)

Group

Group

(Impact)

p-Value

Low emphasis on monitoring and engagement

Earned academic credential

7.9

5.9

2.0

0.325†

High school diploma or equivalency certificate

2.2

2.5

-0.3

0.802†

Post-secondary degree

5.7

3.5

2.3

0.164

Earned occupational credential or license

13.6

11.6

2.0

0.450

Sample size (total = 703)

358

345

Medium emphasis on monitoring and engagement

Earned academic credential

11.1

6.6

4.5**

0.021†

High school diploma or equivalency certificate

2.9

0.7

2.2**

0.016†

Post-secondary degree

8.2

5.9

2.3

0.188

Earned occupational credential or license

14.3

11.5

2.8

0.240

Sample size (total = 839)

427

412

High emphasis on monitoring and engagement

Earned academic credential

7.0

11.1

-4.2

0.187†

High school diploma or equivalency certificate

1.6

2.6

-1.0

0.486†

Post-secondary degree

5.4

8.5

-3.1

0.261

Earned occupational credential or license

19.9

15.9

4.1

0.337

Sample size (total = 394)

203

191

FSS = Family Self-Sufficiency.

NOTES: The FSS 36-Month Survey respondent sample includes Housing Choice Voucher heads-of-household who were randomly assigned between October 18, 2013, and December 22, 2014, were age 18 to 61 at the time of random assignment and responded to the FSS 36-Month Survey. Sample sizes for specific outcomes may vary because of missing values. Estimates were regression-adjusted using ordinary least squares, controlling for differences in sample member characteristics recorded at the time of random assignment.

No special weights were applied to responses to adjust for differences in sample size by housing authority. Rounding may cause slight discrepancies in calculating sums and differences. A two-tailed t-test was applied to differences between the FSS group and the control group. The p-value indicates the likelihood that the difference between the FSS group and control group arose by chance. Statistical significance levels are indicated as: * = 10 percent; ** = 5 percent; and *** = 1 percent. The H-statistic test was used to test for statistically significant differences in impact estimates across subgroups. Statistical significance levels are indicated as: † = 10 percent; †† = 5 percent; and ††† = 1 percent.

SOURCES: MDRC calculations using responses to the FSS 36-Month Survey, data collected from Individual Training and Services Plan forms, and information provided by FSS administrators and case managers.

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 154

Appendix B

Appendix Exhibit B.10. Impacts on Use of Services in the 12 Months Prior to Interview, by Self-Reported Employment Status at the Time of Random Assignment, Family

Self-Sufficiency 36-Month Survey Respondent Sample

FSS

Control Difference

Service Use (%)

Group

Group

(Impact)

p-Value

Not employed

Used any services

78.4

70.9

7.5**

0.012

Job search or post-employment services

49.9

32.3

17.6***

0.000

Financial counseling

40.4

19.7

20.7***

0.000

Education or training

31.7

24.4

7.3**

0.018

Homeownership preparation

16.4

6.8

9.6***

0.000

Health coverage or health assistance

26.2

30.0

-3.8

0.223

Social services

21.6

22.6

-1.1

0.708

Supportive services

20.8

17.7

3.1

0.254

Sample size (total = 867)

447

420

Employed

Used any services

80.5

67.4

13.1***

0.000

Job search or post-employment services

44.2

31.2

13.0***

0.000

Financial counseling

41.1

16.1

25.1***

0.000

Education or training

29.6

25.5

4.1

0.130

Homeownership preparation

18.3

5.7

12.6***

0.000

Health coverage or health assistance

25.4

26.2

-0.8

0.769

Social services

17.6

16.2

1.5

0.538

Supportive services

18.9

17.5

1.4

0.555

Sample size (total = 1,069)

541

528

FSS = Family Self-Sufficiency.

NOTES: The FSS 36-Month Survey respondent sample includes Housing Choice Voucher heads of household who were randomly assigned between October 18, 2013, and December 22, 2014, were age 18 to 61 at the time of random assignment, and responded to the FSS 36-Month Survey. Sample sizes for specific outcomes may vary because of missing values. Estimates were regression-adjusted using ordinary least squares, controlling for differences in sample member characteristics recorded at the time of random assignment.

No special weights were applied to responses to adjust for differences in sample size by housing authority. Rounding may cause slight discrepancies in calculating sums and differences. A two-tailed t-test was applied to differences between the FSS group and the control group. The p-value indicates the likelihood that the difference between the FSS group and control group arose by chance. Statistical significance levels are indicated as: * = 10 percent; ** = 5 percent; and *** = 1 percent. The H-statistic test was used to test for statistically significant differences in impact estimates across subgroups. Statistical significance levels are indicated as: † = 10 percent; †† = 5 percent; and ††† = 1 percent.

SOURCES: MDRC calculations using baseline data and responses to the FSS 36-Month Survey.

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 155

Appendix B

Appendix Exhibit B.11. Impacts on Educational Credential Attainment, by Self-Reported Employment Status at the Time of Random Assignment, Family Self-Sufficiency 36-Month Survey Respondent Sample

FSS

Control Difference

Outcomes (%)

Group

Group

(Impact)

p-Value

Not employed

Earned academic credential

10.4

5.9

4.5**

0.018†

High school diploma or equivalency certificate

2.9

1.5

1.5

0.155

Post-secondary degree

7.5

4.5

3.0*

0.059

Earned occupational credential or license

17.7

13.8

4.0

0.122

Sample size (total = 867)

447

420

Employed

Earned academic credential

8.3

8.0

0.3

0.858†

High school diploma or equivalency certificate

2.2

1.6

0.6

0.495

Post-secondary degree

6.1

6.4

-0.3

0.850

Earned occupational credential or license

13.0

11.5

1.5

0.462

Sample size (total = 1,069)

541

528

FSS = Family Self-Sufficiency.

NOTES: The FSS 36-Month Survey respondent sample includes Housing Choice Voucher heads-of-household who were randomly assigned between October 18, 2013, and December 22, 2014, were age 18 to 61 at the time of random assignment, and responded to the FSS 36-Month Survey. Sample sizes for specific outcomes may vary because of missing values. Estimates were regression-adjusted using ordinary least squares, controlling for differences in sample member characteristics recorded at the time of random assignment.

No special weights were applied to responses to adjust for differences in sample size by housing authority. Rounding may cause slight discrepancies in calculating sums and differences. A two-tailed t-test was applied to differences between the FSS group and the control group. The p-value indicates the likelihood that the difference between the FSS group and control group arose by chance. Statistical significance levels are indicated as: * = 10 percent; ** = 5 percent; and *** = 1 percent. The H-statistic test was used to test for statistically significant differences in impact estimates across subgroups. Statistical significance levels are indicated as: † = 10 percent; †† = 5 percent; and ††† = 1 percent.

SOURCES: MDRC calculations using baseline data and responses to the FSS 36-Month Survey.

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 156

Appendix B

Appendix Exhibit B.12. Impacts on Use of Services in the 12 Months Prior to Interview by Program Emphasis on Job Search and Post-Employment Services, Family Self-Sufficiency 36-Month Survey Respondent Sample

FSS

Control Difference

Service Use (%)

Group

Group

(Impact)

p-Value

Low emphasis on job search and post-employment services

Used any services

78.2

69.9

8.3*

0.059

Job search or post-employment services

40.7

31.7

9.0*

0.061

Financial counseling

37.9

19.1

18.8***

0.000

Education or training

28.5

23.4

5.1

0.236

Homeownership preparation

14.0

6.8

7.2**

0.015

Health coverage or health assistance

23.8

26.5

-2.7

0.539

Social services

21.3

22.7

-1.4

0.741

Supportive services

25.6

16.5

9.2**

0.021

Sample size (total = 443)

223

220

Medium emphasis on job search and post-employment services

Used any services

80.9

70.4

10.5***

0.007

Job search or post-employment services

46.1

35.2

10.9**

0.013

Financial counseling

42.6

18.7

23.8***

0.000

Education or training

30.2

25.8

4.3

0.292

Homeownership preparation

23.1

7.7

15.4***

0.000

Health coverage or health assistance

28.0

32.2

-4.2

0.319

Social services

15.1

18.9

-3.8

0.276

Supportive services

20.3

19.5

0.8

0.828

Sample size (total = 518)

261

257

High emphasis on job search and post-employment services

Used any services

78.9

68.3

10.6***

0.000

Job search or post-employment services

48.9

30.7

18.3***

0.000

Financial counseling

39.6

18.2

21.4***

0.000

Education or training

31.3

25.8

5.4*

0.063

Homeownership preparation

16.1

5.1

11.0***

0.000

Health coverage or health assistance

25.6

26.0

-0.4

0.888

Social services

20.9

17.3

3.6

0.167

Supportive services

17.7

16.1

1.6

0.503

Sample size (total = 975)

504

471

(continued)

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 157

Appendix B

Appendix Exhibit B.12. Impacts on Use of Services in the 12 Months Prior to Interview by Program Emphasis on Job Search and Post-Employment Services, Family Self-Sufficiency 36-Month Survey Respondent Sample continued

FSS = Family Self-Sufficiency.

NOTES: The FSS 36-Month Survey respondent sample includes Housing Choice Voucher heads of household who were randomly assigned between October 18, 2013, and December 22, 2014, were age 18 to 61 at the time of random assignment and responded to the FSS 36-Month Survey. Sample sizes for specific outcomes may vary because of missing values. Estimates were regression-adjusted using ordinary least squares, controlling for differences in sample member characteristics recorded at the time of random assignment.

No special weights were applied to responses to adjust for differences in sample size by housing authority. Rounding may cause slight discrepancies in calculating sums and differences. A two-tailed t-test was applied to differences between the FSS group and the control group. The p-value indicates the likelihood that the difference between the FSS group and control group arose by chance. Statistical significance levels are indicated as: * = 10 percent; ** = 5 percent; and *** = 1 percent. The H-statistic test was used to test for statistically significant differences in impact estimates across subgroups. Statistical significance levels are indicated as: † = 10 percent; †† = 5 percent; and ††† = 1 percent.

SOURCES: MDRC calculations using responses to the FSS 36-Month Survey, data collected from Individual Training and Services Plan forms, and information provided by FSS administrators and case managers.

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 158

Appendix B

Appendix Exhibit B.13. Impacts on Educational Credential Attainment, by Program Emphasis on Job Search and Post-Employment Services, Family Self-Sufficiency 36-Month Survey Respondent Sample

FSS

Control Difference

Outcomes (%)

Group

Group

(Impact)

p-Value

Low emphasis on job search and post-employment services

Earned academic credential

10.4

6.4

4.0

0.147

High school diploma or equivalency certificate

2.0

1.7

0.3

0.827

Post-secondary degree

8.4

4.7

3.7

0.124

Earned occupational credential or license

10.9

11.0

-0.1

0.978

Sample size (total = 443)

223

220

Medium emphasis on job search and post-employment services

Earned academic credential

12.4

7.1

5.3*

0.052

High school diploma or equivalency certificate

2.8

0.7

2.1*

0.084

Post-secondary degree

9.6

6.4

3.2

0.191

Earned occupational credential or license

17.0

15.5

1.6

0.646

Sample size (total = 518)

261

257

High emphasis on job search and post-employment services

Earned academic credential

7.0

7.5

-0.4

0.791

High school diploma or equivalency certificate

2.3

2.3

0.1

0.960

Post-secondary degree

4.7

5.2

-0.5

0.720

Earned occupational credential or license

16.0

11.7

4.3*

0.056

Sample size (total = 975)

504

471

FSS = Family Self-Sufficiency.

NOTES: The FSS 36-Month Survey respondent sample includes Housing Choice Voucher heads-of-household who were randomly assigned between October 18, 2013, and December 22, 2014, were age 18 to 61 at the time of random assignment, and responded to the FSS 36-Month Survey. Sample sizes for specific outcomes may vary because of missing values. Estimates were regression-adjusted using ordinary least squares, controlling for differences in sample member characteristics recorded at the time of random assignment.

No special weights were applied to responses to adjust for differences in sample size by housing authority. Rounding may cause slight discrepancies in calculating sums and differences. A two-tailed t-test was applied to differences between the FSS group and the control group. The p-value indicates the likelihood that the difference between the FSS group and control group arose by chance. Statistical significance levels are indicated as: * = 10 percent; ** = 5 percent; and *** = 1 percent. The H-statistic test was used to test for statistically significant differences in impact estimates across subgroups. Statistical significance levels are indicated as: † = 10 percent; †† = 5 percent; and ††† = 1 percent.

SOURCES: MDRC calculations using responses to the FSS 36-Month Survey, data collected from Individual Training and Services Plan forms, and information provided by FSS administrators and case managers.

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 159

Appendix B

Appendix Exhibit B.14. Impacts on Use of Services in the 12 Months Prior to Interview by Program Emphasis on Education and Training, Family Self-Sufficiency 36-Month Survey Respondent Sample

FSS

Control Difference

Service Use (%)

Group

Group

(Impact)

p-Value

Low emphasis on education and training

Used any services

76.9

70.0

6.9*

0.069

Job search or post-employment services

47.1

35.7

11.4***

0.006

Financial counseling

36.1

20.6

15.5***

0.000†††

Education or training

26.4

25.9

0.5

0.899

Homeownership preparation

16.1

4.3

11.8***

0.000

Health coverage or health assistance

28.6

27.4

1.2

0.764†

Social services

19.2

21.1

-1.8

0.603

Supportive services

24.1

21.7

2.5

0.496

Sample size (total = 586)

286

300

Medium emphasis on education and training

Used any services

82.0

67.8

14.3***

0.000

Job search or post-employment services

48.6

30.4

18.1***

0.000

Financial counseling

47.7

17.1

30.6***

0.000†††

Education or training

30.4

24.0

6.4**

0.034

Homeownership preparation

19.5

8.6

10.9***

0.000

Health coverage or health assistance

26.6

26.4

0.2

0.941†

Social services

20.1

17.3

2.8

0.289

Supportive services

18.7

15.1

3.7

0.157

Sample size (total = 888)

457

431

High emphasis on education and training

Used any services

77.4

70.2

7.2*

0.089

Job search or post-employment services

42.2

29.5

12.7***

0.007

Financial counseling

32.1

15.9

16.3***

0.000†††

Education or training

36.7

24.7

12.0***

0.007

Homeownership preparation

14.7

4.6

10.1***

0.000

Health coverage or health assistance

21.0

31.1

-10.1**

0.019 †

Social services

19.2

18.7

0.5

0.893

Supportive services

18.0

15.2

2.8

0.433

Sample size (total = 462)

245

217

(continued)

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 160

Appendix B

Appendix Exhibit B.14. Impacts on Use of Services in the 12 Months Prior to Interview by Program Emphasis on Education and Training, Family Self-Sufficiency 36-Month Survey Respondent Sample continued

FSS = Family Self-Sufficiency.

NOTES: The FSS 36-Month Survey respondent sample includes Housing Choice Voucher heads of household who were randomly assigned between October 18, 2013, and December 22, 2014, were age 18 to 61 at the time of random assignment, and responded to the FSS 36-Month Survey. Sample sizes for specific outcomes may vary because of missing values. Estimates were regression-adjusted using ordinary least squares, controlling for differences in sample member characteristics recorded at the time of random assignment.

No special weights were applied to responses to adjust for differences in sample size by housing authority. Rounding may cause slight discrepancies in calculating sums and differences. A two-tailed t-test was applied to differences between the FSS group and the control group. The p-value indicates the likelihood that the difference between the FSS group and control group arose by chance. Statistical significance levels are indicated as: * = 10 percent; ** = 5 percent; and *** = 1 percent. The H-statistic test was used to test for statistically significant differences in impact estimates across subgroups. Statistical significance levels are indicated as: † = 10 percent; †† = 5 percent; and ††† = 1 percent.

SOURCES: MDRC calculations using responses to the FSS 36-Month Survey, data collected from Individual Training and Services Plan forms, and information provided by FSS administrators and case managers.

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 161

Appendix B

Appendix Exhibit B.15. Impacts on Educational Credential Attainment, by Program Emphasis on Education and Training, Family Self-Sufficiency 36-Month Survey Respondent Sample FSS

Control Difference

Outcomes (%)

Group

Group

(Impact)

p-Value

Low emphasis on education and training

Earned academic credential

12.3

6.4

5.9**

0.018†

High school diploma or equivalency certificate

3.4

1.8

1.6

0.251

Post-secondary degree

8.9

4.6

4.4**

0.043

Earned occupational credential or license

15.6

12.7

2.9

0.339

Sample size (total = 586)

286

300

Medium emphasis on education and training

Earned academic credential

6.8

8.1

-1.3

0.469†

High school diploma or equivalency certificate

1.8

2.1

-0.3

0.718

Post-secondary degree

5.0

5.9

-1.0

0.530

Earned occupational credential or license

14.2

12.9

1.3

0.579

Sample size (total = 888)

457

431

High emphasis on education and training

Earned academic credential

9.2

7.3

1.9

0.470†

High school diploma or equivalency certificate

2.0

1.0

1.0

0.388

Post-secondary degree

7.2

6.3

0.9

0.713

Earned occupational credential or license

16.5

11.3

5.2

0.136

Sample size (total = 462)

245

217

FSS = Family Self-Sufficiency.

NOTES: The FSS 36-Month Survey respondent sample includes Housing Choice Voucher heads-of-household who were randomly assigned between October 18, 2013, and December 22, 2014, were age 18 to 61 at the time of random assignment, and responded to the FSS 36-Month Survey. Sample sizes for specific outcomes may vary because of missing values. Estimates were regression-adjusted using ordinary least squares, controlling for differences in sample member characteristics recorded at the time of random assignment.

No special weights were applied to responses to adjust for differences in sample size by housing authority. Rounding may cause slight discrepancies in calculating sums and differences. A two-tailed t-test was applied to differences between the FSS group and the control group. The p-value indicates the likelihood that the difference between the FSS group and control group arose by chance. Statistical significance levels are indicated as: * = 10 percent; ** = 5 percent; and *** = 1 percent. The H-statistic test was used to test for statistically significant differences in impact estimates across subgroups. Statistical significance levels are indicated as: † = 10 percent; †† = 5 percent; and ††† = 1 percent.

SOURCES: MDRC calculations using responses to the FSS 36-Month Survey, data collected from Individual Training and Services Plan forms, and information provided by FSS administrators and case managers.

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 162

Appendix B

Appendix Exhibit B.16. Impacts on Use of Services in the 12 Months Prior to Interview by Program Emphasis on Financial Services, Family Self-Sufficiency 36-Month Survey Respondent Sample

FSS

Control Difference

Service Use (%)

Group

Group

(Impact)

p-Value

Low emphasis on financial services

Used any services

75.0

67.8

7.3

0.139

Job search or post-employment services

45.5

31.9

13.6**

0.012

Financial counseling

40.2

15.6

24.6***

0.000

Education or training

21.8

23.9

-2.1

0.655

Homeownership preparation

15.5

6.4

9.1***

0.007

Health coverage or health assistance

27.8

32.7

-5.0

0.329

Social services

23.0

22.0

1.0

0.835

Supportive services

19.7

18.2

1.5

0.725

Sample size (total = 364)

184

180

Medium emphasis on financial services

Used any services

80.9

71.6

9.4***

0.006

Job search or post-employment services

50.4

37.6

12.9***

0.002

Financial counseling

42.1

21.1

21.0***

0.000

Education or training

32.5

27.5

5.0

0.171

Homeownership preparation

14.1

5.3

8.8***

0.000

Health coverage or health assistance

24.7

22.9

1.8

0.609

Social services

20.1

20.8

-0.7

0.834

Supportive services

22.4

20.9

1.5

0.652

Sample size (total = 639)

317

322

High emphasis on financial services

Used any services

80.2

67.6

12.7***

0.000

Job search or post-employment services

44.8

27.5

17.3***

0.000

Financial counseling

40.2

16.1

24.2***

0.000

Education or training

32.7

23.7

9.0***

0.002

Homeownership preparation

20.1

7.0

13.1***

0.000

Health coverage or health assistance

25.7

29.4

-3.7

0.221

Social services

17.6

16.6

1.0

0.702

Supportive services

18.2

14.7

3.5

0.155

Sample size (total = 933)

487

446

(continued)

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 163

Appendix B

Appendix Exhibit B.16. Impacts on Use of Services in the 12 Months Prior to Interview by Program Emphasis on Financial Services, Family Self-Sufficiency 36-Month Survey Respondent Sample continued

FSS = Family Self-Sufficiency.

NOTES: The FSS 36-Month Survey respondent sample includes Housing Choice Voucher heads of household who were randomly assigned between October 18, 2013, and December 22, 2014, were age 18 to 61 at the time of random assignment, and responded to the FSS 36-Month Survey. Sample sizes for specific outcomes may vary because of missing values. Estimates were regression-adjusted using ordinary least squares, controlling for differences in sample member characteristics recorded at the time of random assignment.

No special weights were applied to responses to adjust for differences in sample size by housing authority. Rounding may cause slight discrepancies in calculating sums and differences. A two-tailed t-test was applied to differences between the FSS group and the control group. The p-value indicates the likelihood that the difference between the FSS group and control group arose by chance. Statistical significance levels are indicated as: * = 10 percent; ** = 5 percent; and *** = 1 percent. The H-statistic test was used to test for statistically significant differences in impact estimates across subgroups. Statistical significance levels are indicated as: † = 10 percent; †† = 5 percent; and ††† = 1 percent.

SOURCES: MDRC calculations using responses to the FSS 36-Month Survey, data collected from Individual Training and Services Plan forms, and information provided by FSS administrators and case managers.

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 164

Appendix B

Appendix Exhibit B.17. Impacts on Educational Credential Attainment, by Program Emphasis on Financial Services, Family Self-Sufficiency 36-Month Survey Respondent Sample FSS

Control Difference

Outcomes (%)

Group

Group

(Impact)

p-Value

Low emphasis on financial services

Earned academic credential

7.8

4.3

3.5

0.194

High school diploma or equivalency certificate

1.1

2.2

-1.1

0.394

Post-secondary degree

6.7

2.1

4.6**

0.043

Earned occupational credential or license

13.2

11.6

1.6

0.668

Sample size (total = 364)

184

180

Medium emphasis on financial services

Earned academic credential

10.0

7.1

2.8

0.213

High school diploma or equivalency certificate

2.9

1.5

1.4

0.245

Post-secondary degree

7.0

5.6

1.4

0.469

Earned occupational credential or license

15.1

15.5

-0.3

0.915

Sample size (total = 639)

317

322

High emphasis on financial services

Earned academic credential

9.4

8.1

1.3

0.485

High school diploma or equivalency certificate

2.6

1.5

1.1

0.250

Post-secondary degree

6.8

6.6

0.2

0.899

Earned occupational credential or license

15.4

11.3

4.1*

0.075

Sample size (total = 933)

487

446

FSS = Family Self-Sufficiency.

NOTES: The FSS 36-Month Survey respondent sample includes Housing Choice Voucher heads-of-household who were randomly assigned between October 18, 2013, and December 22, 2014, were age 18 to 61 at the time of random assignment, and responded to the FSS 36-Month Survey. Sample sizes for specific outcomes may vary because of missing values. Estimates were regression-adjusted using ordinary least squares, controlling for differences in sample member characteristics recorded at the time of random assignment.

No special weights were applied to responses to adjust for differences in sample size by housing authority. Rounding may cause slight discrepancies in calculating sums and differences. A two-tailed t-test was applied to differences between the FSS group and the control group. The p-value indicates the likelihood that the difference between the FSS group and control group arose by chance. Statistical significance levels are indicated as: * = 10 percent; ** = 5 percent; and *** = 1 percent. The H-statistic test was used to test for statistically significant differences in impact estimates across subgroups. Statistical significance levels are indicated as: † = 10 percent; †† = 5 percent; and ††† = 1 percent.

SOURCES: MDRC calculations using responses to the FSS 36-Month Survey, data collected from Individual Training and Services Plan forms, and information provided by FSS administrators and case managers.

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 165

Appendix C

Appendix C

Appendix Exhibit C.1. Impacts on Employment and Earnings by Quarter, Family

Self-Sufficiency Impact Sample

FSS

Control Difference

Outcomes

Group

Group

(Impact)

p-Value

Quarterly Employment (%)

Not employed at random assignment

Quarter 2

25.9

30.2

-4.3*

0.079

Quarter 3

33.9

33.2

0.7

0.788

Quarter 4

38.4

35.6

2.7

0.289

Quarter 5

40.8

36.8

4.0

0.126

Quarter 6

42.9

40.2

2.6

0.311

Quarter 7

45.5

44.9

0.5

0.848

Quarter 8

45.0

43.7

1.3

0.628

Quarter 9

44.0

42.8

1.2

0.649

Quarter 10

45.3

45.5

-0.2

0.932

Quarter 11

46.0

46.0

-0.1

0.981

Quarter 12

48.0

47.8

0.2

0.939

Quarter 13

47.9

47.1

0.9

0.755

Employed part-time (1-34 hours) at random assignment

Quarter 2

81.8

80.8

1.1

0.688

Quarter 3

78.4

81.2

-2.8

0.344

Quarter 4

79.7

77.6

2.2

0.489

Quarter 5

81.2

77.0

4.2

0.170

Quarter 6

77.3

74.1

3.2

0.326

Quarter 7

78.2

75.4

2.9

0.376

Quarter 8

75.9

77.5

-1.6

0.622

Quarter 9

77.3

75.7

1.6

0.621

Quarter 10

77.1

74.1

3.0

0.374

Quarter 11

78.8

73.5

5.3

0.103

Quarter 12

75.7

74.9

0.9

0.789

Quarter 13

74.5

71.1

3.4

0.320

(continued)

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 166

Appendix C

Appendix Exhibit C.1. Impacts on Employment and Earnings by Quarter, Family

Self-Sufficiency Impact Sample continued

FSS

Control Difference

Outcomes

Group

Group

(Impact)

p-Value

Employed full-time (35 hours or more) at random assignment

Quarter 2

87.7

88.2

-0.5

0.795

Quarter 3

84.9

86.6

-1.8

0.412

Quarter 4

84.1

86.3

-2.2

0.320

Quarter 5

82.5

84.8

-2.3

0.345

Quarter 6

81.6

84.0

-2.4

0.351

Quarter 7

82.0

84.9

-2.9

0.254

Quarter 8

80.7

84.3

-3.7

0.155

Quarter 9

78.7

84.4

-5.6**

0.035

Quarter 10

80.4

83.3

-2.9

0.271

Quarter 11

81.2

81.8

-0.6

0.824

Quarter 12

81.6

80.0

1.6

0.568

Quarter 13

82.0

80.4

1.6

0.566

Total Impact Sample

Quarter 2

58.8

60.8

-2.0

0.154

Quarter 3

60.7

61.7

-1.0

0.510

Quarter 4

62.9

61.7

1.2

0.421

Quarter 5

63.9

61.5

2.4

0.122

Quarter 6

63.4

62.1

1.3

0.410

Quarter 7

65.1

64.7

0.4

0.799

Quarter 8

63.9

64.5

-0.5

0.734

Quarter 9

63.4

63.5

-0.1

0.947

Quarter 10

64.2

64.1

0.2

0.921

Quarter 11

65.1

63.8

1.3

0.439

Quarter 12

65.2

64.5

0.7

0.686

Quarter 13

65.1

63.3

1.8

0.292

(continued)

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 167

Appendix C

Appendix Exhibit C.1. Impacts on Employment and Earnings by Quarter, Family

Self-Sufficiency Impact Sample continued

FSS

Control Difference

Outcomes

Group

Group

(Impact)

p-Value

Total Earnings ($)

Not employed at random assignment

Quarter 2

610

730

-120

0.189

Quarter 3

1,026

1,022

4

0.973

Quarter 4

1,195

1,205

-10

0.936

Quarter 5

1,374

1,310

64

0.637

Quarter 6

1,585

1,544

41

0.780

Quarter 7

1,910

1,807

103

0.542

Quarter 8

1,970

1,885

85

0.615

Quarter 9

2,028

1,755

273

0.106

Quarter 10

2,182

2,026

156

0.411

Quarter 11

2,288

2,220

68

0.720

Quarter 12

2,386

2,325

61

0.759

Quarter 13

2,369

2,279

90

0.637

Employed part-time (1-34 hours) at random assignment

Quarter 2

3,167

3,186

-19

0.914

Quarter 3

3,241

3,133

108

0.575

Quarter 4

3,307

3,149

158

0.454

Quarter 5

3,571

3,314

257

0.253

Quarter 6

3,707

3,345

362

0.144

Quarter 7

3,512

3,689

-177

0.473

Quarter 8

3,769

3,904

-136

0.610

Quarter 9

3,915

3,922

-7

0.981

Quarter 10

4,079

3,824

255

0.359

Quarter 11

4,052

3,882

170

0.541

Quarter 12

4,140

3,953

187

0.499

Quarter 13

4,134

3,801

333

0.239

(continued)

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 168

Appendix C

Appendix Exhibit C.1. Impacts on Employment and Earnings by Quarter, Family

Self-Sufficiency Impact Sample continued

FSS

Control Difference

Outcomes

Group

Group

(Impact)

p-Value

Employed full-time (35 hours or more) at random assignment

Quarter 2

5,639

5,696

-57

0.768

Quarter 3

5,656

5,765

-109

0.599

Quarter 4

5,497

5,774

-277

0.233

Quarter 5

5,338

5,805

-467*

0.066

Quarter 6

5,736

5,755

-20

0.940

Quarter 7

5,854

6,074

-220

0.401

Quarter 8

5,637

5,936

-300

0.268

Quarter 9

5,885

6,000

-115

0.687

Quarter 10

5,963

6,038

-75

0.800

Quarter 11

5,921

6,062

-141

0.631

Quarter 12

5,944

6,222

-278

0.378

Quarter 13

5,963

6,256

-293

0.340

Total Impact Sample

Quarter 2

2,781

2,870

-90

0.285

Quarter 3

2,983

3,015

-32

0.737

Quarter 4

3,041

3,090

-49

0.636

Quarter 5

3,162

3,169

-6

0.955

Quarter 6

3,399

3,271

128

0.275

Quarter 7

3,520

3,579

-59

0.632

Quarter 8

3,556

3,618

-63

0.621

Quarter 9

3,704

3,574

129

0.325

Quarter 10

3,821

3,698

123

0.372

Quarter 11

3,843

3,811

32

0.814

Quarter 12

3,930

3,911

19

0.897

Quarter 13

3,911

3,880

31

0.827

Sample size (total = 2,548)

1,282

1,266

(continued)

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 169

Appendix C

Appendix Exhibit C.1. Impacts on Employment and Earnings by Quarter, Family

Self-Sufficiency Impact Sample continued

FSS = Family Self-Sufficiency.

NOTES: The FSS impact sample includes Housing Choice Voucher households that were randomly assigned between October 18, 2013, and December 22, 2014, and with a head of household age 18 to 61 at the time of random assignment. Estimates were regression-adjusted using ordinary least squares, controlling for pre-random assignment characteristics of sample members. Rounding may cause slight discrepancies in calculating sums and differences. A two-tailed t-test was applied to differences between research groups. The p-value indicates the likelihood that the difference between the FSS group and control group arose by chance. Statistical significance levels are indicated as: * = 10 percent; ** = 5 percent; and *** = 1 percent.

SOURCES: MDRC calculations using baseline data and quarterly wage data from the National Directory of New Hires.

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 170

Appendix C

Appendix Exhibit C.2. Regression Coefficients for Estimated Impacts on Total Earnings in Quarters 2 to 13, Family Self-Sufficiency Impact Sample

Parameter

Estimate

p-Value

Intercept

7,396

0.142

Assigned to FSS Group (impact)

163

0.884

Covariates

Sample member characteristics

Female

1,797

0.393

Age 18–34

7,556

<.0001

Age 35–44

5,991

0.000

Married or cohabitating

1,394

0.521

Black

960

0.521

1 child

2,055

0.246

2 children

3,830

0.041

3 or more children

866

0.669

Has a child age 5 or younger

570

0.693

Education

High school diploma or GED

-27

0.988

Some col ege

1,742

0.295

2-year col ege degree or higher

9,206

<.0001

Has trade license or training certificate

898

0.438

Public assistance

Received SNAP/food stamps

-665

0.660

Received SSI or SSDI

-6,541

0.000

Received TANF

483

0.794

Received Housing Choice Voucher less than 4 years

17

0.990

Received Housing Choice Voucher 4–7 years

3,280

0.028

Hardship and barriers to employment

Has any barrier to employment

-5,533

<.0001

Reported 1 hardship in the year before random assignment

-111

0.943

Reported 2 hardships in the year before random assignment

-888

0.601

Reported 3 or more hardships in year before random assignment

-1,261

0.413

(continued)

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 171

Appendix C

Appendix Exhibit C.2. Regression Coefficients for Estimated Impacts on Total Earnings in Quarters 2 to 13, Family Self-Sufficiency Impact Sample continued

Parameter

Estimate

p-Value

Employment

Currently employed

6,614

0.000

Currently employed ful -time

4,008

0.019

Employed 1–6 months in the year before random assignment

5,665

0.003

Employed 7–11 months in the year before random assignment

3,065

0.191

Employed 12 months in the year before random assignment

-159

0.945

Earnings

Employed in the quarter before random assignment

-264

0.894

Employed in the second quarter before random assignment

-1,701

0.373

Total earnings in the 2 quarters before random assignment

4

<.0001

Total earnings squared in the 2 quarters before random assignment

0

0.522

Enrollment

Randomly assigned in quarter 4, 2013

-2,817

0.420

Randomly assigned in quarter 1, 2014

-1,767

0.275

Randomly assigned in quarter 2, 2014

1,253

0.402

Housing Authority of the City of Alameda

6,770

0.084

Orange County Housing Authority

3,556

0.384

Housing Authority of the City of Riverside

-615

0.874

Housing Authority of the City of Deerfield Beach

1,783

0.744

Housing Authority of the City of Ft. Lauderdale

-770

0.860

Baltimore County Housing Office

-3,057

0.482

Housing Opportunities Commission of Montgomery County

3,426

0.418

Housing Authority of Kansas City

1,907

0.662

Jersey City Housing Authority

-1,220

0.800

Akron Metropolitan Housing Authority

-6,745

0.194

Columbus Metropolitan Housing Authority

876

0.861

Lucas Metropolitan Housing Authority

-3,202

0.556

Youngstown Metropolitan Housing Authority

136

0.979

Dal as Housing Authority

4,049

0.330

Fort Worth Housing Authority

-1,040

0.831

Houston Housing Authority

2,529

0.531

Tarrant County Housing Assistance Office

-4,467

0.312

(continued)

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 172

Appendix C

Appendix Exhibit C.2. Regression Coefficients for Estimated Impacts on Total Earnings in Quarters 2 to 13, Family Self-Sufficiency Impact Sample continued

Parameter

Estimate

p-Value

Enrol ed in FSS for help with employment

-1,301

0.333

Medical coverage

Public medical insurance

-3,788

0.027

Private medical insurance

3,944

0.051

Savings

Had checking or savings account

3,604

0.008

Had savings between $1–$500

-757

0.574

Had savings greater than $500

1,263

0.544

Debt

$1–$1,000

-1,902

0.363

$1,001–$5,000

1,895

0.314

$5,001–$10,000

3,142

0.124

$10,001–$20,000

1,625

0.390

$20,001 or greater

5,297

0.007

R-squared

0.541

Sample size

2,548

FSS = Family Self-Sufficiency. GED = General Educational Development. SNAP = Supplemental Nutrition Assistance Program. SSI-SSDI = Supplemental Security Income-Social Security Disability Insurance. TANF = Temporary Assistance for Needy Families.

NOTES: The FSS impact sample includes Housing Choice Voucher households that were randomly assigned between October 18, 2013, and December 22, 2014, and with a head of household age 18 to 61 at the time of random assignment. Estimates were regression-adjusted using ordinary least squares, controlling for pre-random assignment characteristics of sample members. No special weights were applied to responses to adjust for differences in sample size by site.

SOURCES: MDRC calculations using baseline data and quarterly wage data from the National Directory of New Hires.

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 173

Appendix C

Appendix Exhibit C.3. Unadjusted and Adjusted Impacts on Employment and Earnings in Quarters 2 to 13, Family Self-Sufficiency Impact Sample

Adjusted Impacts

Unadjusted Impacts

Control

Difference

Control

Difference

Outcomes

FSS Group

Group

(Impact)

p-Value

FSS Group

Group

(Impact)

p-Value

Ever employed (%)

83.4

81.7

1.7

0.170

83.0

82.1

0.8

0.573

Average quarterly employment rate (%)

63.5

63.0

0.5

0.679

62.9

63.5

-0.6

0.699

Total earnings ($)

41,650

41,487

163

0.884

40,751

42,397

-1,646

0.307

Sample size (total = 2,548)

1,282

1,266

1,282

1,266

FSS = Family Self-Sufficiency.

NOTES: The FSS impact sample includes Housing Choice Voucher households that were randomly assigned between October 18, 2013, and December 22, 2014, and with a head of household age 18 to 61 at the time of random assignment. Average quarterly employment rate is calculated as: total quarters with employment divided by total quarters of follow-up, expressed as a percentage. Regression-adjusted estimates used ordinary least squares, controlling for pre-random assignment characteristics of sample members. No special weights were applied to adjust for differences in sample size by housing authority. Rounding may cause slight discrepancies in calculating differences. A two tailed t-test was applied to differences between research groups. The p-value indicates the likelihood that the difference between the FSS group and control group arose by chance. Statistical significance levels are indicated as: * = 10

percent; ** = 5 percent; and *** = 1 percent.

SOURCE: MDRC calculations using quarterly wage data from the National Directory of New Hires.

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 174

Appendix C

Appendix Exhibit C.4. Total Earnings in Quarters 2 to 13 by Level of Exclusion for Outlier Values, Family Self-Sufficiency Impact Sample

Sample

Standard

95th

99th

Outcome

Size

Mean

Deviation

Percentile

Percentile Maximum

Earnings ($)

Include all values

2,548

41,823

41,391

119,665

166,389

313,506

Reset values above

$25,000 to $0

2,548

41,569

40,620

119,453

159,552

211,989

Exclude top 1 percent

2,523

40,280

38,600

114,908

142,987

166,389

Exclude top 5 percent

2,421

36,273

33,885

101,094

115,224

119,665

FSS = Family Self-Sufficiency.

NOTE: The FSS Impact sample includes Housing Choice Voucher households that were randomly assigned between October 18, 2013, and December 22, 2014, and with a head of household age 18 to 61 at the time of random assignment.

SOURCE: MDRC calculations using quarterly wage data from the National Directory of New Hires.

Appendix Exhibit C.5. Impacts on Total Earnings in Quarters 2 to 13 by Level of Exclusion for Outlier Values, Family Self-Sufficiency Impact Sample

Control

Difference

Outcome

FSS Group

Group

(Impact)

p-Value

Earnings ($)

Include all values

41,875

41,770

105

0.926

Reset values above $25,000 to $0

41,650

41,487

163

0.884

Exclude top 1 percent

40,586

39,968

618

0.568

Exclude top 5 percent

36,647

35,894

752

0.448

Sample size (total = 2,548)

1,282

1,266

FSS = Family Self-Sufficiency.

NOTES: The FSS impact sample includes Housing Choice Voucher households that were randomly assigned between October 18, 2013, and December 22, 2014, and with a head of household age 18 to 61 at the time of random assignment. Sample sizes for specific outcomes may vary because of exclusion of outliers and missing values. Estimates were regression-adjusted using ordinary least squares, controlling for pre-random assignment characteristics of sample members. No special weights were applied to responses to adjust for differences in sample size by site. Rounding may cause slight discrepancies in calculating sums and differences. A two-tailed t-test was applied to differences between research groups. The p-value indicates the likelihood that the difference between the FSS group and control group arose by chance. Statistical significance levels are indicated as: * = 10 percent; ** = 5 percent; and *** = 1 percent.

SOURCE: MDRC calculations using quarterly wage data from the National Directory of New Hires.

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 175

Appendix C

Appendix Exhibit C.6. Impacts on Employment and Earnings During Quarters 2 to 13 by Weighting Strategy, Family Self-Sufficiency Impact Sample

FSS

Control

Difference

Outcomes

Group

Group

(Impact)

p-Value

Unweighted

Ever employed (%)

83.4

81.7

1.7

0.170

Average quarterly employment rate (%)

63.5

63.0

0.5

0.679

Total earnings ($)

41,650

41,487

163

0.884

Equal weighting

Ever employed (%)

83.5

81.6

1.9

0.114

Average quarterly employment rate (%)

63.6

62.9

0.6

0.574

Total earnings ($)

41,449

41,755

-306

0.773

Weighting by total FSS householdsa

Ever employed (%)

83.5

81.6

1.9

0.111

Average quarterly employment rate (%)

63.5

63.0

0.5

0.685

Total earnings ($)

41,576

41,628

-52

0.962

Sample size (total = 2,548)

1,282

1,266

FSS = Family Self-Sufficiency.

aTotal FSS caseload includes 5,686 households enrolled in FSS as of December 31, 2014. The total includes FSS group members and FSS participants who enrolled in the program before the start of random assignment, enrolled in the program after the end of random assignment, or withdrew from the research sample.

NOTES: The FSS Impact sample includes Housing Choice Voucher households that were randomly assigned between October 18, 2013, and December 22, 2014, and with a head of household age 18 to 61 at the time of random assignment. Average quarterly employment rate is calculated as: total quarters with employment divided by total quarters of follow-up, expressed as a percentage.

Estimates were regression-adjusted using ordinary least squares, controlling for pre-random assignment characteristics of sample members. Rounding may cause slight discrepancies in calculating sums and differences. A two-tailed t-test was applied to differences between research groups. The p-value indicates the likelihood that the difference between the FSS group and control group arose by chance. Statistical significance levels are indicated as: * = 10 percent; ** = 5 percent; and *** = 1 percent.

SOURCES: MDRC calculations using quarterly wage data from the National Directory of New Hires and December 2014 FSS

caseload data from the U.S. Department of Housing and Urban Development (HUD) Public and Indian Housing Information Center (PIC) database.

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 176

Appendix D

Appendix D

Appendix Exhibit D.1. Differences by Research Group in Estimated Annual Income After 3

Years of Fol ow-Up, Family Self-Sufficiency Impact Sample Members Currently Enrol ed in the Housing Choice Voucher Program

FSS

Control

Outcome

Group

Group

Difference

Head-of-household income

Received income (%)

94.2

94.8

-0.5

Earnings

56.7

54.1

2.6

Business revenue

2.8

3.0

-0.1

Unemployment benefits

3.4

3.6

-0.2

TANF, General Assistance, or food stamps/SNAP

33.1

31.5

1.6

Disability insurance or pension

16.3

15.5

0.8

Child support

19.7

20.4

-0.7

Other income

6.5

9.6

-3.1

Average total annual income ($)

17,604

16,864

740

Earnings

12,247

11,082

1,165

Business revenue

77

69

8

Unemployment benefits

227

161

66

TANF, General Assistance, or food stamps/SNAP

1,706

1,613

93

Pension/disability insurance

1,450

1,395

55

Child support

746

756

-9

Other income

250

316

-66

Total annual income (%)

Less than $5,000

16.7

17.9

-1.1

$5,000–$9,999

14.7

15.8

-1.2

$10,000–$14,999

17.7

16.8

0.9

$15,000–$19,999

12.2

15.3

-3.2

$20,000–$29,999

22.9

18.6

4.3

$30,000 or more

15.9

15.7

0.2

(continued)

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 177

Appendix D

Appendix Exhibit D.1. Differences by Research Group in Estimated Annual Income After 3

Years of Fol ow-Up, Family Self-Sufficiency Impact Sample Members Currently Enrol ed in the Housing Choice Voucher Program continued

FSS

Control

Outcome

Group

Group

Difference

Total household income

Received income (%)

98.5

98.4

0.1

Earnings

64.2

60.3

3.9

Business revenue

3.2

3.1

0.1

Unemployment benefits

4.0

3.8

0.2

TANF, General Assistance, or food stamps/SNAP

34.5

33.5

1.0

Disability insurance or pension

27.7

27.0

0.6

Child support

20.2

21.3

-1.1

Other income

9.7

12.7

-3.0

Total household income

Average total annual income ($)

22,301

21,494

807

Earnings

14,985

14,069

916

Business revenue

110

99

11

Unemployment benefits

284

180

103

TANF, General Assistance, or food stamps/SNAP

1,842

1,774

68

Other income

527

625

-97

Pension/disability insurance

2,691

2,680

11

Child support

770

797

-26

Other income

527

625

-97

Total annual income (%)

Less than $5,000

10.0

10.7

-0.7

$5,000–$9,999

11.4

13.6

-2.2

$10,000–$14,999

15.2

14.4

0.8

$15,000–$19,999

11.8

14.1

-2.4

$20,000–$29,999

25.4

21.8

3.7

$30,000 or more

26.1

25.3

0.8

Sample size (total = 1,836)

919

917

(continued)

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 178

Appendix D

Appendix Exhibit D.1. Differences by Research Group in Estimated Annual Income After 3

Years of Fol ow-Up, Family Self-Sufficiency Impact Sample Members Currently Enrol ed in the Housing Choice Voucher Program continued

FSS = Family Self-Sufficiency. SNAP = Supplemental Nutrition Assistance Program. TANF = Temporary Assistance for Needy Families.

NOTES: The FSS impact sample includes Housing Choice Voucher households that were randomly assigned between October 18, 2013, and December 22, 2014, and with a head of household age 18 to 61 at the time of random assignment. Income calculations used data from each household’s most recent Housing Choice Voucher eligibility reexamination that took place between Months 26 through 37 after their date of random assignment. For these calculations, households with no reported income or who exited or became ineligible from the Housing Choice Voucher program were excluded from the calculations. Estimates of differences between research groups are non-experimental, because enrollment in the FSS program could affect the incidence or timing of exits from the Housing Choice Voucher program after random assignment. If so, FSS and control group members who remain in the Housing Choice Voucher program could differ in observed or unobserved characteristics or experiences that may affect their income levels. Estimates were regression-adjusted using ordinary least squares, controlling for pre-random assignment characteristics of sample members. For each dollar amount outcome, values above the 99th percentile were considered as outliers and dropped from the calculations. As a result of this procedure, adjusted mean values for specific types of income do not sum to total income. Sample sizes for specific outcomes may vary because of missing values. Rounding may cause slight discrepancies in calculating sums and differences. Detail may sum to more than 100 percent because study participants may receive more than one type of income.

SOURCES: MDRC calculations using U.S. Department of Housing and Urban Development Inventory Management System (IMS)/

PIH Information Center (PIC) data.

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 179

Appendix D

Appendix Exhibit D.2. Differences by Research Group in Estimated Annual Income After 3

Years of Fol ow-Up, Family Self-Sufficiency Impact Sample Members

FSS

Control

Outcome

Group

Group

Difference

Head-of-household income

Received income (%)

83.7

82.5

1.2

Earnings

51.2

48.0

3.2

Business revenue

2.5

2.5

-0.1

Unemployment benefits

2.9

3.0

-0.1

TANF, General Assistance, or food stamps/SNAP

28.4

27.4

1.1

Disability insurance or pension

13.7

13.2

0.5

Child support

18.4

18.2

0.1

Other income

6.0

8.1

-2.1

Average total annual income ($)

15,829

15,287

542

Earnings

11,115

10,217

898

Business revenue

63

53

10

Unemployment benefits

180

156

24

TANF, General Assistance, or food stamps/SNAP

1,452

1,440

12

Pension/disability insurance

1,205

1,158

47

Child support

698

639

59

Other income

211

287

-75

Total annual income (%)

Less than $5,000

26.6

27.9

-1.3

$5,000–$9,999

13.0

13.3

-0.3

$10,000–$14,999

14.5

14.6

-0.1

$15,000–$19,999

10.8

12.4

-1.6

$20,000–$29,999

20.2

16.5

3.7

$30,000 or more

14.9

15.3

-0.4

Total household income

Received income (%)

87.0

85.5

1.4

Earnings

57.0

53.1

3.9

Business revenue

2.7

2.7

0.0

Unemployment benefits

3.5

3.2

0.3

TANF, General Assistance, or food stamps/SNAP

30.0

28.9

1.1

Disability insurance or pension

24.3

23.1

1.2

(continued)

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 180

Appendix D

Appendix Exhibit D.2. Differences by Research Group in Estimated Annual Income After 3

Years of Fol ow-Up, Family Self-Sufficiency Impact Sample Members continued FSS

Control

Outcome

Group

Group

Difference

Child support

18.7

19.0

-0.3

Other income

8.5

10.5

-2.0

Total household income

Average total annual income ($)

20,078

19,184

894

Earnings

13,478

12,651

826

Business revenue

86

77

9

Unemployment benefits

228

169

59

TANF, General Assistance, or food stamps/SNAP

1,594

1,562

32

Other income

441

541

-101

Pension/disability insurance

2,366

2,270

97

Child support

716

671

44

Other income

441

541

-101

Total annual income (%)

Less than $5,000

20.9

22.1

-1.2

$5,000–$9,999

10.3

11.5

-1.3

$10,000–$14,999

12.2

12.2

0.0

$15,000–$19,999

10.2

11.9

-1.7

$20,000–$29,999

22.4

19.4

2.9

$30,000 or more

24.1

22.9

1.2

Sample size (total = 2,492)

1,252

1,240

FSS = Family Self-Sufficiency. SNAP = Supplemental Nutrition Assistance Program. TANF = Temporary Assistance for Needy Families.

NOTES: The FSS impact sample includes Housing Choice Voucher households that were randomly assigned between October 18, 2013, and December 22, 2014, and with a head of household age 18 to 61 at the time of random assignment. Income calculations used data from each household’s most recent Housing Choice Voucher eligibility reexamination that took place between Months 26 through 37 after their date of random assignment. For these calculations, households with no reported income or who exited or became ineligible from the Housing Choice Voucher program were set to zero in the calculations. Estimates of differences between research groups are non-experimental, because enrollment in the FSS program could affect the incidence or timing of exits from the Housing Choice Voucher program after random assignment. If so, FSS and control group members who remain in the Housing Choice Voucher program could differ in observed or unobserved characteristics or experiences that may affect their income levels. Estimates were regression-adjusted using ordinary least squares, controlling for pre-random assignment characteristics of sample members. For each dollar amount outcome, values above the 99th percentile were considered as outliers and dropped from the calculations. As a result of this procedure, adjusted mean values for specific types of income do not sum to total income. Sample sizes for specific outcomes may vary because of missing values. Rounding may cause slight discrepancies in calculating sums and differences. Detail may sum to more than 100.0 percent because study participants may receive more than one type of income.

SOURCES: MDRC calculations using U.S. Department of Housing and Urban Development Inventory Management System (IMS)/

PIH Information Center (PIC) data.

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 181

Appendix E

Appendix E

Appendix Exhibit E.1. Impacts on Selected Employment and Earnings Outcomes in Years 1 to 3, by Selected Baseline Characteristics, Family Self-Sufficiency Impact Sample Sample Sizes

FSS

Control Difference

FSS

Control

Outcome

Group

Group

(Impact)

p-Value Group Group

Average quarterly employment rate (%)

Employment status

Not employed

42.0

41.2

0.8

0.667

575

551

Employed part-time

78.0

76.1

1.9

0.391

335

313

Employed ful -time

82.3

84.1

-1.8

0.311

372

402

Educational attainment

No degree or credential

58.8

54.9

3.9

0.154

257

253

High school degree or GED

60.6

60.7

-0.1

0.965

319

297

Some col ege

66.5

66.6

-0.1

0.967

470

512

2-year col ege degree or higher

67.0

66.8

0.2

0.936

236

204

Total household income

$1–$10,000

50.5

53.0

-2.4

0.321

318

320

$10,001–$20,000

58.6

56.6

2.0

0.343

438

401

More than $20,000

75.6

73.5

2.1

0.203

526

545

Reported barrier to employment

Yes

50.6

49.3

1.3

0.478

534

517

No

72.1

73.1

-1.0

0.501

748

749

Disability status

Received SSI/SSDI

38.3

34.4

3.8

0.173

177

188

Did not receive SSI/SSDI

67.6

67.9

-0.3

0.821

1,105 1,078

Percentage of rent and utilities expenses paid by household

0–25

54.5

56.0

-1.6

0.406

583

550

25.01–50

64.1

62.2

1.9

0.338

419

410

More than 50

80.6

77.2

3.4*

0.086

280

306

Percentage of household income used for rent and utilities

0–30

64.3

63.0

1.3

0.363

832

816

More than 30

62.1

62.7

-0.6

0.748

450

450

(continued)

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 182

Appendix E

Appendix Exhibit E.1. Impacts on Selected Employment and Earnings Outcomes in Years 1 to 3, by Selected Baseline Characteristics, Family Self-Sufficiency Impact Sample continued Sample Sizes

FSS

Control Difference

FSS

Control

Outcome

Group

Group

(Impact)

p-Value Group Group

Total earnings ($)

Employment status

Not employed

20,923

20,109

814

0.572

575

551

Employed part-time

44,596

43,103

1,492

0.512

335

313

Employed ful -time

69,033

71,383

-2,350

0.338

372

402

Educational attainment

No degree or credential

32,548

31,561

986

0.641

257

253

High school degree or GED

38,105

37,140

965

0.639

319

297

Some col ege

44,019

44,285

-266

0.888

470

512

2-year col ege degree or higher

52,794

51,765

1,029

0.768

236

204

Total household income

$1–$10,000

25,185

27,041

-1,856

0.380

318

320

$10,001–$20,000

30,180

30,312

-132

0.937

438

401

More than $20,000

61,093

58,270

2,824

0.153

526

545

Reported barrier to employment

Yes

27,651

28,770

-1,119

0.481

534

517

No

51,369

50,552

817

0.603

748

749

Disability status

Received SSI/SSDI

17,890

15,692

2,198

0.281

177

188

Did not receive SSI/SSDI

45,483

45,966

-482

0.702

1,105 1,078

Percentage of rent and utilities expenses paid by household

0–25

27,989

29,144

-1,156

0.457

583

550

25.01–50

40,596

40,693

-96

0.961

419

410

More than 50

69,520

66,738

2,782

0.318

280

306

Percentage of household income used for rent and utilities

0–30

42,242

41,497

745

0.585

832

816

More than 30

40,980

41,064

-84

0.966

450

450

Average annual earnings greater than $25,000 (%)

Employment status

Not employed

5.7

4.8

0.8

0.513

575

551

Employed part-time

18.7

16.4

2.3

0.408

335

313

(continued)

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 183

Appendix E

Appendix Exhibit E.1. Impacts on Selected Employment and Earnings Outcomes in Years 1 to 3, by Selected Baseline Characteristics, Family Self-Sufficiency Impact Sample continued Sample Sizes

FSS

Control Difference

FSS

Control

Outcome

Group

Group

(Impact)

p-Value Group Group

Employed ful -time

42.5

45.7

-3.2

0.329

372

402

Educational attainment

No degree or credential

12.7

11.2

1.5

0.514

257

253

High school degree or GED

19.8

15.1

4.7*

0.055

319

297

Some col ege

20.2

23.0

-2.8

0.214

470

512

2-year col ege degree or higher

31.2

29.1

2.2

0.581

236

204

Total household income

$1–$10,000

6.0

9.1

-3.1

0.116†

318

320

$10,001–$20,000

8.7

9.2

-0.5

0.779† 438

401

More than $20,000

38.6

34.9

3.7

0.134†

526

545

Reported barrier to employment

Yes

9.9

12.6

-2.6

0.116†

534

517

No

27.5

25.7

1.8

0.344† 748

749

Disability status

Received SSI/SSDI

7.0

4.6

2.4

0.282

177

188

Did not receive SSI/SSDI

22.3

23.0

-0.7

0.635

1,105 1,078

Percentage of rent and utilities expenses paid by household

0–25

8.1

10.0

-1.9

0.226

583

550

25.01–50

18.1

19.3

-1.1

0.626

419

410

More than 50

46.9

41.8

5.1

0.168

280

306

Percentage of household income used for rent and utilities

0–30

20.6

20.4

0.2

0.889

832

816

More than 30

20.0

19.6

0.3

0.882

450

450

Sample size (total = 2,548)

1,282

1,266

FSS = Family Self-Sufficiency. GED = General Educational Development. SSI-SSDI = Supplemental Security Income-Social Security Disability Insurance.

NOTES: The FSS impact sample includes Housing Choice Voucher households that were randomly assigned between October 18, 2013, and December 22, 2014, and with a head of household age 18 to 61 at the time of random assignment. Average quarterly employment rate is calculated as: total quarters with employment divided by total quarters of follow-up, expressed as a percentage.

Estimates were regression-adjusted using ordinary least squares, controlling for pre-random assignment characteristics of sample members. Rounding may cause slight discrepancies in calculating differences. A two-tailed t-test was applied to differences between research groups. The p-value indicates the likelihood that the difference between the FSS group and control group arose by chance.

Statistical significance levels are indicated as: * = 10 percent; ** = 5 percent; and *** = 1 percent. The H-statistic test was used to test for statistically significant differences in impact estimates across different subgroups. Statistical significance levels are indicated as: † = 10

percent; †† = 5 percent; and ††† = 1 percent.

SOURCES: MDRC calculations using baseline data, U.S. Department of Housing and Urban Development Inventory Management System (IMS)/PIH Information Center (PIC) data, and quarterly wage data from the National Directory of New Hires.

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 184

Appendix E

Appendix Exhibit E.2. Characteristics, Family Self-Sufficiency 36-Month Survey Respondent Sample

Sample Sizes

Control Difference

FSS

Control

Outcome

FSS Group

Group

(Impact)

p-Value

Group

Group

Currently employed with full-time hours (%)

Employment status

Not employed

28.7

22.6

6.1**

0.036

446

417

Employed part-time

36.9

31.3

5.6

0.211

254

240

Employed ful -time

64.3

65.4

-1.1

0.798

286

286

Educational attainment

No degree or credential

38.3

26.1

12.1**

0.013

179

178

High school degree or GED

40.0

35.6

4.5

0.310

242

231

Some col ege

43.4

37.8

5.6*

0.098

362

375

2-year col ege degree or higher

47.3

46.0

1.3

0.795

203

159

Total household income

No income

35.6

41.6

-6.0

0.467

109

92

$1–$10,000

32.9

22.8

10.1***

0.004

349

312

$10,001–$20,000

38.4

34.8

3.5

0.384

269

278

More than $20,000

59.5

56.5

3.0

0.478

257

257

Reported barrier to employment

Yes

29.4

29.1

0.3

0.914†

410

387

No

50.7

42.6

8.1***

0.004†

576

556

Disability status

Received SSI/SSDI

18.4

13.1

5.3

0.238

124

154

Did not receive SSI/SSDI

45.6

41.3

4.3*

0.060

862

789

Percentage of rent and utilities expenses paid by household

0–25

34.8

27.2

7.6**

0.019

414

383

25.01–50

41.0

37.1

4.0

0.315

293

280

More than 50

56.3

52.4

3.9

0.338

245

256

Rent burdena

Lower

41.5

33.3

8.1***

0.009†

445

440

Higher

42.8

39.6

3.2

0.279†

469

446

Average total household income in prior month ($)

Employment status

Not employed

1,419

1,327

92

0.187

429

402

(continued)

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 185

Appendix E

Appendix Exhibit E.2. Characteristics, Family Self-Sufficiency 36-Month Survey Respondent Sample continued

Sample Sizes

Control Difference

FSS

Control

Outcome

FSS Group

Group

(Impact)

p-Value

Group

Group

Employed part-time

1,631

1,671

-41

0.723

246

230

Employed ful -time

2,122

2,084

38

0.694

274

272

Educational attainment

No degree or credential

1,539

1,473

66

0.549

171

170

High school degree or GED

1,613

1,499

114

0.250

236

222

Some col ege

1,656

1,661

-6

0.942

350

360

2-year col ege degree or higher

2,055

1,821

234

0.100

192

152

Total household income

No income

1,214

1,268

-54

0.713

108

91

$1–$10,000

1,290

1,292

-2

0.977

336

300

$10,001–$20,000

1,763

1,648

116

0.245

261

267

More than $20,000

2,303

2,246

57

0.618

242

243

Reported barrier to employment

Yes

1,588

1,488

100

0.175

394

368

No

1,767

1,720

47

0.499

555

536

Disability status

Received SSI/SSDI

1,655

1,387

268**

0.037†

120

146

Did not receive SSI/SSDI

1,697

1,673

24

0.664†

829

758

Percentage of rent and utilities expenses paid by household

0–25

1,340

1,333

6

0.928

400

366

25.01–50

1,803

1,697

106

0.278

281

278

More than 50

2,121

2,001

120

0.273

236

238

Rent burdena

Lower

1,625

1,472

153**

0.023††† 427

424

Higher

1,788

1,743

45

0.575†††

453

425

Has bank account (%)

Employment status

Not employed

61.9

54.5

7.4**

0.019

441

409

Employed part-time

65.7

60.4

5.3

0.179

250

238

Employed ful -time

75.5

73.9

1.6

0.637

283

283

(continued)

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 186

Appendix E

Appendix Exhibit E.2. Characteristics, Family Self-Sufficiency 36-Month Survey Respondent Sample continued

Sample Sizes

Control Difference

FSS

Control

Outcome

FSS Group

Group

(Impact)

p-Value

Group

Group

Educational attainment

No degree or credential

57.7

52.4

5.3

0.310

178

176

High school degree or GED

57.5

54.8

2.7

0.523

240

228

Some col ege

71.3

64.6

6.8**

0.039

356

367

2-year col ege degree or higher

82.1

71.6

10.6**

0.011

200

159

Total household income

No income

63.2

51.8

11.4

0.109

109

89

$1–$10,000

58.2

48.2

10.0***

0.008

344

307

$10,001–$20,000

71.5

66.9

4.6

0.231

267

274

More than $20,000

77.3

75.4

1.9

0.613

252

256

Reported barrier to employment

Yes

62.8

61.7

1.2

0.712††

407

383

No

70.7

61.1

9.7***

0.000††

567

547

Disability status

Received SSI/SSDI

72.1

63.8

8.3

0.167

126

154

Did not receive SSI/SSDI

66.7

60.9

5.7***

0.007

848

776

Percentage of rent and utilities expenses paid by household

0–25

59.0

51.3

7.7**

0.019

411

373

25.01–50

69.4

63.5

5.9

0.113

291

279

More than 50

78.3

74.5

3.8

0.305

239

255

Rent burdena

Lower

67.2

61.1

6.0**

0.039

439

432

Higher

68.3

60.8

7.5***

0.010

465

442

Average savings ($)

Employment status

Not employed

114

124

-10

0.785

421

403

Employed part-time

156

160

-3

0.954

241

230

Employed ful -time

185

245

-61

0.323

273

268

Educational attainment

No degree or credential

101

133

-31

0.548

171

175

High school degree or GED

156

149

7

0.905

233

223

(continued)

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 187

Appendix E

Appendix Exhibit E.2. Characteristics, Family Self-Sufficiency 36-Month Survey Respondent Sample continued

Sample Sizes

Control Difference

FSS

Control

Outcome

FSS Group

Group

(Impact)

p-Value

Group

Group

Some col ege

125

200

-75

0.111

342

352

2-year col ege degree or higher

251

121

129*

0.093

189

151

Total household income

No income

103

151

-48

0.589

105

88

$1–$10,000

105

139

-34

0.440

326

303

$10,001–$20,000

188

150

38

0.491

257

264

More than $20,000

200

204

-5

0.941

245

243

Reported barrier to employment

Yes

129

162

-33

0.413

389

374

No

160

172

-12

0.754

546

527

Disability status

Received SSI/SSDI

182

89

93

0.176

120

151

Did not receive SSI/SSDI

148

177

-30

0.322

815

750

Percentage of rent and utilities expenses paid by household

0–25

109

146

-37

0.352

391

365

25.01–50

185

144

40

0.459

281

270

More than 50

149

218

-69

0.235

231

244

Rent burdena

Lower

146

124

22

0.519†

415

420

Higher

161

191

-30

0.506†

451

429

Average debt ($)

Employment status

Not employed

13,575

12,181

1,394

0.227

430

404

Employed part-time

15,472

14,209

1,263

0.459

241

231

Employed ful -time

18,289

15,701

2,589

0.108

275

271

Educational attainment

No degree or credential

6,353

6,083

271

0.857

167

173

High school degree or GED

9,718

8,856

862

0.515

233

223

Some col ege

16,636

14,257

2,379*

0.063

353

366

2-year col ege degree or higher

29,964

26,585

3,379

0.259

193

144

(continued)

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 188

Appendix E

Appendix Exhibit E.2. Characteristics, Family Self-Sufficiency 36-Month Survey Respondent Sample continued

Sample Sizes

Control Difference

FSS

Control

Outcome

FSS Group

Group

(Impact)

p-Value

Group

Group

Total household income

No income

16,247

12,438

3,809

0.194

107

88

$1–$10,000

13,512

13,500

12

0.993

328

301

$10,001–$20,000

16,815

13,123

3,691**

0.030

261

266

More than $20,000

16,005

15,585

419

0.796

248

247

Reported barrier to employment

Yes

15,136

12,754

2,382*

0.069

397

376

No

15,651

14,448

1,202

0.256

549

530

Disability status

Received SSI/SSDI

14,745

11,958

2,787

0.235

122

150

Did not receive SSI/SSDI

15,480

14,162

1,318

0.134

824

756

Percentage of rent and utilities expenses paid by household

0–25

14,724

13,369

1,355

0.276

399

363

25.01–50

16,358

12,676

3,682**

0.026

277

274

More than 50

15,853

14,804

1,050

0.523

237

246

Rent burdena

Lower

15,554

13,332

2,222*

0.067

429

419

Higher

14,838

14,115

723

0.544

452

434

Financial situation better than last year (%)

Employment status

Not employed

53.0

43.8

9.2***

0.009

439

412

Employed part-time

54.4

42.9

11.6**

0.015

251

239

Employed ful -time

56.7

55.8

1.0

0.825

285

284

Educational attainment

No degree or credential

56.0

45.1

10.9*

0.067

175

175

High school degree or GED

53.3

47.7

5.6

0.237

239

228

Some col ege

53.9

49.9

4.0

0.301

362

372

2-year col ege degree or higher

53.5

44.8

8.7

0.127

199

160

Total household income

No income

60.4

43.7

16.7**

0.044

109

92

$1–$10,000

50.8

45.9

5.0

0.231

344

310

(continued)

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 189

Appendix E

Appendix Exhibit E.2. Characteristics, Family Self-Sufficiency 36-Month Survey Respondent Sample continued

Sample Sizes

Control Difference

FSS

Control

Outcome

FSS Group

Group

(Impact)

p-Value

Group

Group

$10,001–$20,000

52.9

47.4

5.5

0.223

266

275

More than $20,000

57.6

50.3

7.2

0.112

254

254

Reported barrier to employment

Yes

54.2

44.6

9.6***

0.009

401

385

No

55.1

48.5

6.7**

0.031

574

550

Disability status

Received SSI/SSDI

52.3

41.9

10.4

0.129

121

152

Did not receive SSI/SSDI

55.1

47.8

7.3***

0.004

854

783

Percentage of rent and utilities expenses paid by household

0–25

52.6

44.5

8.1**

0.028

410

378

25.01–50

56.3

46.2

10.1**

0.020

291

282

More than 50

56.2

50.9

5.3

0.263

241

251

Rent burdena

Lower

55.2

45.1

10.0***

0.003

441

436

Higher

53.3

48.6

4.8

0.168

463

443

In good, very good, or excellent health (%)

Employment status

Not employed

59.7

57.3

2.4

0.479

441

418

Employed part-time

66.3

62.3

3.9

0.382

255

239

Employed ful -time

72.0

70.3

1.6

0.689

284

281

Educational attainment

No degree or credential

63.3

51.0

12.4**

0.029

175

179

High school degree or GED

61.6

58.0

3.6

0.442

242

229

Some col ege

64.2

66.3

-2.0

0.577

359

372

2-year col ege degree or higher

74.9

69.2

5.7

0.249

204

158

Total household income

No income

62.3

54.6

7.7

0.364

108

91

$1–$10,000

65.5

59.3

6.1

0.118

345

312

$10,001–$20,000

61.3

60.7

0.7

0.880

269

277

More than $20,000

69.8

70.3

-0.5

0.905

256

254

(continued)

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 190

Appendix E

Appendix Exhibit E.2. Characteristics, Family Self-Sufficiency 36-Month Survey Respondent Sample continued

Sample Sizes

Control Difference

FSS

Control

Outcome

FSS Group

Group

(Impact)

p-Value

Group

Group

Reported barrier to employment

Yes

60.6

54.7

6.0

0.100

406

384

No

68.3

67.7

0.7

0.813

574

554

Disability status

Received SSI/SSDI

45.7

50.2

-4.5

0.507

124

154

Did not receive SSI/SSDI

67.9

64.8

3.1

0.183

856

784

Percentage of rent and utilities expenses paid by household

0–25

62.8

61.6

1.2

0.726

411

381

25.01–50

65.2

58.6

6.7

0.115

293

280

More than 50

67.0

68.0

-1.0

0.825

242

253

Rent burdena

Lower

65.7

59.3

6.4*

0.055

444

437

Higher

65.0

64.7

0.3

0.927

464

444

Sample size (total = 1,936)

988

948

FSS = Family Self-Sufficiency. GED = General Educational Development. SSI-SSDI = Supplemental Security Income-Social Security Disability Insurance.

aBased on HUD Housing Choice Voucher program regulations, households described as having a “lower rent burden” did not have to pay more than required out-of-pocket expenses for rent and utilities in month 1 (the month of random assignment) because their gross rent was less than or equal to the area payment standard (representing the maximum housing subsidy allowed) and because the household was paying less than or equal to 30 percent of their adjusted monthly income for rent and utilities (representing the expected percentage of household income to be paid for rent and utilities). In contrast, households described as having a “higher rent burden” paid more than required out-of-pocket expenses for rent and utilities because their gross rent was higher than the area payment standard and because the household was paying more than 30 percent of their adjusted monthly income for rent and utilities. Not shown in Exhibit: Results for 130

respondents (7 percent) who had a combination of “lower” and “higher” rent burden in month 1.

NOTES: The FSS 36-Month Survey respondent sample includes Housing Choice Voucher heads of household who were randomly assigned between October 18, 2013, and December 22, 2014, were age 18 to 61 at the time of random assignment, and responded to the FSS 36-Month Survey. Sample sizes for specific outcomes may vary because of missing values. Estimates were regression-adjusted using ordinary least squares, controlling for differences in sample member characteristics recorded at the time of random assignment.

No special weights were applied to responses to adjust for differences in sample size by housing authority. Rounding may cause slight discrepancies in calculating differences. A two-tailed t-test was applied to differences between the FSS group and the control group for continuous variables and selected outcomes expressed as proportions. The p-value indicates the likelihood that the difference between the FSS group and control group arose by chance. Statistical significance levels are indicated as: * = 10 percent; ** = 5 percent; ***

= 1 percent. The H-statistic test was used to test for statistically significant differences in impact estimates across different subgroups.

Statistical significance levels are indicated as: † = 10 percent; †† = 5 percent; and ††† = 1 percent.

SOURCES: MDRC calculations using baseline data, U.S. Department of Housing and Urban Development Inventory Management System (IMS)/PIH Information Center (PIC) data, and responses to the FSS 36-Month Survey.

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 191

Appendix E

Appendix Exhibit E.3. Impacts on Selected Indicators of Rent and Subsidies in Month 36, by Selected Baseline Characteristics, Family Self-Sufficiency Impact Sample

Sample Size

FSS

Control Difference

FSS

Control

Outcome

Group

Group

(Impact)

p-Value

Group

Group

Enrolled in HCV program in Month 36 (%)

Employment status

Not employed

83.5

81.5

2.0

0.374

572

550

Employed part-time

82.6

82.8

-0.2

0.953

336

312

Employed ful -time

76.6

70.2

6.4**

0.046

373

405

Educational attainment

No degree or credential

81.7

80.1

1.7

0.654

257

256

High school degree or GED

81.5

79.5

1.9

0.545

319

297

Some col ege

80.6

77.5

3.1

0.237

470

510

2-year col ege degree or higher

79.6

78.4

1.3

0.760

235

204

Total household income

No income

79.3

79.4

-0.1

0.990

115

108

$1–$10,000

83.8

81.2

2.6

0.340

433

410

$10,001–$20,000

85.4

83.3

2.1

0.465

342

352

More than $20,000

73.2

72.5

0.7

0.826

391

397

Reported barrier to employment

Yes

84.1

80.6

3.5

0.149

532

516

No

78.3

77.5

0.8

0.699

749

751

Disability status

Received SSI/SSDI

82.0

84.9

-2.9

0.488

176

187

Did not receive SSI/SSDI

80.9

77.4

3.5**

0.042

1105

1080

Percentage of rent and utilities expenses paid by household

0–25

85.4

82.1

3.3

0.153

529

520

25.01–50

83.8

85.1

-1.3

0.633

386

366

More than 50

70.2

67.5

2.7

0.439

325

358

Rent burdena

Lower

81.1

82.2

-1.1

0.633

591

593

Higher

80.0

75.3

4.6*

0.055

599

604

(continued)

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 192

Appendix E

Appendix Exhibit E.3. Impacts on Selected Indicators of Rent and Subsidies in Month 36, by Selected Baseline Characteristics, Family Self-Sufficiency Impact Sample continued Sample Size

FSS

Control Difference

FSS

Control

Outcome

Group

Group

(Impact)

p-Value

Group

Group

Average gross rent in Month 36, if received HCV ($)

Employment status

Not employed

1,365

1,365

0

479

447

Employed part-time

1,407

1,426

-19

275

261

Employed ful -time

1,452

1,448

3

286

284

Educational attainment

No degree or credential

1,417

1,496

-78

210

205

High school degree or GED

1,347

1,351

-4

263

233

Some col ege

1,384

1,395

-11

381

393

2-year col ege degree or higher

1,470

1,413

57

186

161

Total household income

No income

1,233

1,231

1

93

84

$1–$10,000

1,297

1,340

-43

366

330

$10,001–$20,000

1,391

1,381

11

293

292

More than $20,000

1,575

1,575

0

288

286

Reported barrier to employment

Yes

1,347

1,363

-15

450

413

No

1,436

1,440

-4

590

579

Disability status

Received SSI/SSDI

1,254

1,256

-2

144

159

Did not receive SSI/SSDI

1,424

1,433

-9

896

833

Percentage of rent and utilities expenses paid by household

0–25

1,420

1,430

-10

452

427

25.01–50

1,394

1,376

18

326

309

More than 50

1,329

1,386

-57

229

241

Rent burdena

Lower

1,352

1,334

18

481

486

Higher

1,451

1,486

-35

481

453

(continued)

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 193

Appendix E

Appendix Exhibit E.3. Impacts on Selected Indicators of Rent and Subsidies in Month 36, by Selected Baseline Characteristics, Family Self-Sufficiency Impact Sample continued Sample Size

FSS

Control Difference

FSS

Control

Outcome

Group

Group

(Impact)

p-Value

Group

Group

Average family share in Month 36, if received HCV ($)

Employment status

Not employed

409

403

6

479

447

Employed part-time

532

506

26

275

261

Employed ful -time

675

654

21

286

284

Educational attainment

No degree or credential

515

502

13

210

205

High school degree or GED

489

478

11

263

233

Some col ege

517

491

27

381

393

2-year col ege degree or higher

588

513

75

186

161

Total household income

No income

350

348

2

93

84

$1–$10,000

360

361

0

366

330

$10,001–$20,000

497

477

20

293

292

More than $20,000

767

750

16

288

286

Reported barrier to employment

Yes

443

457

-14

450

413

No

577

526

50

590

579

Disability status

Received SSI/SSDI

458

449

9

144

159

Did not receive SSI/SSDI

525

510

15

896

833

Percentage of rent and utilities expenses paid by household

0–25

398

371

28

452

427

25.01–50

535

535

0

326

309

More than 50

725

681

43

229

241

Rent burdena

Lower

488

451

38

481

486

Higher

552

550

2

481

453

(continued)

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 194

Appendix E

Appendix Exhibit E.3. Impacts on Selected Indicators of Rent and Subsidies in Month 36, by Selected Baseline Characteristics, Family Self-Sufficiency Impact Sample continued Sample Size

FSS

Control Difference

FSS

Control

Outcome

Group

Group

(Impact)

p-Value

Group

Group

Average housing subsidy in Month 36, if received HCV ($)

Employment status

Not employed

963

962

1

479

447

Employed part-time

883

920

-36

275

261

Employed ful -time

793

800

-6

286

284

Educational attainment

No degree or credential

911

993

-82

210

205

High school degree or GED

858

873

-15

263

233

Some col ege

883

908

-25

381

393

2-year col ege degree or higher

893

901

-8

186

161

Total household income

No income

882

883

-1

93

84

$1–$10,000

943

979

-36

366

330

$10,001–$20,000

901

904

-3

293

292

More than $20,000

827

831

-4

288

286

Reported barrier to employment

Yes

914

906

8

450

413

No

868

917

-50

590

579

Disability status

Received SSI/SSDI

800

810

-9

144

159

Did not receive SSI/SSDI

909

924

-15

896

833

Percentage of rent and utilities expenses paid by household

0–25

1,024

1,059

-35

452

427

25.01–50

865

841

25

326

309

More than 50

630

712

-82

229

241

Rent burdena

Lower

870

882

-13

481

486

Higher

911

939

-28

481

453

Sample size (total = 2,548)

1,281

1,267

(continued)

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 195

Appendix E

Appendix Exhibit E.3. Impacts on Selected Indicators of Rent and Subsidies in Month 36, by Selected Baseline Characteristics, Family Self-Sufficiency Impact Sample continued FSS = Family Self-Sufficiency. GED = General Educational Development. HCV = Housing Choice Voucher. SSI-SSDI = Supplemental Security Income-Social Security Disability Insurance.

aBased on HUD Housing Choice Voucher program regulations, households described as having a “lower rent burden” did not have to pay more than required out-of-pocket expenses for rent and utilities in month 1 (the month of random assignment) because their gross rent was less than or equal to the area payment standard (representing the maximum housing subsidy allowed) and because the household was paying less than or equal to 30 percent of their adjusted monthly income for rent and utilities (representing the expected percentage of household income to be paid for rent and utilities). In contrast, households described as having a “higher rent burden” paid more than required out-of-pocket expenses for rent and utilities because their gross rent was higher than the area payment standard and because the household was paying more than 30 percent of their adjusted monthly income for rent and utilities. Not shown in exhibit: Results for 160

respondents (6 percent) who had a combination of “lower” and “higher” rent burden in month 1.

NOTES: The FSS impact sample includes Housing Choice Voucher households that were randomly assigned between October 18, 2013, and December 22, 2014, and with a head of household age 18 to 61 at the time of random assignment. Sample sizes for specific outcomes may vary because of missing values. Estimates were regression-adjusted using ordinary least squares, controlling for differences in sample member characteristics recorded at the time of random assignment. No special weights were applied to responses to adjust for differences in sample size by housing authority. Rounding may cause slight discrepancies in calculating differences. A two-tailed t-test was applied to differences between the FSS group and the control group for continuous variables and selected outcomes expressed as proportions. The p-value indicates the likelihood that the difference between the FSS group and control group arose by chance. Statistical significance levels are indicated as: * = 10 percent; ** = 5 percent; *** = 1 percent. The H-statistic test was used to test for statistically significant differences in impact estimates across different subgroups. Statistical significance levels are indicated as: † = 10 percent; †† =

5 percent; and ††† = 1 percent. Results displayed in italics are non-experimental. No tests of statistical significance were performed on differences between research groups in means or proportions.

SOURCES: MDRC calculations using baseline data and U.S. Department of Housing and Urban Development Inventory Management System (IMS)/PIH Information Center (PIC) data.

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 196

Appendix E

Appendix Exhibit E.4. Confirmatory Comparisons (Impacts) of Employment and Earnings Outcomes in Years 1 to 3, by Program Approach, Family Self-Sufficiency Impact Sample Sample Sizes

FSS

Control Difference

FSS

Control

Outcome

Group

Group

(Impact)

p-Value

Group

Group

Average quarterly employment rate (%)

Emphasis on job search and post-employment services

Low

65.0

63.4

1.7

0.445

286

289

Medium

61.4

61.0

0.4

0.860

351

352

High

64.0

63.9

0.1

0.938

645

625

Emphasis on education and training

Low

63.8

61.6

2.2

0.299†

386

392

Medium

62.9

65.4

-2.5

0.138†

578

568

High

64.0

60.5

3.5

0.141†

318

306

Emphasis on financial services

Low

57.7

58.3

-0.6

0.831

239

235

Medium

62.8

64.0

-1.2

0.561

420

419

High

66.3

64.0

2.2

0.175

623

612

Program emphasis on monitoring and engagement

Low

62.3

60.8

1.5

0.446

469

468

Medium

65.8

64.5

1.3

0.434

551

547

High

60.4

64.2

-3.8

0.140

262

251

Total earnings ($)

Emphasis on job search and post-employment services

Low

39,713

40,352

-639

0.776

286

289

Medium

42,647

39,295

3,352

0.128

351

352

High

41,797

43,435

-1,638

0.316

645

625

Emphasis on education and training

Low

40,189

38,363

1,826

0.356

386

392

Medium

40,954

42,922

-1,968

0.232

578

568

High

44,315

43,244

1,071

0.670

318

306

Emphasis on financial services

Low

33,145

32,933

212

0.924

239

235

Medium

41,452

43,739

-2,287

0.261

420

419

High

45,089

43,201

1,889

0.261

623

612

(continued)

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 197

Appendix E

Appendix Exhibit E.4. Confirmatory Comparisons (Impacts) of Employment and Earnings Outcomes in Years 1 to 3, by Program Approach, Family Self-Sufficiency Impact Sample continued

Sample Sizes

FSS

Control Difference

FSS

Control

Outcome

Group

Group

(Impact)

p-Value

Group

Group

Program emphasis on monitoring and engagement

Low

41,128

39,388

1,740

0.345†††

469

468

Medium

42,195

40,374

1,821

0.279†††

551

547

High

40,664

48,671

-8,006***

0.003†††

262

251

Average annual earnings greater than $25,000 (%)

Emphasis on job search and post-employment services

Low

17.2

20.0

-2.8

0.336

286

289

Medium

21.8

18.6

3.1

0.218

351

352

High

20.5

21.6

-1.1

0.558

645

625

Emphasis on education and training

Low

20.1

16.2

3.9*

0.098

386

392

Medium

18.9

21.1

-2.2

0.253

578

568

High

23.6

23.2

0.4

0.889

318

306

Emphasis on financial services

Low

14.2

14.1

0.1

0.977

239

235

Medium

19.3

22.4

-3.1

0.180

420

419

High

23.2

21.1

2.1

0.286

623

612

Program emphasis on monitoring and engagement

Low

20.4

19.6

0.8

0.706†

469

468

Medium

19.9

18.0

1.8

0.375†

551

547

High

20.3

27.1

-6.8**

0.026†

262

251

Sample size (total = 2,548)

1,282

1,266

FSS = Family Self-Sufficiency.

NOTES: The FSS impact sample includes Housing Choice Voucher households that were randomly assigned between October 18, 2013, and December 22, 2014, and with a head of household age 18 to 61 at the time of random assignment. Average quarterly employment rate is calculated as: total quarters with employment divided by total quarters of follow-up, expressed as a percentage.

Estimates were regression-adjusted using ordinary least squares, controlling for pre-random assignment characteristics of sample members. Rounding may cause slight discrepancies in calculating differences. A two-tailed t-test was applied to differences between research groups. The p-value indicates the likelihood that the difference between the FSS group and control group arose by chance.

Statistical significance levels are indicated as: * = 10 percent; ** = 5 percent; and *** = 1 percent. The H-statistic test was used to test for statistically significant differences in impact estimates across different subgroups. Statistical significance levels are indicated as: † = 10

percent; †† = 5 percent; and ††† = 1 percent. “Confirmatory comparisons” use qualitative and quantitative data collected at the time of random assignment or data collected for FSS program participants who enrolled in FSS before the start of the evaluation and are not included in the outcomes.

SOURCES: MDRC calculations using baseline data, data collected from Individual Training and Services Plan forms, information provided by FSS administrators and case managers, and quarterly wage data from the National Directory of New Hires.

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 198

Appendix E

Appendix Exhibit E.5. Impacts on Selected Survey Outcomes by Selected Program Implementation Features, Family Self-Sufficiency 36-Month Survey Respondent Sample Sample Sizes

FSS

Control Difference

FSS

Control

Outcomes

Group

Group

(Impact)

p-Value

Group

Group

Currently employed with full-time hours (%)

Emphasis on job search and post-employment services

Low

39.0

34.1

4.8

0.267

223

217

Medium

48.3

34.5

13.8***

0.001

260

256

High

39.6

39.9

-0.3

0.913

503

470

Emphasis on education and training

Low

40.1

32.6

7.6**

0.050

285

297

Medium

43.8

43.9

-0.1

0.982

456

429

High

38.2

32.0

6.3

0.149

245

217

Emphasis on financial services

Low

32.6

31.1

1.5

0.752

184

177

Medium

41.1

38.6

2.5

0.502

317

320

High

44.2

40.1

4.1

0.171

485

446

Emphasis on monitoring and engagement

Low

41.9

34.4

7.5**

0.033†

358

343

Medium

44.8

38.2

6.6**

0.038†

425

410

High

34.6

40.4

-5.7

0.228†

203

190

Average total household income in prior month ($)

Emphasis on job search and post-employment services

Low

1,629

1,413

216**

0.033

214

209

Medium

1,710

1,635

75

0.466

252

246

High

1,724

1,706

18

0.804

483

449

Emphasis on education and training

Low

1,565

1,531

34

0.699

278

290

Medium

1,711

1,719

-8

0.916

440

406

High

1,773

1,619

154

0.170

231

208

Emphasis on financial services

Low

1,641

1,521

119

0.313

178

175

Medium

1,649

1,740

-91

0.304

302

306

High

1,712

1,618

94

0.202

469

423

(continued)

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 199

Appendix E

Appendix Exhibit E.5. Impacts on Selected Survey Outcomes by Selected Program Implementation Features, Family Self-Sufficiency 36-Month Survey Respondent

Sample continued

Sample Sizes

FSS

Control Difference

FSS

Control

Outcomes

Group

Group

(Impact)

p-Value

Group

Group

Emphasis on monitoring and engagement

Low

1,579

1,648

-68

0.405††

345

334

Medium

1,753

1,558

195***

0.009††

412

391

High

1,701

1,805

-104

0.414††

192

179

Has bank account (%)

Emphasis on job search and post-employment services

Low

62.9

53.6

9.3**

0.026

218

216

Medium

71.0

63.0

8.0**

0.043

258

252

High

67.7

63.9

3.8

0.163

498

462

Emphasis on education and training

Low

65.9

57.6

8.3**

0.027

283

291

Medium

65.8

59.6

6.2**

0.036

448

425

High

72.7

69.3

3.4

0.401

243

214

Emphasis on financial services

Low

65.9

59.4

6.4

0.162

181

178

Medium

63.0

54.7

8.3**

0.021

309

315

High

71.0

66.6

4.4

0.119

484

437

Emphasis on monitoring and engagement

Low

65.5

59.2

6.3*

0.060

353

336

Medium

67.1

61.6

5.4*

0.072

421

405

High

72.5

63.5

8.9**

0.043

200

189

Average savings ($)

Emphasis on job search and post-employment services

Low

96

142

-47

0.283

207

210

Medium

210

189

21

0.735

248

249

High

142

162

-20

0.614

480

442

Emphasis on education and training

Low

132

129

3

0.946

272

287

Medium

132

197

-65

0.104

432

411

High

198

159

39

0.555

231

203

(continued)

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 200

Appendix E

Appendix Exhibit E.5. Impacts on Selected Survey Outcomes by Selected Program Implementation Features, Family Self-Sufficiency 36-Month Survey Respondent

Sample continued

Sample Sizes

FSS

Control Difference

FSS

Control

Outcomes

Group

Group

(Impact)

p-Value

Group

Group

Emphasis on financial services

Low

50

113

-63

0.146

177

171

Medium

149

154

-5

0.908

296

308

High

180

204

-24

0.595

462

422

Emphasis on monitoring and engagement

Low

117

134

-17

0.694

341

325

Medium

136

208

-72

0.104

402

393

High

216

150

65

0.310

192

183

Average debt ($)

Emphasis on job search and post-employment services

Low

16,397

14,633

1,764

0.329

213

208

Medium

14,223

12,761

1,462

0.340

254

242

High

15,694

13,814

1,880

0.107

479

456

Emphasis on education and training

Low

17,411

15,335

2,076

0.191

272

289

Medium

15,015

13,462

1,553

0.204

438

411

High

12,883

13,287

-404

0.808

236

206

Emphasis on financial services

Low

16,912

14,204

2,708

0.177

179

176

Medium

15,010

13,721

1,289

0.374

302

309

High

15,075

13,644

1,431

0.223

465

421

Emphasis on monitoring and engagement

Low

12,432

11,820

612

0.637

339

330

Medium

16,531

15,160

1,370

0.287

411

391

High

17,597

14,963

2,634

0.194

196

185

Financial situation better than last year (%)

Emphasis on job search and post-employment services

Low

54.6

45.3

9.2*

0.069

221

217

Medium

55.3

45.6

9.8**

0.036

257

254

High

55.1

47.7

7.4**

0.024

497

464

(continued)

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 201

Appendix E

Appendix Exhibit E.5. Impacts on Selected Survey Outcomes by Selected Program Implementation Features, Family Self-Sufficiency 36-Month Survey Respondent

Sample continued

Sample Sizes

FSS

Control Difference

FSS

Control

Outcomes

Group

Group

(Impact)

p-Value

Group

Group

Emphasis on education and training

Low

53.2

46.4

6.9

0.120

285

294

Medium

55.9

52.5

3.4

0.319

449

425

High

54.8

36.1

18.7***

0.000

241

216

Emphasis on financial services

Low

58.2

43.3

14.8***

0.007

183

179

Medium

53.5

50.5

3.0

0.459

313

314

High

53.6

46.4

7.2**

0.033

479

442

Emphasis on monitoring and engagement

Low

51.3

41.3

9.9**

0.012

353

339

Medium

56.9

51.4

5.5

0.122

421

408

High

55.7

47.9

7.7

0.146

201

188

In good, very good, or excellent health (%)

Emphasis on job search and post-employment services

Low

68.8

61.4

7.4

0.114

222

217

Medium

67.2

65.9

1.3

0.771

259

255

High

62.2

61.1

1.1

0.735

499

466

Emphasis on education and training

Low

65.9

67.1

-1.3

0.759

285

294

Medium

63.3

60.4

2.9

0.362

453

429

High

66.5

61.0

5.5

0.254

242

215

Emphasis on financial services

Low

52.9

61.6

-8.7*

0.099

182

178

Medium

71.2

64.4

6.8*

0.072

317

319

High

65.0

62.0

3.0

0.360

481

441

Emphasis on monitoring and engagement

Low

63.0

60.1

2.9

0.438

357

341

Medium

67.2

62.9

4.3

0.199

421

406

High

66.0

63.7

2.3

0.638

202

191

Sample size (total = 1,936)

988

948

(continued)

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 202

Appendix E

Appendix Exhibit E.5. Impacts on Selected Survey Outcomes by Selected Program Implementation Features, Family Self-Sufficiency 36-Month Survey Respondent

Sample continued

FSS = Family Self-Sufficiency.

NOTES: The FSS 36-Month Survey respondent sample includes Housing Choice Voucher heads of household who were randomly assigned between October 18, 2013, and December 22, 2014, were age 18 to 61 at the time of random assignment, and responded to the FSS 36-Month Survey. Sample sizes for specific outcomes may vary because of missing values. Estimates were regression-adjusted using ordinary least squares, controlling for pre-random assignment characteristics of sample members. No special weights were applied to responses to adjust for differences in sample size by housing authority. Rounding may cause slight discrepancies in calculating differences. A two-tailed t-test was applied to differences between the FSS group and the control group for continuous variables and selected outcomes expressed as proportions. The p-value indicates the likelihood that the difference between the FSS group and control group arose by chance. Statistical significance levels are indicated as: * = 10 percent; ** = 5 percent; and *** = 1 percent. The H-statistic test was used to test for statistically significant differences in impact estimates across different subgroups. Statistical significance levels are indicated as: † = 10 percent; †† = 5 percent; and ††† = 1 percent.

SOURCES: MDRC calculations using baseline data, data collected from Individual Training and Services Plan forms, information provided by FSS administrators and case managers, and responses to the FSS 36-Month Survey.

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 203

Appendix E

Appendix Exhibit E.6. Impacts on Selected Indicators of Rent and Subsidies in Month 36, by Selected Program Implementation Features, Family Self-Sufficiency Impact Sample Sample Size

FSS

Control Difference

FSS

Control

Outcome

Group

Group

(Impact)

p-Value

Group

Group

Enrolled in HCV program in Month 36 (%)

Emphasis on job search and post-employment services

Low

79.4

78.8

0.5

0.883

282

287

Medium

87.9

86.8

1.1

0.666

351

352

High

77.9

73.8

4.1*

0.086

648

628

Emphasis on education and training

Low

82.2

79.7

2.5

0.378

385

392

Medium

79.1

75.8

3.2

0.189

577

567

High

81.7

83.3

-1.6

0.616

319

308

Emphasis on financial services

Low

73.0

69.9

3.2

0.450

238

235

Medium

85.5

83.7

1.7

0.486

419

418

High

80.4

78.9

1.6

0.495

624

614

Emphasis on monitoring and engagement

Low

83.5

80.7

2.7

0.289

472

472

Medium

78.3

75.2

3.2

0.217

547

545

High

82.2

81.4

0.8

0.827

262

250

Average gross rent in Month 36, if received HCV ($)

Emphasis on job search and post-employment services

Low

1,164

1,143

21

224

226

Medium

1,504

1,496

9

308

306

High

1,442

1,472

-30

508

460

Emphasis on education and training

Low

1,198

1,208

-10

319

310

Medium

1,381

1,403

-22

460

426

High

1,679

1,647

32

261

256

Emphasis on financial services

Low

1,096

1,090

7

174

164

Medium

1,484

1,509

-25

360

348

High

1,432

1,450

-17

506

480

(continued)

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 204

Appendix E

Appendix Exhibit E.6. Impacts on Selected Indicators of Rent and Subsidies in Month 36, by Selected Program Implementation Features, Family Self-Sufficiency Impact Sample continued

Sample Size

FSS

Control Difference

FSS

Control

Outcome

Group

Group

(Impact)

p-Value

Group

Group

Emphasis on monitoring and engagement

Low

1,427

1,459

-32

395

380

Medium

1,206

1,214

-7

430

408

High

1,742

1,682

61

215

204

Average family share in Month 36, if received HCV ($)

Emphasis on job search and post-employment services

Low

438

397

41

224

226

Medium

584

525

59

308

306

High

508

536

-29

508

460

Emphasis on education and training

Low

469

434

35

319

310

Medium

492

512

-20

460

426

High

625

552

73

261

256

Emphasis on financial services

Low

436

419

17

174

164

Medium

517

525

-8

360

348

High

542

511

31

506

480

Emphasis on monitoring and engagement

Low

528

514

14

395

380

Medium

483

436

47

430

408

High

567

595

-28

215

204

Average housing subsidy in Month 36, if received HCV ($)

Emphasis on job search and post-employment services

Low

729

746

-17

224

226

Medium

935

976

-41

308

306

High

944

935

8

508

460

Emphasis on education and training

Low

740

775

-34

319

310

Medium

898

896

1

460

426

High

1,062

1,094

-32

261

256

(continued)

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 205

Appendix E

Appendix Exhibit E.6. Impacts on Selected Indicators of Rent and Subsidies in Month 36, by Selected Program Implementation Features, Family Self-Sufficiency Impact Sample continued

Sample Size

FSS

Control Difference

FSS

Control

Outcome

Group

Group

(Impact)

p-Value

Group

Group

Emphasis on financial services

Low

666

670

-4

174

164

Medium

967

984

-17

360

348

High

909

942

-34

506

480

Emphasis on monitoring and engagement

Low

899

945

-46

395

380

Medium

746

781

-35

430

408

High

1,175

1,086

89

215

204

Sample size (total = 2,548)

1,281

1,267

FSS = Family Self-Sufficiency. HCV = Housing Choice Voucher.

NOTES: The FSS impact sample includes Housing Choice Voucher households that were randomly assigned between October 18, 2013, and December 22, 2014, and with a head of household age 18 to 61 at the time of random assignment. Sample sizes for specific outcomes may vary because of missing values. Estimates were regression-adjusted using ordinary least squares, controlling for differences in sample member characteristics recorded at the time of random assignment. No special weights were applied to responses to adjust for differences in sample size by housing authority. Rounding may cause slight discrepancies in calculating differences. A two-tailed t-test was applied to differences between the FSS group and the control group for continuous variables and selected outcomes expressed as proportions. The p-value indicates the likelihood that the difference between the FSS group and control group arose by chance. Statistical significance levels are indicated as: * = 10 percent; ** = 5 percent; *** = 1 percent. The H-statistic test was used to test for statistically significant differences in impact estimates across different subgroups. Statistical significance levels are indicated as: † = 10 percent; †† =

5 percent; and ††† = 1 percent. Results displayed in italics are non-experimental. No tests of statistical significance were performed on differences between research groups in means or proportions.

SOURCES: MDRC calculations using baseline data, data collected from Individual Training and Services Plan forms, information provided by FSS administrators and case managers, and U.S. Department of Housing and Urban Development Inventory Management System (IMS)/PIH Information Center (PIC) data.

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 206

Appendix E

Appendix Exhibit E.7. Exploratory Comparisons of Employment and Earnings Outcomes in Years 1 to 3 for Sites with Similar Escrow Accrual and Service Use Outcomes, Family Self-Sufficiency Impact Sample

Sample Sizes

FSS

Control Difference

FSS

Control

Outcome

Group

Group

(Impact)

p-Value

Group

Group

Average quarterly employment rate (%)

Composite indicator of escrow balance in Month 36

Low

64.3

65.4

-1.1

0.544

452

446

Medium

65.8

65.8

0.0

0.995

345

342

High

61.0

58.9

2.1

0.301

485

478

Composite indicator of FSS service use in Year 3a

Low

61.6

64.0

-2.4

0.317

287

287

Medium

63.7

63.6

0.0

0.994

325

320

High

61.2

61.6

-0.4

0.868

349

336

Total earnings ($)

Composite indicator of escrow balance in Month 36

Low

41,159

44,326

-3,167*

0.097††

452

446

Medium

43,076

43,800

-724

0.744††

345

342

High

41,021

37,278

3,743**

0.041††

485

478

Composite indicator of FSS service use in Year 3a

Low

39,697

39,586

110

0.958

287

287

Medium

38,474

39,094

-621

0.777

325

320

High

43,212

47,341

-4,129*

0.084

349

336

Average annual earnings greater than $25,000 (%)

Composite indicator of escrow balance in Month 36

Low

20.1

21.5

-1.4

0.532

452

446

Medium

20.7

22.7

-2.0

0.471

345

342

High

20.3

17.1

3.2

0.114

485

478

Composite indicator of FSS service use in Year 3a

Low

19.8

19.6

0.2

0.930

287

287

Medium

16.3

18.8

-2.5

0.351

325

320

High

22.6

26.3

-3.7

0.159

349

336

Sample size (total = 2,548)

1,282

1,266

(continued)

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 207

Appendix E

Appendix Exhibit E.7. Exploratory Comparisons of Employment and Earnings Outcomes in Years 1 to 3 for Sites with Similar Escrow Accrual and Service Use Outcomes, Family Self-Sufficiency Impact Sample continued

FSS = Family Self-Sufficiency.

aEstimates include impact sample members randomly assigned in the 12 PHAs in the FSS service-use sample.

NOTES: The FSS impact sample includes Housing Choice Voucher households that were randomly assigned between October 18, 2013, and December 22, 2014, and with a head of household age 18 to 61 at the time of random assignment. Average quarterly employment rate is calculated as: total quarters with employment divided by total quarters of follow-up, expressed as a percentage.

Estimates were regression-adjusted using ordinary least squares, controlling for pre-random assignment characteristics of sample members. Rounding may cause slight discrepancies in calculating differences. A two-t ailed t-test was applied to differences between research groups. The p-value indicates the likelihood that the difference between the FSS group and control group arose by chance.

Statistical significance levels are indicated as: * = 10 percent; ** = 5 percent; and *** = 1 percent. The H-statistic test was used to test for statistically significant differences in impact estimates across different subgroups. Statistical significance levels are indicated as: † = 10

percent; †† = 5 percent; and ††† = 1 percent. For escrow and service use categories, composite scores of -1.0 or lower were considered to be “low;” scores between -1.0 and +1.0 were considered to be “medium;” and scores of +1.0 or higher were considered to be “high.” See exhibit 7.2 and exhibit E.9. Results are displayed in italics to emphasize their exploratory nature.

SOURCES: MDRC calculations from baseline data, housing authority administrative data, and quarterly wage data from the National Directory of New Hires.

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 208

Appendix E

Appendix Exhibit E.8. Impacts on Employment and Earnings, Years 1 to 3, by Self-Reported Employment Status at Random Assignment, Family Self-Sufficiency Impact Sample FSS

Control Difference

Outcomes

Group

Group

(Impact)

p-Value

Not employed

Ever employed (%)

68.6

66.5

2.1

0.378

Average quarterly employment rate (%)

42.0

41.2

0.8

0.667

Employed in all quarters (%)

8.9

11.2

-2.3

0.178

Total earnings ($)

20,923

20,109

814

0.572

Sample size (total = 1,126)

575

551

Employed

Ever employed (%)

95.2

93.7

1.5

0.164

Average quarterly employment rate (%)

80.5

80.3

0.2

0.898

Employed in all quarters (%)

51.6

54.7

-3.1

0.209

Total earnings ($)

58,060

58,403

-343

0.835

Sample size (total = 1,422)

707

715

FSS = Family Self-Sufficiency.

NOTES: The FSS impact sample includes Housing Choice Voucher households that were randomly assigned between October 18, 2013, and December 22, 2014, and with a head of household age 18 to 61 at the time of random assignment. Average quarterly employment rate is calculated as total quarters with employment divided by total quarters of follow-up, expressed as a percentage.

Estimates were regression-adjusted using ordinary least squares, controlling for pre-random assignment characteristics of sample members. Rounding may cause slight discrepancies in calculating differences. A two-tailed t-test was applied to differences between research groups. The p-value indicates the likelihood that the difference between the FSS group and control group arose by chance.

Statistical significance levels are indicated as: * = 10 percent; ** = 5 percent; and *** = 1 percent. The H-statistic test was used to test for statistically significant differences in impact estimates across different subgroups. Statistical significance levels are indicated as: † = 10

percent; †† = 5 percent; and ††† = 1 percent.

SOURCES: MDRC calculations using baseline data and quarterly wage data from the National Directory of New Hires.

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 209

Appendix E

Appendix Exhibit E.9. Methodology for Creating Site Clusters

Site clusters were created from a variety of source data, described in exhibit 7.2. Some site clusters were created from a single measure, whereas others were created from two or three measures. Measures were created using the fol owing procedure:

• Calculate site-level mean values for each source measure.

• Calculate a cross-site mean (“mean of means”) for each source measure.

• Calculate the cross-site standard deviation for each source measure.

• Calculate site-level z-scores for each source measure. For each site (PHA), subtract the site-level mean from the cross-site mean. Then, divide the difference by the cross-site standard deviation.

• For site clusters created from two or three source measures, sum the z-scores and then divide by the number of component measures to create an average composite score.

• Group sites with similar scores into high, medium, or low categories. Sites with an average z-score value above 0.5 were grouped in the high category, whereas sites with an average z-score below -0.5 were grouped in the low category. The remaining sites with average z-scores of between -0.5 and +0.5 were grouped in the medium sites.

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 210

Appendix F. Family Self-Sufficiency 36-Month Survey Response Analysis Appendix F. Family Self-Sufficiency

36-Month Survey Response Analysis

Many of the estimates of FSS’s effects on use of services and on financial and material wel -being outcomes were calculated using responses to the FSS 36-Month Survey. When only a subset of the sample completes a survey, potential issues can arise about the reliability of results estimated for survey respondents and whether results for respondents can be generalized to all study participants.

This appendix summarizes the results of tests of the reliability and generalizability of impact estimates calculated with survey responses. First, the appendix assesses whether research group differences in the 36-month survey-based outcome measures are unbiased (and therefore reliable) indicators of FSS’s effects.

Survey results are considered to be unbiased if a large proportion of each research group responded to the survey and if respondents in both research groups closely resemble each other in baseline characteristics, such as employment status and educational attainment, that would be likely to affect study participants’

employment, earnings, and financial wel -being after study entry.

Second, this appendix considers whether impact results estimated for survey respondents may be generalized to all study participants. Survey results are considered to be generalizable if it can be inferred with confidence that the analysis would have reached similar conclusions about FSS’s effects on financial outcomes had every study participant completed a 36-month survey interview.

Overal , the results show that the survey is reliable and that results for the 36-month survey respondent sample can be generalized to the report sample.

Main Findings

• A high response rate (77 percent) was achieved: more than three-fourths of sample members of each research group responded to the 36-month survey.

• Among 36-month survey respondents, characteristics at baseline were similar for the two research groups.

No systematic differences between the groups were found.

• Some caution about the generalizability of survey findings is warranted be- cause a comparison of survey respondents and non-respondents shows statistically significant differences in pre-random-assignment characteristics. Most of these differences are small (less than 5 percentage points), which limits concerns about the generalizability of the findings.

• FSS’s impacts on employment and earnings among respondents are similar to the impacts for the larger impact sample. This finding provides additional evidence that findings from the survey are generalizable to the larger impact sample.

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 211

Appendix F. Family Self-Sufficiency 36-Month Survey Response Analysis Survey Sample Selection and Administration

As noted in chapter 1, the impact sample includes 2,556 sample members who were enrol ed in October 2013

through December 2014 in the 18 public housing agencies participating in the national evaluation. Nearly al impact sample members were eligible to respond to the FSS 36-Month Survey. Thirty-eight study participants (or 1 percent of impact sample) were excluded because of death, incarceration, or lack of fluency in English or Spanish. The remaining study participants are referred to as the fielded sample, which totaled 2,518 sample members. From February 25, 2017, through August 20, 2017, the survey firm for the study, M. Davis and Company (MDAC), attempted to conduct 40-minute phone interviews with everyone in the fielded sample.

Survey Response Rates

Sample members who were interviewed for the 36-month survey are referred to as “survey respondents,” or the respondent sample, while sample members who were not interviewed are known as “non-respondents” or the non-respondent sample. As exhibit F.1 shows, a total of 1,936 sample members, or about 77 percent of the fielded sample, completed the survey.122 The majority of the non-respondent sample either refused to be interviewed or could not be located. The response rates by research group were also similar: About 78 percent of the FSS

group and 76 percent of the control group completed the 36-month survey.

Although the overall response rates are high, whenever the response rate is lower than 100 percent, nonresponse bias may occur. Differences may exist between the respondent sample and the larger impact and fielded samples, owing to differences between the sample members who completed a survey and those who did not.

Furthermore, the estimates may be biased if background characteristics differ between the research groups in the respondent sample.

Comparison of Respondents and Non-Respondents Within the

Fielded Survey Sample

In order to examine whether there are systematic differences between those who responded to the survey and those who did not, a (0/1) indicator of survey respondent status was created (in which survey respondents receive a 1 and non-respondents receive a 0), and then logistic regression analysis was used to identify whether any pre-random-assignment characteristics were significantly related to the indicator.

Exhibit F.2 shows the estimated regression coefficients for the probability of being a respondent. As can be noted from this exhibit, besides background characteristics such as race, age, and number of children, a (0/1) indicator of membership in the FSS group was included in the model. This procedure tests for differences in characteristics likely to affect service-use and financial outcomes. The second column of the exhibit provides the parameter estimates that indicate the effect of each variable on the probability of completing the survey.

The p-values show the level of statistical significance of this relationship.

The results show that there were a few characteristics that were statistically significant in predicting whether someone would complete a 36-month interview. Older adults were more likely to respond to the survey.123

Fielded sample members who are African American, female, attained a high school degree or equivalent or 122 One respondent did not complete the survey but answered most questions and was included in the respondent sample.

123 Adults age 45 to 61 are the reference group for comparisons by age. The negative coefficients for the two younger age groups, shown in exhibit F.2, imply that response rates were highest for the oldest group.

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 212

Appendix F. Family Self-Sufficiency 36-Month Survey Response Analysis higher,124 had a checking or savings account, or had a small amount of savings also had a greater likelihood of responding to the survey. Finally, fielded sample members randomly assigned in two housing agencies (Montgomery County, Maryland, and Houston, Texas) were less likely to respond.

The p-values for the entire model displayed at the bottom of exhibit F.2 show that the differences in sample member characteristics between the survey respondents and the survey non-respondents are statistically significant. Nonetheless, the R-squared value (a summary indicator of the predictive power of the effects) of 0.049 is low, which suggests that sample member characteristics have a very small effect on the likelihood of responding to the 36-month survey. Furthermore, results from this test show that membership in the FSS

group did not predict whether someone would complete the survey. This finding implies that the survey results are unbiased.

Appendix exhibits F.3 (for households) and F.4 (for study participants) display demographic characteristics of survey respondents and non-respondents. For these comparisons, differences in proportions of 5 percentage points or more (if statistically significant) and differences in means of 0.3 of a standard deviation or more (if statistically significant) are sufficiently large to be of concern. Across the two exhibits, only small differences were found, except for country of origin (survey respondents include a larger proportion of U.S.-born study participants) and educational attainment (survey respondents include a larger proportion of study participants with a 2-year col ege degree or higher). These results lend support for generalizing survey findings to the larger impact sample.

Comparison of the Research Groups in the Survey Respondent

Sample

Random assignment designs minimize the possibility of potential biases in the results. Although the response rates were similarly high in both research groups, there is still the possibility that different types of sample members within each research group responded to the survey. If so, the impact estimates for the respondent sample may be biased.

Appendix F shows the results of two tests for variation between research groups within the survey respondent sample. First, these differences are estimated in a logistic regression framework, in which the likelihood of being a member of the FSS group is regressed on a range of baseline characteristics (shown in exhibit F.5). The p-values for the entire model displayed at the bottom of exhibit F.5 show that the differences in sample member characteristics between research groups are not statistically significant. Furthermore, the R-squared value (a summary indicator of the predictive power of the effects) of 0.029 is very low, which suggests that sample member characteristics have a very small effect on the likelihood of being an FSS or control group member.

Similarly, a comparison of baseline characteristics of survey respondents shows only scattered and smal differences between the research groups (see exhibits F.6 and F.7). These findings strongly imply that the survey results are unbiased.

124 Study participants with no educational credential are the reference group for comparisons by highest credential earned. The positive coefficients for three educational attainment groups, shown in Exhibit F.2, imply that response rates were lowest for the no-credential group.

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 213

Appendix F. Family Self-Sufficiency 36-Month Survey Response Analysis Consistency of Impacts

Using quarterly wage data from the National Directory of New Hires, this section continues the discussion of whether the survey respondents’ impacts can be generalized to the report sample. Exhibit F.8 shows 3-year impacts on employment and earnings outcomes among survey respondents and among the larger impact sample. As exhibit F.8 shows, impact estimates are similar for both samples, including mean values for each research group, the magnitude of research group differences, and the absence of statistical significance for any of the comparisons. These results further support generalizing the findings from survey responses to the entire impact sample.

Appendix Exhibit F.1. Family Self-Sufficiency 36-Month Survey Samples and Response Rates Outcome

FSS Group

Control Group

Total

Impact sample

1,285

1,271

2,556

Fielded sample

1,265

1,253

2,518

Survey respondents

988

948

1,936

Survey response rate (% of Fielded)

78.1

75.7

76.9

FSS = Family Self-Sufficiency.

SOURCES: Baseline data and FSS 36-Month Survey responses.

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 214

Appendix F. Family Self-Sufficiency 36-Month Survey Response Analysis Appendix Exhibit F.2. Estimated Regression Coefficients of the Probability of Being a Respondent to the Family Self-Sufficiency 36-Month Survey, Family Self-Sufficiency Impact Sample

Parameter

Variable

Estimate

p-Value

Intercept

0.339

0.425

Assigned to FSS Group (impact)

0.108

0.273

Sample member characteristics

Female

0.658

0.000

Age 18–34

- 0.476

0.002

Age 35–44

-0.275

0.049

Married or cohabitating

0.180

0.355

Black

0.224

0.084

1 child

-0.196

0.207

2 children

-0.043

0.795

3 or more children

-0.007

0.967

Has a child age 5 or younger

0.120

0.341

Education

High school diploma or GED

0.396

0.008

Some col ege

0.325

0.021

2-year col ege degree or higher

0.833

<.0001

Has trade license or training certificate

-0.097

0.347

Public assistance

Received SNAP/food stamps

0.229

0.082

Received SSI or SSDI

-0.001

0.995

Received TANF

-0.163

0.308

Housing assistance

Received Housing Choice Voucher less than 4 years

0.101

0.405

Received Housing Choice Voucher 4–7 years

0.107

0.416

Hardship and barriers to employment

Has any barrier to employment

0.020

0.854

Reported 1 hardship in the year before random assignment

0.059

0.670

Reported 2 hardships in the year before random assignment

0.021

0.889

Reported 3 or more hardships in year before random assignment

-0.030

0.823

(continued)

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 215

Appendix F. Family Self-Sufficiency 36-Month Survey Response Analysis Appendix Exhibit F.2. Estimated Regression Coefficients of the Probability of Being a Respondent to the Family Self-Sufficiency 36-Month Survey, Family Self-Sufficiency Impact Sample continued

Parameter

Variable

Estimate

p-Value

Employment

Currently employed

-0.222

0.198

Currently employed ful -time

-0.036

0.814

Employed 1–6 months in the year before random assignment

0.170

0.305

Employed 7–11 months in the year before random assignment

-0.175

0.362

Employed 12 months in the year before random assignment

-0.044

0.828

Head-of-household earnings

$1–$5,000

-0.003

0.991

$5,001–$7,500

0.194

0.486

$7,501–$10,000

0.120

0.671

$10,001–$12,500

0.335

0.238

$12,501–$15,000

0.101

0.719

$15,001–$17,500

-0.100

0.694

$17,501–$20,000

0.801

0.008

$20,001–$22,500

0.279

0.335

$22,501–$25,000

0.251

0.385

$25,001–$30,000

0.029

0.907

$30,001–$35,000

-0.009

0.977

$35,001–$70,000

-0.278

0.345

Enrollment

Randomly assigned in quarter 4, 2013

-0.408

0.164

Randomly assigned in quarter 1, 2014

0.332

0.025

Randomly assigned in quarter 2, 2014

0.070

0.592

Housing Authority of the City of Alameda

-0.216

0.520

Orange County Housing Authority

-0.224

0.519

Housing Authority of the City of Riverside

-0.452

0.163

Housing Authority of the City of Deerfield Beach

0.146

0.789

Housing Authority of the City of Ft. Lauderdale

-0.417

0.275

Baltimore County Housing Office

-0.562

0.129

Housing Opportunities Commission of Montgomery County

-0.840

0.018

(continued)

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 216

Appendix F. Family Self-Sufficiency 36-Month Survey Response Analysis Appendix Exhibit F.2. Estimated Regression Coefficients of the Probability of Being a Respondent to the Family Self-Sufficiency 36-Month Survey, Family Self-Sufficiency Impact Sample continued

Parameter

Variable

Estimate

p-Value

Housing Authority of Kansas City

-0.176

0.642

Jersey City Housing Authority

-0.329

0.429

Akron Metropolitan Housing Authority

0.549

0.308

Columbus Metropolitan Housing Authority

-0.656

0.124

Lucas Metropolitan Housing Authority

-0.286

0.540

Youngstown Metropolitan Housing Authority

0.571

0.291

Dal as Housing Authority

-0.249

0.487

Fort Worth Housing Authority

-0.350

0.407

Houston Housing Authority

-0.624

0.068

Tarrant County Housing Assistance Office

-0.478

0.205

Enrol ed in FSS for help with employment

0.089

0.451

Had checking or savings account

0.223

0.059

Had savings between $1–$500

0.210

0.081

Had savings greater than $500

0.105

0.567

Debt

$1–$1,000

0.059

0.748

$1,001–$5,000

0.129

0.439

$5,001–$10,000

-0.002

0.993

$10,001–$20,000

0.102

0.544

$20,001 or greater

-0.142

0.407

Likelihood ratio

125.3

<.0001

Wald statistic

113.3

0.001

R-squared (.0486)

Sample size

2,518

FSS = Family Self-Sufficiency. GED = General Educational Development. SSI-SSDI = Supplemental Security Income-Social Security Disability Insurance. SNAP = Supplemental Nutrition Assistance Program. TANF = Temporary Assistance for Needy Families.

SOURCES: Baseline data, U.S. Department of Housing and Urban Development Inventory Management System (IMS)/PIH

Information Center (PIC) data, and responses to the FSS 36-Month Survey.

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 217

Appendix F. Family Self-Sufficiency 36-Month Survey Response Analysis Appendix Exhibit F.3. Baseline Characteristics of Households in the Family Self-Sufficiency Impact Sample by Family Self-Sufficiency 36-Month Survey Respondent Status

Non-

Characteristic

Respondent Respondent

Total

Average number of household membersa

3.2

3.2

3.2

Average number of adults in householda

1.4

1.5

1.5

Households with more than one adult (%)

33.4

34.4

33.7

Average number of children in household

1.8

1.7

1.8

Number of children in household (%)

0

24.0

23.2

23.8

1

21.9

24.7

22.6

2

24.7

24.8

24.8

3 or more

29.3

27.3

28.8

For households with children, age of youngest child (%)

0–2 years

21.0

20.1

20.8

3–5 years

20.9

18.8

20.4

6–12 years

41.8

40.0

41.3

13–17 years

16.3

21.1

17.5

Primary language spoken at home is English (%)

93.1

89.3

92.2***

Receives TANF (%)

15.4

17.0

15.8

Receives food stamps/SNAP (%)

70.6

66.3

69.6**

Length of time receiving Section 8 Housing Choice Voucher (%)

Less than 1 year

5.5

3.4

5.0

1–3 years

27.7

27.4

27.6

4–6 years

21.2

23.1

21.6

7–9 years

15.4

14.6

15.2

10 years or more

30.3

31.5

30.6

Total household income (%)

$0

4.5

4.6

4.5

$1–$4,999

16.9

17.4

17.0

$5,000–$9,999

18.6

19.1

18.7

$10,000–$14,999

18.3

15.5

17.6

$15,000–$19,999

14.7

12.9

14.3

$20,000–$24,999

11.2

11.1

11.2

$25,000–$29,999

8.1

8.3

8.2

$30,000 or more

7.7

11.2

8.5

(continued)

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 218

Appendix F. Family Self-Sufficiency 36-Month Survey Response Analysis Appendix Exhibit F.3. Baseline Characteristics of Households in the Family Self-Sufficiency Impact Sample by Family Self-Sufficiency 36-Month Survey Respondent Status continued Non-Characteristic

Respondent Respondent

Total

Payment for rent and utilities (%)

$0

2.0

1.6

1.9

$1–$99

5.2

6.3

5.5

$100–$199

9.8

8.6

9.5

$200–$299

12.0

8.9

11.2

$300–$399

12.8

13.8

13.0

$400–$499

11.0

9.1

10.5

$500–$599

11.0

9.9

10.8

$600–$699

7.4

9.1

7.8

$700–$799

7.2

7.5

7.3

$800–$899

5.2

6.5

5.5

$900–$999

3.0

4.1

3.3

$1,000 or more

13.2

14.6

13.6

During the past 12 months, household experienced at least

one financial hardship (%)

59.2

58.5

59.0

Not able to buy prescription drug

13.7

12.1

13.3

Not able to buy food

29.1

28.2

28.9

Not able to pay telephone bil

28.8

26.5

28.2

Not able to pay rent

17.9

20.6

18.5

Not able to pay utility bil

42.8

45.1

43.4

Sample size

1,936

620

2,556

FSS = Family Self-Sufficiency. SNAP = Supplemental Nutrition Assistance Program. TANF = Temporary Assistance for Needy Families.

aMaximum response option for number of adults in a household is four.

NOTES: The FSS impact sample includes Housing Choice Voucher households that were randomly assigned between October 18, 2013, and December 22, 2014, and with a head of household age 18 to 61 at the time of random assignment. Sample sizes for specific measures may vary because of missing values. A chi-square test for categorical variables and a t-test for continuous variables were run to determine whether there is a difference in the distribution of the characteristics by respondent status. Statistical significance levels are indicated as: * = 10 percent; ** = 5 percent; and *** = 1 percent. Rounding may cause slight discrepancies in calculating sums and differences. Detail may sum to more than 100.0 percent for questions that allow more than one response.

SOURCES: MDRC calculations from Baseline Information Form data and responses to the FSS 36-Month Survey.

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 219

Appendix F. Family Self-Sufficiency 36-Month Survey Response Analysis Appendix Exhibit F.4. Baseline Characteristics of Heads of Households in the Family Self-Sufficiency Impact Sample by Family Self-Sufficiency 36-Month Survey Respondent Status

Non-

Characteristic

Respondent Respondent

Total

Sample member characteristics

Female (%)

91.6

87.3

90.6***

Age (%)

19–24 years

2.3

1.9

2.2

25–34 years

33.5

35.0

33.9

35–44 years

35.3

36.3

35.6

45–59 years

27.6

26.1

27.3

60–61 years

1.2

0.6

1.1

Average age (years)

39

39

39

Marital status (%)

*

Married, living with spouse

7.5

8.4

7.7

Married, not living with spouse

7.6

4.5

6.8

Cohabitating

1.3

1.6

1.4

Single, widowed, or divorced

83.5

85.5

84.0

Citizenship status (%)

***

U.S.-born

89.2

83.1

87.7

Naturalized

7.2

10.9

8.1

Non-citizen

3.6

6.0

4.2

Race/ethnicity (%)a

***

Black, non-Hispanic/Latino

74.3

70.2

73.3

Hispanic/Latino

15.5

16.8

15.8

White, non-Hispanic/Latino

6.7

6.6

6.7

Native American

0.3

0.3

0.3

Asian

1.3

4.2

2.0

Other

0.2

0.8

0.4

Multi-racial

1.7

1.1

1.6

Education

Highest degree or diploma earned (%)

***

GED certificate

3.2

2.6

3.0

High school diploma

10.5

11.0

10.6

Some col ege or received technical/trade license

54.4

57.0

55.0

(continued)

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 220

Appendix F. Family Self-Sufficiency 36-Month Survey Response Analysis Appendix Exhibit F.4. Baseline Characteristics of Heads of Households in the Family Self-Sufficiency Impact Sample by Family Self-Sufficiency 36-Month Survey Respondent Status continued

Non-

Characteristic

Respondent Respondent

Total

Associate’s or 2-year col ege degree

11.9

7.3

10.8

4-year col ege or graduate degree

7.0

5.0

6.5

None of the above

13.0

17.0

14.0

Has trade license or training certificate (%)

46.8

47.6

47.0

Employment status

Currently employed (%)

55.5

58.6

56.2

Regular job

47.5

51.3

48.4

Self-employed

4.2

4.4

4.2

Temporary or seasonal job

3.7

2.9

3.5

Currently working 35 hours or more per week (%)

29.6

33.4

30.5*

Average hours worked per week

18.0

19.4

18.3*

Average weekly earnings ($)

209

227

213

Barriers to employment

Has any problem that limits work (%)

41.3

40.6

41.2

Physical health

19.1

17.9

18.8

Emotional or mental health

7.6

7.8

7.6

Childcare access or cost

17.9

17.7

17.8

Need to care for disabled household member

7.4

7.0

7.3

Previously convicted of a felony

6.1

7.0

6.3

Limited English-speaking ability (%)

3.2

5.8

3.8***

Does not have access to transportation for employment (%)

No public transportation access

17.6

18.5

17.8

No automobile access

17.5

20.5

18.2*

FSS program

Heard of escrow before random assignment (%)

44.6

42.2

44.0

Interest in FSS services related to (%)

Job-related services

70.7

69.8

70.5

Social services

32.4

32.5

32.4

Financial services

95.1

96.4

95.5

Sample size

1,936

620

2,556

(continued)

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 221

Appendix F. Family Self-Sufficiency 36-Month Survey Response Analysis Appendix Exhibit F.4. Baseline Characteristics of Heads of Households in the Family Self-Sufficiency Impact Sample by Family Self-Sufficiency 36-Month Survey Respondent Status continued

FSS = Family Self-Sufficiency. GED = General Educational Development.

aChi-square test may not be valid due to small cell sizes.

NOTES: The FSS impact sample includes Housing Choice Voucher households that were randomly assigned between October 18, 2013, and December 22, 2014, and with a head of household age 18 to 61 at the time of random assignment. Sample sizes for specific measures may vary because of missing values. A chi-square test for categorical variables and a t-test for continuous variables were run to determine whether there is a difference in the distribution of the characteristics by respondent status. Statistical significance levels are indicated as: * = 10 percent; ** = 5 percent; and *** = 1 percent. Rounding may cause slight discrepancies in calculating sums and differences. Detail may sum to more than 100.0 percent for questions that allow more than one response.

SOURCE: MDRC calculations from Baseline Information Form data and responses to the FSS 36-Month Survey.

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 222

Appendix F. Family Self-Sufficiency 36-Month Survey Response Analysis Appendix Exhibit F.5. Estimated Regression Coefficients of the Probability of Being a Family Self-Sufficiency Group Member, Family Self-Sufficiency 36-Month Survey Respondents Parameter

Variable

Estimate

p-Value

Intercept

-0.452

0.299

Sample member characteristics

Female

0.434

0.018

Age 18–34

0.100

0.496

Age 35–44

0.087

0.513

Married or cohabitating

0.100

0.581

Black

-0.088

0.488

1 child

0.080

0.593

2 children

-0.141

0.371

3 or more children

-0.059

0.727

Has a child age 5 or younger

-0.066

0.582

Education

High school diploma or GED

0.071

0.630

Some col ege

0.003

0.985

2-year col ege degree or higher

0.302

0.075

Has trade license or training certificate

0.008

0.933

Public assistance

Received SNAP/food stamps

0.061

0.630

Received SSI or SSDI

-0.526

0.001

Received TANF

-0.203

0.198

Housing assistance

Received Housing Choice Voucher less than 4 years

-0.014

0.902

Received Housing Choice Voucher 4–7 years

0.054

0.667

Hardship and barriers to employment

Has any barrier to employment

0.037

0.728

Reported 1 hardship in the year before random assignment

-0.093

0.472

Reported 2 hardships in the year before random assignment

-0.131

0.359

Reported 3 or more hardships in year before random assignment

-0.021

0.872

(continued)

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 223

Appendix F. Family Self-Sufficiency 36-Month Survey Response Analysis Appendix Exhibit F.5. Estimated Regression Coefficients of the Probability of Being a Family Self-Sufficiency Group Member, Family Self-Sufficiency 36-Month Survey Respondents continued

Parameter

Variable

Estimate

p-Value

Employment

Currently employed

0.119

0.466

Currently employed ful -time

-0.010

0.948

Employed 1–6 months in the year before random assignment

-0.232

0.128

Employed 7–11 months in the year before random assignment

-0.589

0.001

Employed 12 months in the year before random assignment

-0.405

0.032

Head-of-household earnings

$1–$5,000

0.487

0.050

$5,001–$7,500

0.241

0.353

$7,501–$10,000

-0.102

0.700

$10,001–$12,500

-0.094

0.712

$12,501–$15,000

-0.099

0.709

$15,001–$17,500

-0.019

0.940

$17,501–$20,000

-0.050

0.838

$20,001–$22,500

-0.293

0.281

$22,501–$25,000

-0.209

0.447

$25,001–$30,000

-0.160

0.518

$30,001–$35,000

-0.422

0.156

$35,001–$70,000

-0.056

0.858

Enrollment

Randomly assigned in quarter 4, 2013

0.104

0.734

Randomly assigned in quarter 1, 2014

-0.001

0.997

Randomly assigned in quarter 2, 2014

-0.068

0.586

Housing Authority of the City of Alameda

0.202

0.549

Orange County Housing Authority

0.233

0.508

Housing Authority of the City of Riverside

0.248

0.457

Housing Authority of the City of Deerfield Beach

0.269

0.553

Housing Authority of the City of Ft. Lauderdale

0.337

0.368

Baltimore County Housing Office

0.483

0.202

Housing Opportunities Commission of Montgomery County

0.288

0.434

(continued)

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 224

Appendix F. Family Self-Sufficiency 36-Month Survey Response Analysis Appendix Exhibit F.5. Estimated Regression Coefficients of the Probability of Being a Family Self-Sufficiency Group Member, Family Self-Sufficiency 36-Month Survey Respondents continued

Parameter

Variable

Estimate

p-Value

Housing Authority of Kansas City

0.233

0.531

Jersey City Housing Authority

0.127

0.758

Akron Metropolitan Housing Authority

0.138

0.746

Columbus Metropolitan Housing Authority

0.191

0.657

Lucas Metropolitan Housing Authority

0.273

0.553

Youngstown Metropolitan Housing Authority

0.116

0.787

Dal as Housing Authority

0.365

0.307

Fort Worth Housing Authority

0.400

0.334

Houston Housing Authority

0.213

0.543

Tarrant County Housing Assistance Office

0.321

0.401

Enrol ed in FSS for help with employment

-0.002

0.985

Had checking or savings account

-0.011

0.920

Had savings between $1-$500

0.158

0.162

Had savings greater than $500

0.174

0.325

Debt

$1–$1,000

0.199

0.262

$1,001–$5,000

0.039

0.808

$5,001–$10,000

-0.061

0.722

$10,001–$20,000

-0.064

0.689

$20,001 or greater

-0.088

0.596

Likelihood ratio

57.0

0.869

Wald statistic

54.4

0.915

R-squared (.0290)

Sample size

1,936

FSS = Family Self-Sufficiency. GED = General Educational Development. SNAP = Supplemental Nutrition Assistance Program. SSI-SSDI = Supplemental Security Income-Social Security Disability Insurance. TANF = Temporary Assistance for Needy Families.

NOTES: The FSS 36-Month Survey respondent sample includes Housing Choice Voucher heads of household who were randomly assigned between October 18, 2013, and December 22, 2014, were age 18 to 61 at the time of random assignment, and responded to the FSS 36-Month Survey.

SOURCES: Baseline data, U.S. Department of Housing and Urban Development Inventory Management System (IMS)/PIH

Information Center (PIC) data, and responses to the FSS 36-Month Survey.

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 225

Appendix F. Family Self-Sufficiency 36-Month Survey Response Analysis Appendix Exhibit F.6. Baseline Characteristics of Households by Research Group, Family Self-Sufficiency 36-Month Survey Respondent Sample

Control

Characteristic

FSS Group

Group

Total

Average number of household membersa

3.2

3.2

3.2

Average number of adults in householda

1.5

1.4

1.4

Households with more than one adult (%)

34.5

32.3

33.4

Average number of children in household

1.8

1.8

1.8

Number of children in household (%)

0

23.7

24.4

24.0

1

23.1

20.7

21.9

2

24.0

25.5

24.7

3 or more

29.3

29.4

29.3

For households with children, age of youngest child (%)

0–2 years

21.5

20.5

21.0

3–5 years

19.9

21.9

20.9

6–12 years

42.9

40.6

41.8

13–17 years

15.8

16.9

16.3

Primary language spoken at home is English (%)

93.3

92.9

93.1

Receives TANF (%)

15.4

15.4

15.4

Receives food stamps/SNAP (%)

71.6

69.6

70.6

Length of time receiving Section 8 Housing Choice Voucher (%)

Less than 1 year

5.4

5.5

5.5

1–3 years

28.0

27.3

27.7

4–6 years

20.3

22.0

21.2

7–9 years

15.6

15.2

15.4

10 years or more

30.6

30.0

30.3

Total household income (%)

$0

4.8

4.3

4.5

$1–$4,999

17.7

16.1

16.9

$5,000–$9,999

18.3

18.9

18.6

$10,000–$14,999

18.8

17.8

18.3

$15,000–$19,999

14.5

14.9

14.7

$20,000–$24,999

10.5

11.8

11.2

$25,000–$29,999

7.6

8.6

8.1

(continued)

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 226

Appendix F. Family Self-Sufficiency 36-Month Survey Response Analysis Appendix Exhibit F.6. Baseline Characteristics of Households by Research Group, Family Self-Sufficiency 36-Month Survey Respondent Sample continued

Control

Characteristic

FSS Group

Group

Total

$30,000 or more

7.9

7.5

7.7

Payment for rent and utilities (%)

$0

2.2

1.8

2.0

$1–$99

5.6

4.8

5.2

$100–$199

10.4

9.2

9.8

$200–$299

12.2

11.8

12.0

$300–$399

12.8

12.7

12.8

$400–$499

11.5

10.5

11.0

$500–$599

10.8

11.3

11.0

$600–$699

7.0

8.0

7.4

$700–$799

6.6

7.9

7.2

$800–$899

4.8

5.6

5.2

$900–$999

2.9

3.2

3.0

$1,000 or more

13.2

13.3

13.2

During the past 12 months, household experienced at least

one financial hardship (%)

58.1

60.3

59.2

Not able to buy prescription drug

13.7

13.7

13.7

Not able to buy food

26.3

32.1

29.1***

Not able to pay telephone bil

28.7

28.8

28.8

Not able to pay rent

17.5

18.3

17.9

Not able to pay utility bil

43.7

41.9

42.8

Sample size

988

948

1,936

FSS = Family Self-Sufficiency. SNAP = Supplemental Nutrition Assistance Program. TANF = Temporary Assistance for Needy Families.

aMaximum response option for number of adults in a household is four.

NOTES: The FSS 36-Month Survey respondent sample includes Housing Choice Voucher heads of household who were randomly assigned between October 18, 2013, and December 22, 2014, were age 18 to 61 at the time of random assignment, and responded to the FSS 36-Month Survey. Sample sizes for specific measures may vary because of missing values. A chi-square test for categorical variables and a t-test for continuous variables were run to determine whether there is a difference in the distribution of the characteristics by research group. Statistical significance levels are indicated as: * = 10 percent; ** = 5 percent; and *** = 1 percent. Rounding may cause slight discrepancies in calculating sums and differences. Detail may sum to more than 100.0 percent for questions that allow more than one response.

SOURCES: MDRC calculations from Baseline Information Form data and responses to the FSS 36-Month Survey.

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 227

Appendix F. Family Self-Sufficiency 36-Month Survey Response Analysis Appendix Exhibit F.7. Baseline Characteristics of Heads of Households by Research Group, Family Self-Sufficiency 36-Month Survey Respondent Sample

Control

Characteristic

FSS Group

Group

Total

Sample member characteristics

Female (%)

92.9

90.3

91.6**

Age (%)

19–24 years

2.0

2.5

2.3

25–34 years

33.9

33.1

33.5

35–44 years

35.8

34.8

35.3

45–59 years

27.0

28.3

27.6

60–61 years

1.2

1.3

1.2

Average age (years)

39

39

39

Marital status (%)

Married, living with spouse

7.8

7.3

7.5

Married, not living with spouse

8.0

7.2

7.6

Cohabitating

1.2

1.5

1.3

Single, widowed, or divorced

83.0

84.1

83.5

Citizenship status (%)

U.S.-born

88.9

89.5

89.2

Naturalized

7.3

7.0

7.2

Non-citizen

3.8

3.5

3.6

Race/ethnicity (%)a

Black, non-Hispanic/Latino

73.7

74.9

74.3

Hispanic/Latino

15.2

15.9

15.5

White, non-Hispanic/Latino

7.2

6.1

6.7

Native American

0.3

0.3

0.3

Asian

1.5

1.1

1.3

Other

0.1

0.3

0.2

Multi-racial

2.0

1.4

1.7

(continued)

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 228

Appendix F. Family Self-Sufficiency 36-Month Survey Response Analysis Appendix Exhibit F.7. Baseline Characteristics of Heads of Households by Research Group, Family Self-Sufficiency 36-Month Survey Respondent Sample continued

Control

Characteristic

FSS Group

Group

Total

Education

Highest degree or diploma earned (%)

GED certificate

3.6

2.8

3.2

High school diploma

10.3

10.8

10.5

Some col ege or received technical/trade license

52.8

56.1

54.4

Associate’s or 2-year col ege degree

12.6

11.1

11.9

4-year col ege or graduate degree

8.1

5.8

7.0

None of the above

12.7

13.3

13.0

Has trade license or training certificate (%)

46.7

47.0

46.8

Employment status

Currently employed (%)

54.9

56.0

55.5

Regular job

47.2

47.9

47.5**

Self-employed

3.3

5.2

4.2**

Temporary or seasonal job

4.5

2.9

3.7**

Currently working 35 hours or more per week (%)

28.9

30.4

29.6

Average hours worked per week

18.0

19.4

18.3*

Average weekly earnings ($)

209

227

213

Barriers to employment

Has any problem that limits work (%)

41.6

41.0

41.3

Physical health

17.7

20.7

19.1*

Emotional or mental health

7.3

7.9

7.6

Childcare access or cost

19.1

16.7

17.9

Need to care for disabled household member

7.7

7.1

7.4

Previously convicted of a felony

6.3

5.8

6.1

Limited English-speaking ability (%)

2.7

3.7

3.2

Does not have access to transportation for employment (%)

No public transportation access

16.4

18.7

17.6

No automobile access

17.6

17.3

17.5

(continued)

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 229

Appendix F. Family Self-Sufficiency 36-Month Survey Response Analysis Appendix Exhibit F.7. Baseline Characteristics of Heads of Households by Research Group, Family Self-Sufficiency 36-Month Survey Respondent Sample continued

Control

Characteristic

FSS Group

Group

Total

FSS program

Heard of escrow before random assignment (%)

45.1

44.1

44.6

Interest in FSS services related to (%)

Job-related services

70.9

70.6

70.7

Social services

32.1

32.7

32.4

Financial services

95.2

95.0

95.1

Sample size

988

948

1,936

FSS = Family Self-Sufficiency. GED = General Educational Development.

aChi-square test may not be valid due to small cell sizes.

NOTES: The FSS 36-Month Survey respondent sample includes Housing Choice Voucher heads of household who were randomly assigned between October 18, 2013, and December 22, 2014, were age 18 to 61 at the time of random assignment, and responded to the FSS 36-Month Survey. Sample sizes for specific measures may vary because of missing values. A chi-square test for categorical variables and a t-test for continuous variables were run to determine whether there is a difference in the distribution of the characteristics by research group. Statistical significance levels are indicated as: * = 10 percent; ** = 5 percent; and *** = 1 percent. Rounding may cause slight discrepancies in calculating sums and differences. Detail may sum to more than 100.0 percent for questions that allow more than one response.

SOURCES: MDRC calculations from Baseline Information Form data and responses to the FSS 36–Month Survey.

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 230

Appendix F. Family Self-Sufficiency 36-Month Survey Response Analysis Appendix Exhibit F.8. Comparison of Impacts on Employment and Earnings in Years 1 to 3, Family Self-Sufficiency Impact and 36-Month Survey Respondent Samples

FSS

Control Difference

Outcome

Group

Group

(Impact)

p-Value

Ever employed in years 1 to 3 (%)

Impact sample

83.4

81.7

1.7

0.170

Survey respondents

83.7

81.4

2.3

0.103

Average quarterly employment rate (%)

Impact sample

63.5

63.0

0.5

0.679

Survey respondents

64.2

62.9

1.3

0.309

Total earnings ($)

Impact sample

41,650

41,487

163

0.884

Survey respondents

41,838

40,723

1,115

0.376

Average annual earnings greater than $25,000 (%)

Impact sample

20.3

20.1

0.2

0.903

Survey respondents

19.9

19.6

0.2

0.877

Sample sizes

Impact Sample (total = 2,548)

1,282

1,266

36-Month Survey Respondents (total = 1,930)

986

944

FSS = Family Self-Sufficiency.

NOTES: The FSS impact sample includes Housing Choice Voucher households that were randomly assigned between October 18, 2013, and December 22, 2014, and with a head of household age 18 to 61 at the time of random assignment. The FSS 36-Month Survey respondent sample includes Housing Choice Voucher heads of household who were randomly assigned between October 18, 2013, and December 22, 2014, were age 18 to 61 at the time of random assignment, and responded to the FSS 36-Month Survey.

Average quarterly employment rate is calculated as: total quarters with employment divided by total quarters of follow-up, expressed as a percentage. Estimates were regression-adjusted using ordinary least squares, controlling for pre-random assignment characteristics of sample members. Rounding may cause slight discrepancies in calculating differences. A two-tailed t-test was applied to differences between research groups. The p-value indicates the likelihood that the difference between the FSS group and control group arose by chance. Statistical significance levels are indicated as: * = 10 percent; ** = 5 percent; and *** = 1 percent.

SOURCES: MDRC calculations using baseline data, quarterly wage data from the National Directory of New Hires, and responses to the FSS 36-Month Survey.

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 231

References

References

Abbi, Sarika. 2012. A Need for Product Innovation to Help LMI Consumers Manage Financial Emergencies. Al ston, MA: Doorways to Dreams Fund.

Abraham, Katherine, John C. Haltiwanger, Kristin Sandusky, and James Spletzer. 2009. Exploring Differences in Employment Between Household and Establishment Data.NBER Working Paper No. 14805.

Cambridge, MA: National Bureau of Economic Research.

Azurdia, Gilda, and Stephen Freedman. 2016. Encouraging Nonretirement Savings at Tax Time: Impact Findings from the SaveUSA Evaluation. New York: MDRC.

Bloom, Howard S., Carolyn J. Hil , and James Riccio. 2001. Modeling the Performance of Welfare-to-Work Programs: The Effects of Program Management and Services, Economic Environment, and Client Characteristics. MDRC Working Papers on Research Methodology. New York: MDRC.

Bloom, Howard S., Stephen W. Raudenbush, Michael J. Weiss, and Kristen Porter. 2017. “Using Multisite Experiments to Study Cross-Site Variation in Treatment Effects: A Hybrid Approach With Fixed Intercepts and a Random Treatment Coefficient,” Journal of Research on Educational Effectiveness 10 (4): 817–842.

Center on Budget and Policy Priorities (CBPP). 2017. Policy Basics: The Housing Choice Voucher Program.

www.cbpp.org.

Chetty, Raj, and Nathaniel Hendren. 2017a. The Impacts of Neighborhoods on Intergenerational Mobility I: Childhood Exposure Effects. NBER Working Paper No. 23001. Cambridge, MA: National Bureau of Economic Research. www.equality-of-opportunity.org/assets/documents/movers_paper1.pdf. (Forthcoming in the Quarterly Journal of Economics. )

Chetty, Raj, and Nathaniel Hendren. 2017b. The Impacts of Neighborhoods on Intergenerational Mobility II: County-Level Estimates. NBER Working Paper No. 23002. www.equality-of-opportunity.org/assets/

documents/movers_paper2.pdf. (Forthcoming in the Quarterly Journal of Economics. ) Col ins, J. Michael, and Lean Gjertson. 2013. Emergency Savings for Low-Income Consumers. Madison, WI: Institute for Research on Poverty.

Cramer, Reid, Justin King, and El iot Schreur. 2015. Flexible Savings: The Missing Foundation for Financial Security and Economic Mobility. Washington, DC: New America Foundation.

Cramer, Reid, and Jeff Lubel . 2005. The Family Self-Sufficiency Program: A Promising, Low-Cost Vehicle to Promote Savings and Asset Building for Recipients of Federal Housing Assistance. Washington, DC: New America Foundation.

Geyer, Judy, Lesley Freiman, Jeffrey Lubel , and Micah Vil arreal. 2017. Evaluation of the Compass Family Self-Sufficiency (FSS) Programs Administered in Partnership With Public Housing Agencies in Lynn and Cambridge, Massachuset s. Cambridge, MA: Abt Associates.

Greenberg, David M., Aurelia De La Rosa Aceves, M. Victoria Quiroz-Becerra, David H. Greenberg, and Ari Oppenheim. 2015. The Second Generation of the Jobs-Plus Program: Implementation Lessons From San Antonio and the Bronx. New York: MDRC.

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 232

References

Guare, Dick, and Peg Dawson. 2016. A Manual for Executive Skil s Coaching With Adults Affected by Conditions of Poverty and Stress. Fort Col ins, CO: TuaPath.

Hannagan, Anthony, and Jonathan Morduch. 2015. “Income Gains and Month-to-Month Income Volatility: Household Evidence from the U.S. Financial Diaries.” NYU Wagner Research Paper 2659883.

Jacob, Brian, and Jens Ludwig. 2008. Improving Educational Outcomes for Poor Children. Cambridge, MA: National Bureau of Economic Research.

Lopez-Fernandini, Alejandra. 2012. Unrestricted Savings: Their Role in Household Economic Security and the Case for Policy Action. Washington, DC: New America Foundation.

McKernan, Signe-Mary, Caroline Ratcliffe, and Katie Vinopal. 2009. Do Assets Help Families Cope With Adverse Events? Washington, DC: Urban Institute.

Mil s, Greg, Daniel Gubits, Larry Orr, David Long, Judie Feins, Bulbul Kaul, and Michel e Wood. 2006.

Effects of Housing Vouchers on Welfare Families. Washington, DC: U.S. Department of Housing and Urban Development, Office of Policy Development and Research.

Nuñez, Stephen, Nandita Verma, and Edith Yang. 2015. Building Self-Sufficiency for Housing Voucher Recipients: Interim Findings From the Work Rewards Demonstration in New York City. New York: MDRC.

Pew Charitable Trusts. 2015. The Precarious State of Family Balance Sheets. Washington, DC: Pew Charitable Trusts. https://www.pewtrusts.org/en/research-and-analysis/reports/2015/01/the-precarious-state-of-family-

balance-sheets

Popkin, Susan J., Larry F. Buron, Diane K. Levy, and Mary K. Cunningham. 2000. “The Gautreaux Legacy: What Might Mixed-Income and Dispersal Strategies Mean for the Poorest Public Housing Tenants?” Housing Policy Debate 11 (4): 911–942.

Popkin, Susan J., Mary K. Cunningham, and Martha Burt. 2005. “Public Housing Transformation and the Hard to House,” Housing Policy Debate 16 (1): 1–24.

Popkin, Susan J., Brett Theodos, Liza Getsinger, and Joe Paril a. 2010. A New Model for Integrating Housing and Services. Washington, DC: Urban Institute.

Riccio, James A., Victoria Deitch, and Nandita Verma. 2017. Reducing Work Disincentives in the Housing Choice Voucher Program: Rent Reform Demonstration Baseline Report. New York: MDRC.

Rohe, Wil iam M. 1995. “Converting Public Housing to Cooperatives: The Experience of Three Developments,” Housing Policy Debate 6 (2): 439–479.

Rohe, Wil iam M., and Rachel Garshick Kleit. 1999. “Housing, Welfare Reform, and Self-Sufficiency: An Assessment of the Family Self-Sufficiency Program,” Housing Policy Debate 10 (2): 333–369.

Sard, Barbara, and Margy Wal er. 2002. Housing Strategies to Strengthen Welfare Policy and Support Working Families. Washington, DC: Brookings Institution, Center on Urban and Metropolitan Policy.

Schwartz, Alex F. 2006. Housing Policy in the United States. New York: Routledge.

Shroder, Mark D. “Housing Subsidies and Work Incentives” (October 15, 2010). Available at SSRN: https://

ssrn.com/abstract=1691112 or http://dx.doi.org/10.2139/ssrn.1691112

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 233

References

Smith, Robert E., Susan J. Popkin, Taz George, and Jennifer Comey. 2015. What Happens to Housing Assistance Leavers? Washington, DC: Cityscape, U.S. Department of Housing and Urban Development, Office of Policy Development and Research.

U.S. Department of Housing and Urban Development (HUD). n.d.a. Family Self-Sufficiency 2020 Summary of Resources. Washington, DC: U.S. Department of Housing and Urban Development. https://www.hud.gov/

sites/dfiles/CFO/documents/2020CJ-FSS.pdf

———. n.d.b. Outreach and Goal Set ing—Module 2.3: Contract of Participation and Individual Training and Services Plan. https://www.hudexchange.info/trainings/fss-program-online-training/2.3-contract---services-plan.html

Accessed 03/16/2020.

———. 2015. Financial Reporting for the Family Self-Sufficiency (FSS) Program: Accounting Brief #23. PIH-REAC: PHA-Finance Accounting Briefs. Washington, DC: U.S. Department of Housing and Urban Development, Office of Public and Indian Housing Real Estate Assessment Center. https://www.hud.gov/

sites/documents/ACCTBRIEF23FSSVF051415.PDF

———. 2016a. Fact Sheet: Family Self-Sufficiency (FSS) Program. Washington, DC: Office of Public Housing and Voucher Programs & Office of Public Housing Investments, U.S. Department of Housing and Urban Development. www.hud.gov/sites/documents/FSSFACTSHEET_FEB2016.PDF.

———. 2016b. Report to Congress on Family Self-Sufficiency Best Practices. Washington, DC: U.S. Department of Housing and Urban Development. www.hud.gov/sites/documents/FY16FSS_BPREPORTCOCHRAN.

PDF.

———. 2016c. Family Self-Sufficiency Program: Funder Report # FR-6000-N-04. Washington, DC: U.S.

Department of Housing and Urban Development.

———. 2017a. Administering an Effective Family Self-Sufficiency Program: A Guidebook Based on Evidence and Promising Practices. Washington, DC: U.S. Department of Housing and Urban Development.

———. 2017b. Family Self-Sufficiency Program: Funder Report. Washington, DC: U.S. Department of Housing and Urban Development. www.hud.gov/sites/documents/FR-6100-N-05_FY17_FSS_NOFA.PDF.

———. 2017c. FSS Escrow Account Credit Worksheet. HUD Form 52652, Family Self-Sufficiency Program.

www.hud.gov/sites/documents/52652.pdf.

———. 2018a. Family Self-Sufficiency Program—Renewal FR-6200-N-04. Washington, DC: U.S. Department of Housing and Urban Development. https://www.hud.gov/sites/dfiles/SPM/documents/FY18-FSS-

Renewals-NOFA.pdf

———. 2018b. Family Self-Sufficiency Performance Measurement System (“Composite Score”). Section 42

U.S.C. 1437u(i). Washington, DC: U.S. Department of Housing and Urban Development. https://www.

federalregister.gov/documents/2018/11/15/2018-24949/family-self-sufficiency-performance-measurement-

system-composite-score.

Verma, Nandita, Stephen Freedman, Betsy L. Tessler, Stephen Nuñez, and Barbara Fink. 2019. Promoting Work and Self-Sufficiency for Housing Voucher Recipients: Early Findings From the Family Self-Sufficiency Program Evaluation. Washington, DC: U.S. Department of Housing and Urban Development, Office of Policy Development and Research.

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 234

References

Verma, Nandita, Betsy Tessler, Cynthia Mil er, James A. Riccio, Zawadi Rucks, and Edith Yang. 2012.

Working Toward Self-Sufficiency: Early Findings From a Program for Housing Voucher Recipients in New York City.

New York: MDRC.

Verma, Nandita, Edith Yang, Stephen Nuñez, and David Long. 2017. Learning From the Work Rewards Demonstration. New York: MDRC.

Work, Engagement, and Well-Being at the Midpoint: Findings From the Family Self-Sufficiency Evaluation 235

U.S. Department of Housing and Urban Development

Office of Policy Development and Research

Washington, DC 20410-6000

July 2021

Document Outline

Acknowledgments

Foreword

Contents

List of Exhibits

Executive Summary

Chapter 1. Introduction

Chapter 2. Program Delivery and Implementation Update

Chapter 3. Participant Engagement

Chapter 4. Escrow Accruals and Graduation

Chapter 5. Impacts on Employment and Earnings

Chapter 6. Impacts on Income, Subsidy Receipt, and Well-Being

Chapter 7. Variation in Program Impacts

Chapter 8. Summary and Looking Forward

Appendix A

Appendix B

Appendix C

Appendix D

Appendix E

Appendix F. Family Self-Sufficiency 36-Month Survey Response Analysis

References